Question

How can we use erasure coding to reduce redundancy in peer-to-peer file system and maintain high availability? Does this method have advantage in case of high disk capacity?

Answer 1

Erasure Coding: Erasure coding is a space-saving mechanism where multiple redundant copies are reduced to a single parity block

In the Erasure coding, Data reliability is a significant issue in large-scale storage systems. Erasure codes provide high data

reliability via data recovery, which however generates a large amount of data transmission in the network. The bandwidth cost

of the data transmission in recovery significantly impacts the performance of the located cluster. Existing work considers the

single-failure as the most common failure pattern and mainly focuses on reducing the data transmission cost of single-failure

recovery. Propose to reduce redundancy to reduce the duplication and redundancy in the multi- failure recovery of erasure-

coded storages. Erasure codes provide a storage efficient solution and ensure high data availability using significantly less

storage space. when erasures occur and erased data needs to be restored for long-term persistence, the repairing process of

erasure-coded data is less efficient than in replication. When replicated data is erased, repairing is simply done by replicating

one of the remaining replicas (when exists). On the other hand, when encoded data is erased, the repairing node first needs

to download k chunks and reclaim a whole copy of the original file.

1. A parity based protection technique.

-data are broken into fragments and encosed

-Storage across the different location with a configurable number of redundant pieces

pros--

--consumes less storage than replication

--good for cheep\deep

--Allows for the failure of two or more elements of a storage system