Getting the Most out of Erasure Codes

Erasure codes are a recent addition to many storage technologies. They provide increased reliability with less overhead. Yet, they are not without downsides. Selecting the best parameters for erasure coding is a complex optimization problem. As one varies the threshold, write threshold, number of pieces, system capacity, and site count, there may be drastic effects on reliability, availability, storage overhead, rebuilding cost, and CPU expense. Selecting erasure code parameters without weighed consideration may have catastrophic results. In this presentation we present the techniques we developed and use for designing erasure coded systems with the best combination of storage overhead, computational efficiency, reliability, availability, and rebuilding cost for any given system constraints. Finally, we introduce some advanced techniques for reducing rebuilding cost.

Learning Objectives

What the various parameters are in an erasure coded system
The interrelationships between the parameters and how they effect the system's properties
Our methods for selecting the parameters to optimally achieve the goals for the system
Advanced techniques for mitigating rebuilding cost in an erasure coded system