Fine Grained Block Translation

Block translation is often used to linearize writes. This is especially useful for Flash SSDs as well as SMR hard disks. It likewise improves write performance in parity-based arrays. Block translation has an additional capability that can be exploited to create whole new storage paradigms. Block translation lets you construct efficient atomic updates that let you use nearly 100% of your arrays’ write bandwidth to store actual, useful, data. Atomic linearization opens an array of possibilities. Here are some examples that have actually been implemented: Write a file system that lets you create hundreds of thousands of small files in less than a second, from a single thread, with greater than 90% space efficiency. Export blocks to other file systems or a SAN with near array speed writes regardless of workload block size or pattern. Reduce data with in-line compression and de-duplication without suffering a massive memory footprint or glacial performance. The magic which makes such things possible is efficient, linear, generation based, atomic writes. Instead of scattering data, metadata, allocation bitmaps, and journal elements across the array, this solution writes elements together as part of a linear, atomic, string. Linear IO: Each atomic write maintains absolute linearity and optimal alignment at the SSD and the array level. Inline: Each atomic write contains the actual, live, data. There are no additional copies or journals. Merge-able: Each atomic write can be appended with new data. Thousands of transactions can be combined eliminating intermediate updates for maximum bandwidth and space efficiency. Flexible: Atomic updates can include any collection of data blocks or even the absence of blocks. You are not limited in what you can put into a transaction. Variable Block Size: The blocks that you can store can contain from 16 bytes to 1 megabyte or more of payload for a single LBA. You can map your storage structure directly to blocks, even if the structure has variable sized elements. Safe: All IO is validated end-to-end with CPU-assisted checksums. Optimized for Bandwidth: The update structure allows for hundreds of megabytes in a single IO “write segment”. Big data can finally move at device speed. This structure not only optimizes performance, but cost. SSDs are used with ideal write workloads, lowering both wear and cost. Linear writes and alignment mean that erasure codes out-perform mirroring and reach theoretical device speed. One user summed it up well: “We write faster than we read.”

Learning Objectives

Using block translation in new ways,Storage revolves around atomic updates,Consider a new paradigm for file systems and databases