An examination of the claim that flash will replace hard drives in the data center “Hard drives will soon be a thing of the past.” “The data center of the future is all-flash.” Such predictions foretelling hard drives’ demise, perennially uttered by a few vocal proponents of flash-only technology, have not aged well. Without question, flash storage is well-suited to support applications that require high-performance and speed. And flash revenue is growing, as is all-flash-array (AFA) revenue. But not at the expense of hard drives. We are living in an era where the ubiquity of the cloud and the emergence of AI use cases have driven up the value of massive data sets. Hard drives, which today store by far the majority of the world’s exabytes (EB), are more indispensable to data center operators than ever. Industry analysts expect hard drives to be the primary beneficiary of continued EB growth, especially in enterprise and large cloud data centers—where the vast majority of the world’s data sets reside. Myth: SSD pricing will soon match the pricing of hard drives. Truth: SSD and hard drive pricing will not converge at any point in the next decade. Hard drives hold a firm cost-per-terabyte (TB) advantage over SSDs, which positions them as the unquestionable cornerstone of data center storage infrastructure. Analysis of research by IDC, TRENDFOCUS, and Forward Insights confirms that hard drives will remain the most cost-effective option for most enterprise tasks. The price-per-TB difference between enterprise SSDs and enterprise hard drives is projected to remain at or above a 6 to 1 premium through at least 2027. This differential is particularly evident in the data center, where device acquisition cost is by far the dominant component in total cost of ownership (TCO). Taking all storage system costs into consideration—including device acquisition, power, networking, and compute costs—a far superior TCO is rendered by hard drive-based systems on a per-TB basis. Myth: Supply of NAND can ramp to replace all hard drive capacity. Truth: Entirely replacing hard drives with NAND would require untenable CapEx investments. The notion that the NAND industry would or could rapidly increase its supply to replace all hard drive capacity isn’t just optimistic—such an attempt would lead to financial ruin. According to the Q4 2023 NAND Market Monitor report from industry analyst Yole Intelligence, the entire NAND industry shipped 3.1 zettabytes (ZB) from 2015 to 2023, while having to invest a staggering $208 billion in CapEx—approximately 47% of their combined revenue. In contrast, the hard drive industry addresses the vast majority—almost 90%—of large-scale data center storage needs in a highly capital-efficient manner. To help crystalize it, let's do a thought experiment. Note that there are 3 hard drive manufacturers in the world. Let's take one of them, for whom we have the numbers, as an example. Look at the chart below where byte production efficiency of the NAND and hard drive industries are compared, using this hard drive manufacturer as a proxy. You can easily see, even with just one manufacturer represented, that the hard drive industry is far more efficient at delivering ZBs to the data center. Could the flash industry fully replace the entire hard drive industry’s capacity output by 2028? Yole Intelligence report cited above indicates that from 2025 to 2027, the NAND industry will invest about $73 billion, which is estimated to yield 963EB of output for enterprise SSDs as well as other NAND products for tablets and phones. This translates to an investment of about $76 per TB of flash storage output. Applying that same capital price per bit, it would require a staggering $206 billion in additional investment to support the 2.723ZB of hard drive capacity forecast to ship in 2027. In total, that’s nearly $279 billion of investment for a total addressable market of approximately $25 billion. A 10:1 loss. This level of investment is unlikely for an industry facing uncertain returns, especially after losing money throughout 2023. Myth: Only AFAs can meet the performance requirements of modern enterprise workloads. Truth: Enterprise storage architecture usually mixes media types to optimize for the cost, capacity, and performance needs of specific workloads. At issue here is a false dichotomy. All-flash vendors advise enterprises to “simplify” and “future-proof” by going all-in on flash for high performance. Otherwise, they posit, enterprises risk finding themselves unable to keep pace with the performance demands of modern workloads. This zero-sum logic fails because:

1. Most modern workloads do not require the performance advantage offered by flash.
2. Enterprises must balance capacity and cost, as well as performance.
3. The purported simplicity of a single-tier storage architecture is a solution in search of a problem.

Let’s address these one by one. First, most of the world’s data resides in the cloud and large data centers. There, only a small percentage of the workload requires a significant percentage of the performance. This is why according to IDC[1] over the last five years, hard drives have amounted to almost 90% of the storage installed base in cloud service providers and hyperscale data centers. In some cases, all-flash systems are not even required at all as part of the highest performance solutions. There are hybrid storage systems that perform as well as or faster than all-flash. Second, TCO considerations are key to most data center infrastructure decisions. This forces a balance of cost, capacity, and performance. Optimal TCO is achieved by aligning the most cost-effective media—hard drive, flash, or tape—to the workload requirement. Hard drives and hybrid arrays (built from hard drives and SSDs) are a great fit for most enterprise and cloud storage and application use cases. While flash storage excels in read-intensive scenarios, its endurance diminishes with increased write activity. Manufacturers address this with error correction and overprovisioning—extra, unseen storage, to replace worn cells. However, overprovisioning greatly increases the imbedded product cost and constant power is needed to avoid data loss, posing cost challenges in data centers. Additionally, while technologies like triple level cell (TLC) and quad-level cell (QLC) allow flash to handle data-heavy workloads like hard drives, the economic rationale weakens for larger data sets or long-term retention. In these cases, disk drives, with their growing areal density, offer a more cost-effective solution. Third, the claim that using an AFA is “simpler” than adopting a mix of media types in a tiered architecture is a solution in search of a problem. Many hybrid storage systems employ a well-proven and finely tuned software-defined architecture that seamlessly integrates and harnesses the strengths of diverse media types into singular units. In scale-out private or public cloud data center architectures, file systems or software defined storage is used to manage the data storage workloads across data center locations and regions. AFAs and SSDs are a great fit for high-performance, read-intensive workloads. But it’s a mistake to extrapolate from niche use cases or small-scale deployments to the mass market and hyperscale where AFAs provide an unnecessarily expensive way to do what hard drives already deliver at a much lower TCO. The data bears it out. Analysis of data from IDC and TRENDFOCUS predicts an almost 250% increase in EB outlook for hard drives by 2028. Extrapolating further out in time, that ratio holds well into the next decade. Hard drives, indeed, are here to stay—in synergy with flash storage.  And the continued use of hard drives and hybrid array solutions supports, in turn, the continued use of SAS technologies in the infrastructure data centers. About the Author Jason Feist is senior vice president of marketing, products and markets, at Seagate Technology.   [1] IDC, Multi-Client Study, Cloud Infrastructure Index 2023: Compute and Storage Consumption by 100 Service Providers, November 2023

In our recent live SNIA Network Storage Forum webcast, “Next-generation Interconnects: The Critical Importance of Connectors and Cables” provided an outstanding tutorial on the latest in the impressive array of data center infrastructure components designed to address expanding requirements for higher-bandwidth and lower-power. They covered common pluggable connectors and media types, copper cabling and transceivers, and real world use cases. If you missed the live event, it is available on-demand. We ran out of time to answer all the questions from the live audience. As promised, here are answers to them all. Q. For 25GbE, is the industry consolidating on one of the three options? A. The first version of 25 GbE (i.e. 25GBASE-CR1) was specified by the Ethernet Technology Consortium around 2014.  This was followed approximately two years later by the IEEE 802.3 versions of 25 GbE (i.e. 25GBASE-CR-S and 25GBASE-CR). As a result of the timing, the first 25 GbE capable products to market only supported consortium mode. More recently developed switches and server products support both consortium and IEEE 802.3 modes, with the link establishment protocol favoring the IEEE 802.3 mode when it is available. Therefore, IEEE 802.3 will likely be the incumbent over the long term. Please note that there is no practical difference between Consortium vs. IEEE 802.3 for short reaches (<=3m)  between end points, where a 25G-CA-N cable is used. The longer cable reaches above 3m (CA-25G-L) requires the IEEE 802.3 modes with forward error correction, which adds latency.  The CA-25G-S type of cable is the least common. See slide 16 and slide 17 from the presentation. Q. What’s the max DAC cables (passive and/or active) length for 100G PAM4? A. The IEEE P802.3ck specification for 100 Gb/s/lane copper cable PHYs targets a reach of at least 2 meters for a passive copper cable. Because the specification is still in development, the exact reach is still being worked out. Expect >2 meters for passive cables and 3-4 meters for active cables. Note that this is a reduction of reach from previous rates, as illustrated on slide 27, and that DAC cables are not for long range and generally used for very short interconnections between devices. For longer reaches, AOC cables are preferred. The passive copper cable length is primarily driven by the performance of the SERDES in the host (i.e. switch, server, FPGA, etc) and the construction materials of the cable assembly. Active copper cables (ACCs or AECs) use several different methods of signal conditioning to increase the cable reach; more sophisticated solutions have greater reach, greater latency and greater cost. See slide 35. Q. What’s the latency difference between active and passive DAC (PAM4 encoding)? A. Passive copper cables do not contain signal conditioning electronics. Active copper cables (ACCs or AECs) use several different methods of signal conditioning to increase the cable reach; more sophisticated solutions have greater reach, greater latency and greater cost. See slide 35. A simple active cable may use one or more linear equalizer IC where as a complex cable uses a full retimer that may use FEC logic embedded inside. For 50G PAM4 rates, the difference in one-way latency between a passive copper cable and a simple active copper cable is ~20 nsec. The difference between a passive copper cable and a complex active copper cable could be as high as ~80 nsec. Q. Can you comment about “gearbox” cables (200G 4lane (@56G) to 100G 4 lane (@28G)? A. A few companies are supplying cables that have 50G PAM4 on one end and 25G-NRZ on the other with gear box. We see it as a niche; used to link new to older equipment. Q. Showing QSFP instead of OSFP on slide 29 in the image at the top right? A. Good catch. That was a mistake on the slide. It has been corrected. Thanks.

Object storage is a secure, simple, scalable, and cost-effective means of embracing the explosive growth of unstructured data enterprises generate every day. Object storage adoption is on the rise. That’s why the SNIA Networking Storage Forum (NSF) is hosting “Object Storage: What, How and Why.”  This webcast, with experts Chris Evans of Bookend LTD, Rick Vanover of Veeam, and Alex McDonald, Vice Chair of SNIA NSF and NetApp, the will explain how object storage works, its benefits and why it’s important. Like other storage technologies, object storage brings its own set of unique characteristics to the market. Join us on February 19^th at 10:00 am PT/1:00 pm ET to learn:

• What object storage is and what it does
• How to use object storage
• Essential characteristics of typical consumption
• Why object storage is important to data creators and consumers everywhere

Save your place now and register for this webcast on February 19^th. We look forward to seeing you.

Reviews are in for the 20^th Storage Developer Conference (SDC) and they are thumbs up! The 2017 SDC was the largest ever- expanding to four full days with seven keynotes, five SNIA Tutorials, and 92 sessions.  The SNIA Technical Council, who oversees conference content, compiled a rich agenda of 18 topic categories focused on technology growth markets of physical storage, storage management, data, object storage, and cloud storage.  Storage Architecture led the way with 20 individual sessions, followed by 15 Solid State/Non-Volatile Memory, eight SMB, and six NVMe sessions.  Storage management, including SMI and SNIA Swordfish® presentations, had an entire track with eight sessions on Thursday. Attendees called out a number of sessions as their favorites - and the ratings proved it.  But don’t be alarmed if you missed out – SNIA has recordings and downloads of the presentations at your fingertips.  Check out these top-rated sessions and more:

• Remote Persistent Memory With Nothing But Net by Tom Talpey of Microsoft, discussing RDMA NIC as a Persistent Memory storage adapter and the steps to get there
• Programming the Path by James Cain of Snell Advanced Media, on virtual file systems
• Self-Optimizing Caches by Irfan Ahmad of CachePhysics, on how you can work towards a self-optimizing data path
• SoftFlash by Jae Do of Microsoft Research, on programmable storage in future data centers

Attendees were also treated to NVMe and Persistent Memory demonstrations by SDC sponsors, three Plugfests, and a host of networking conversations happening up and down the hallways.  The “high caliber speakers and presentation content”, learning about "recent developments in the industry", and “connecting directly with other developers who are tackling the same problems” were cited by attendees as some of the most beneficial aspects of the conference. Whether you participated in person or virtually by viewing videos, downloading presentations, or listening to podcasts, let us know what you would like to see for future SDCs.  Is it Modern Storage for Modern Data Centers? NVMe and NVMe-oF? Persistent Memory and PM-oF? Artificial intelligence? New directions?  We want to know!  Watch for our Post Event Survey to be sent out shortly.  And thank you for contributing to a great 2017 SDC!

SNIA will be out and about in February in San Francisco and Santa Clara, CA, focused on their security, container, and file storage activities. February 14-17 2017, join SNIA in San Francisco at the RSA Conference in the OASIS Interop: KMIP & PKCS11 booth S2115. OASIS and SNIA member companies will be demonstrating OASIS Key Management Interoperability Protocol (KMIP) through live interoperability across all participants. SNIA representatives will be on hand in the booth to answer questions about the Storage Security Industry Forum KMIP Conformance Test Program, which enables organizations with KMIP implementations to validate the protocol conformance of those products and meet market requirements for secure, plug-and-play storage solutions. And Eric Hibbard, Chair of the SNIA Security Technical Work Group and CTO Security and Privacy, HDS Corporation, will participate in the “Can I Get a Witness? Technical Witness Bootcamp” session on February 17. The following week, February 21-23, join SNIA at Container World in Santa Clara CA. Enabling access to memory is an important concern to container designers, and Arthur Sainio, SNIA NVDIMM Special Interest Group Co-Chair from SMART Modular, will speak on Boosting Performance of Data Intensive Applications via Persistent Memory. Integrating containers into legacy solutions will be a focus of a panel where Mark Carlson, SNIA Technical Council Co-Chair from Toshiba, will speak on Container Adoption Paths into Legacy Infrastructure. SNIA experts will be joined by other leaders in the container ecosystem like Docker, Twitter, ADP, Google, and Expedia . The SNIA booth will feature cloud infrastructure and storage discussions and a demonstration of a multi-vendor persistent memory solution featuring NVDIMM!  (P.S. – Are you new to containers? Get a head start on conference discussions by checking out a December 2016 SNIA blog on Containers, Docker, and Storage.)   Closing out February, find SNIA at their booth at USENIX FAST from February 27-March 2 in Santa Clara, CA, where you can engage with SNIA Technical Council leaders on the latest activities in file and storage technologies. We look forward to seeing you at one (or more) of these events!  

Here at the SNIA Ethernet Storage Forum (ESF), we’ve been discussing how hyperconverged adoption will impact storage. Converged Infrastructure (CI), Hyperconverged Infrastructure (HCI), along with Cluster or Cloud In a Box (CIB) are popular trend topics that have gained both industry and customer adoption. As part of data infrastructures, CI, HCI, and CIB enable simplified deployment of resources (servers, storage, I/O networking, hypervisor, application software) across different environments.

But what do these approaches mean for the storage environment? What are the key concerns and considerations related specifically to storage? How will the storage be connected to (or included in) the platform? Who will protect and backup the data? And most importantly, how do you know that you’re asking the right questions in order to get to the right answers?

Find out on March 15th in a live SNIA-ESF webcast, “What Does Hyperconverged Mean to Storage.” We’ve invited expert Greg Schulz, founder and analyst of Server StorageIO, to answer the questions we’ve been debating. Join us, as Greg will move beyond the hype (pun intended) to discuss:

• What are the storage considerations for CI, CIB and HCI
• Why fast applications and fast servers need fast I/O
• Networking and server-storage I/O considerations
• How to avoid aggravation-causing aggregation (bottlenecks)
• Aggregated vs. disaggregated vs. hybrid converged
• Planning, comparing, benchmarking and decision-making
• Data protection, management and east-west I/O traffic
• Application and server north-south I/O traffic

Register today and please bring your questions. We’ll be on-hand to answer them during this event. We hope to see you there!

Buffers and Queues are part of every data center architecture, and a critical part of performance – both in improving it as well as hindering it. A well-implemented buffer can mean the difference between a finely run system and a confusing nightmare of troubleshooting. Knowing how buffers and queues work in storage can help make your storage system shine.

However, there is something of a mystique surrounding these different data center components, as many people don’t realize just how they’re used and why. Join our team of carefully-selected experts on February 14^th in the next live webcast in our “Too Proud to Ask” series, “Everything You Wanted to Know About Storage But Were Too Proud To Ask – Part Teal: The Buffering Pod” where we’ll demystify this very important aspect of data center storage. You’ll learn:

• What are buffers, caches, and queues, and why you should care about the differences?
• What’s the difference between a read cache and a write cache?
• What does “queue depth” mean?
• What’s a buffer, a ring buffer, and host memory buffer, and why does it matter?
• What happens when things go wrong?

These are just some of the topics we’ll be covering, and while it won’t be exhaustive look at buffers, caches and queues, you can be sure that you’ll get insight into this very important, and yet often overlooked, part of storage design.

Register today and spend Valentine’s Day with our experts who will be on-hand to answer your questions on the spot!

Did you know that SNIA has a YouTube Channel?  SNIAVideo is the place designed for You to visit for the latest technical and educational content - all free to download - from SNIA thought leaders and events.  Our latest videos cover a wide range of topics discussed at last month's SNIA Storage Developer Conference.  Enjoy The Ride Cast video playlist where industry expert Marc Farley (@GoFarley) motors around Silicon Valley with SNIA member volunteers Richelle Ahlvers(@rahlvers), Stephen Bates (@stepbates), Mark Carlson(@macsun), and storage and solid state technology analysts Tom Coughlin (@ThomasaCoughlin), and Jim Handy chatting about persistent memory, SNIA Swordfish, NVMe, storage end users, and more.  You'll also want to check out onsite interviews from Kinetic open storage project participants Seagate, Scality, and Open vStorage on their experiences at an SDC solutions plugfest. Featured on the SNIAVideo YouTube Channel are SNIA thought leaders weighing in on the trends and activities that will revolutionize enterprise data centers and consumer applications over the next decade.  New ways to unify the management of storage and servers in hyperscale and cloud environments; an ecosystem driving system memory and storage into a single, unified “persistent memory” entity; and how security is being managed at enterprises today are just a few of the topics covered by speakers from Microsoft, Intel, Toshiba, Cryptsoft, and more. Bookmark our site and return often for fresh, new content on how SNIA helps You understand and solve the thorny storage issues facing your career and your organization.

SNIA's 18th Storage Developer Conference is officially a success, with 124 general and breakout sessions;  Cloud Interoperability, Kinetic Storage, and SMB3 plugfests; ten Birds-of-a-Feather Sessions, and amazing networking among 450+ attendees.  Sessions on NVMe over Fabrics won the title of most attended, but Persistent Memory, Object Storage, and Performance were right behind.  Many thanks to SDC 2016 Sponsors, who engaged attendees in exciting technology discussions. For those not familiar with SDC, this technical industry event is designed for a variety of storage technologists at various levels from developers to architects to product managers and more.  And, true to SNIA's commitment to educating the industry on current and future disruptive technologies, SDC content is now available to all - whether you attended or not - for download and viewing. You'll want to stream keynotes from Citigroup, Toshiba, DSSD, Los Alamos National Labs, Broadcom, Microsemi, and Intel - they're available now on demand on SNIA's YouTube channel, SNIAVideo. All SDC presentations are now available for download; and over the next few months, you can continue to download SDC podcasts which combine audio and slides. The first podcast from SDC 2016 - on hyperscaler (as well as all 2015 SDC Podcasts) are available here, and more will be available in the coming weeks. SNIA thanks all its members and colleagues who contributed to make SDC a success! A special thanks goes out to the SNIA Technical Council, a select group of acknowledged industry experts who work to guide SNIA technical efforts. In addition to driving the agenda and content for SDC, the Technical Council oversees and manages SNIA Technical Work Groups, reviews architectures submitted by Work Groups, and is the SNIA's technical liaison to standards organizations. Learn more about these visionary leaders at http://www.snia.org/about/organization/tech_council. And finally, don't forget to mark your calendars now for SDC 2017 - September 11-14, 2017, again at the Hyatt Regency Santa Clara. Watch for the Call for Presentations to open in February 2017.

Mark Carlson is the current Chair of the SNIA Technical Council (TC). Mark has been a SNIA member and volunteer for over 18 years, and also wears many other SNIA hats.   Recently, SNIA on Storage sat down with Mark to discuss his first nine months as the TC Chair and his views on the industry. SNIA on Storage (SoS):  Within SNIA, what is the most important activity of the SNIA Technical Council? Mark Carlson (MC): The SNIA Technical Council works to coordinate and approve the technical work going on within SNIA. This includes both SNIA Architecture (standards) and SNIA Software. The  work is conducted within 13 SNIA Technical Work Groups (TWGs).  The members of the TC are elected from the voting companies of SNIA, and the Council also includes appointed members and advisors as well as SNIA regional affiliate advisors.  SoS:  What has been your focus this first nine months of 2016?    MC: The SNIA Technical Council has overseen a major effort to integrate a new standard organization into SNIA.  The creation of the new SNIA SFF Technology Affiliate (TA) Technical Work Group has brought in a very successful group of folks and standards related to storage connectors and transceivers. This work group, formed in June 2016, carries forth the longstanding SFF Committee work efforts that has operated since 1990 until mid-2016.  In 2016, SFF Committee leaders transitioned the organizational stewardship to SNIA, to operate under a special membership class named Technology Affiliate, while retaining the long standing technical focus on specifications in a similar fashion as all SNIA TWGs do. SoS:  What changes did SNIA implement to form the new Technology Affiliate membership class and why? MC: The SNIA Policy and Procedures were changed to account for this new type of membership.  Companies can now join an Affiliate TWG without having to join SNIA as a US member.  Current SNIA members who want to participate in a Technology Affiliate like SFF can join a Technology Affiliate and pay the separate dues.  The SFF was a catalyst - we saw an organization looking for a new home as its membership evolved and its leadership transitioned.  They felt SNIA could be this home but we needed to complete some activities to make it easier for them to seamlessly continue their work.   The SFF is now fully active within SNIA and also working closely with T10 and T11, groups that SNIA members have long participated in. SoS:  Is forming this Technology Affiliate a one-time activity? MC: Definitely not.  The SNIA is actively seeking organizations who are looking for a structure that SNIA provides with IP policies, established infrastructure to conduct their work, and 160+ leading companies with volunteers who know storage and networking technology. SoC:  What are some of the customer pain points you see in the industry? MC: Critical pain points the TC has started to address with new TWGs over the last 24 months include: performance of solid state storage arrays, where the SNIA Solid State Storage Systems (S4) TWG is working to identify, develop, and coordinate system performance standards for solid state storage systems; and object drives, where work is being done by the Object Drive TWG to identify, develop, and coordinate standards for object drives operating as storage nodes in scale out storage solutions.  With the number of different future disk drive interfaces emerging that add value from external storage to in-storage compute, we want to make sure they can be managed at scale and are interoperable. SoS:  What's upcoming for the next six months? MC: The TC is currently working on a white paper to address data center drive requirements and the features and existing interface standards that satisfy some of those requirements.  Of course, not all the solutions to these requirements will come from SNIA, but we think SNIA is in a unique position to bring in the data center customers that need these new features and work with the drive vendors to prototype solutions that then make their way into other standards efforts.  Features that are targeted at the NVM Express, T10, and T13 committees would be coordinated with these customers. SoS:  Can non-members get involved with SNIA? MC:   Until very recently, if a company wanted to contribute to a software project within SNIA, they had to become a member. This was limiting to the community, and cut off contributions from those who were using the code, so SNIA has developed a convenient Contributor License Agreement (CLA) for contributions to individual projects.  This allows external contributions but does not change the software licensing. The CLA is compatible with the IP protections that the SNIA IP Policy provides to our members.  Our hope is that this will create a broader community of contributors to a more open SNIA, and facilitate open source project development even more. SoS:  Will you be onsite for the upcoming SNIA Storage Developer Conference (SDC)? MC: Absolutely!  I look forward to meeting SNIA members and colleagues September 19-22 at the Hyatt Regency Santa Clara.  We have a great agenda, now online, that the TC has developed for this, our 18th conference, and registration is now open.  SDC brings in more than 400 of the leading storage software and hardware developers, storage product and solution architects, product managers, storage product quality assurance engineers, product line CTOs, storage product customer support engineers, and in–house IT development staff from around the world.  If technical professionals are not familiar with the education and knowledge that SDC can provide, a great way to get a taste is to check out the SDC Podcasts now posted, and the new ones that will appear leading up to SDC 2016.