In a recent survey of enterprise hybrid cloud users, the Evaluator Group saw that nearly 60% of respondents indicated that lack of interoperability is a significant technology issue that they must overcome in order to move forward. In fact, lack of interoperability was the number one issue, surpassing public cloud security and network security as significant inhibitors.

The SNIA Cloud Storage Initiative (CSI) is pleased to have John Webster, Senior Partner at Evaluator Group, who will join us on December 12^th for a live webcast to dive into the findings of their research. In this webcast, Multi-Cloud Storage: Addressing the Need for Portability and Interoperability, my SNIA Cloud colleague, Mark Carlson, and John will discuss enterprise hybrid cloud objectives and barriers to adoption. John and Mark will focus on cloud interoperability within the storage domain and the CSI’s work that promotes interoperability and portability of data stored in the cloud.

As moderator of this webcast, I’ll make sure we offer great insights on real-world cloud deployment challenges. As always, we will be available to answer your questions on the spot. I encourage you to register today. We hope to see you on the 12^th.

 

In our most recent SNIA Cloud webcast, “Cloud Object Storage and the Use of Gateways,” we discussed market trends toward the adoption of object storage and the use of gateways to execute on a cloud strategy.  If you missed the live event, it’s now available on-demand together with the webcast slides. There were many good questions at the live event and our expert, Dan Albright, has graciously answered them in this blog. Q. Can object storage be accessed by tools for use with big data? A. Yes. Technically, access to big data is in real-time with HDFS connectors like S3, but it is  conditional on latency and if it is based on local hard drives, it should not be used as the primary storage as it would run very slowly. The guidance is to use hard drive based object storage either as an online archive or a backup target for HDFS. Q. Will current block storage or NAS be replaced with cloud object storage + gateway? A. Yes and no.  It’s dependent on the use case. For ILM (Information Lifecycle Management) uses, only the aged and infrequently accessed data is moved to the gateway+cloud object storage, to take advantage of a lower cost tier of storage, while the more recent and active data remains on the primary block or file storage.  For file sync and share, the small office/remote office data is moved off of the local NAS and consolidated/centralized and managed on the gateway file system. In practice, these methods will vary based on the enterprise’s requirements. Q. Can we use cloud object storage for IoT storage that may require high IOPS? A. High IOPS workloads are best supported by local SSD based Object, Block or NAS storage.  remote or hard drive based Object storage is better deployed with low IOPS workloads. Q. What about software defined storage? A. Cloud object storage may be implemented as SDS (Software Defined Storage) but may also be implemented by dedicated appliances. Most cloud Object storage services are SDS based. Q. Can you please define NAS? A. The SNIA Dictionary defines Network Attached Storage (NAS) as: 1. [Storage System] A term used to refer to storage devices that connect to a network and provide file access services to computer systems. These devices generally consist of an engine that implements the file services, and one or more devices, on which data is stored. 2. [Network] A class of systems that provide file services to host computers using file access protocols such as NFS or CIFS. Q. What are the challenges with NAS gateways into object storage? Aren’t there latency issues that NAS requires that aren’t available in a typical Object store solution? A. The key factor to consider is workload.  If the workload of applications accessing data residing on NAS experiences high frequency of reads and writes then that data is not a good candidate for remote or hard drive based object storage. However, it is commonly known that up to 80% of data residing on NAS is infrequently accessed.  It is this data that is best suited for migration to remote object storage. Thanks for all the great questions. Please check out our library of SNIA Cloud webcasts to learn more. And follow us on Twitter @sniacloud_com for announcements of future webcasts.  

In our most recent SNIA Cloud webcast, “Cloud Object Storage and the Use of Gateways,” we discussed market trends toward the adoption of object storage and the use of gateways to execute on a cloud strategy.  If you missed the live event, it’s now available on-demand together with the webcast slides. There were many good questions at the live event and our expert, Dan Albright, has graciously answered them in this blog. Q. Can object storage be accessed by tools for use with big data? A. Yes. Technically, access to big data is in real-time with HDFS connectors like S3, but it is  conditional on latency and if it is based on local hard drives, it should not be used as the primary storage as it would run very slowly. The guidance is to use hard drive based object storage either as an online archive or a backup target for HDFS. Q. Will current block storage or NAS be replaced with cloud object storage + gateway? A. Yes and no.  It’s dependent on the use case. For ILM (Information Lifecycle Management) uses, only the aged and infrequently accessed data is moved to the gateway+cloud object storage, to take advantage of a lower cost tier of storage, while the more recent and active data remains on the primary block or file storage.  For file sync and share, the small office/remote office data is moved off of the local NAS and consolidated/centralized and managed on the gateway file system. In practice, these methods will vary based on the enterprise’s requirements. Q. Can we use cloud object storage for IoT storage that may require high IOPS? A. High IOPS workloads are best supported by local SSD based Object, Block or NAS storage.  remote or hard drive based Object storage is better deployed with low IOPS workloads. Q. What about software defined storage? A. Cloud object storage may be implemented as SDS (Software Defined Storage) but may also be implemented by dedicated appliances. Most cloud Object storage services are SDS based. Q. Can you please define NAS? A. The SNIA Dictionary defines Network Attached Storage (NAS) as: 1. [Storage System] A term used to refer to storage devices that connect to a network and provide file access services to computer systems. These devices generally consist of an engine that implements the file services, and one or more devices, on which data is stored. 2. [Network] A class of systems that provide file services to host computers using file access protocols such as NFS or CIFS. Q. What are the challenges with NAS gateways into object storage? Aren’t there latency issues that NAS requires that aren’t available in a typical Object store solution? A. The key factor to consider is workload.  If the workload of applications accessing data residing on NAS experiences high frequency of reads and writes then that data is not a good candidate for remote or hard drive based object storage. However, it is commonly known that up to 80% of data residing on NAS is infrequently accessed.  It is this data that is best suited for migration to remote object storage. Thanks for all the great questions. Please check out our library of SNIA Cloud webcasts to learn more. And follow us on Twitter @sniacloud_com for announcements of future webcasts.  

Reviews are in for the 20^th Storage Developer Conference (SDC) and they are thumbs up! The 2017 SDC was the largest ever- expanding to four full days with seven keynotes, five SNIA Tutorials, and 92 sessions.  The SNIA Technical Council, who oversees conference content, compiled a rich agenda of 18 topic categories focused on technology growth markets of physical storage, storage management, data, object storage, and cloud storage.  Storage Architecture led the way with 20 individual sessions, followed by 15 Solid State/Non-Volatile Memory, eight SMB, and six NVMe sessions.  Storage management, including SMI and SNIA Swordfish® presentations, had an entire track with eight sessions on Thursday. Attendees called out a number of sessions as their favorites - and the ratings proved it.  But don’t be alarmed if you missed out – SNIA has recordings and downloads of the presentations at your fingertips.  Check out these top-rated sessions and more:

• Remote Persistent Memory With Nothing But Net by Tom Talpey of Microsoft, discussing RDMA NIC as a Persistent Memory storage adapter and the steps to get there
• Programming the Path by James Cain of Snell Advanced Media, on virtual file systems
• Self-Optimizing Caches by Irfan Ahmad of CachePhysics, on how you can work towards a self-optimizing data path
• SoftFlash by Jae Do of Microsoft Research, on programmable storage in future data centers

Attendees were also treated to NVMe and Persistent Memory demonstrations by SDC sponsors, three Plugfests, and a host of networking conversations happening up and down the hallways.  The “high caliber speakers and presentation content”, learning about "recent developments in the industry", and “connecting directly with other developers who are tackling the same problems” were cited by attendees as some of the most beneficial aspects of the conference. Whether you participated in person or virtually by viewing videos, downloading presentations, or listening to podcasts, let us know what you would like to see for future SDCs.  Is it Modern Storage for Modern Data Centers? NVMe and NVMe-oF? Persistent Memory and PM-oF? Artificial intelligence? New directions?  We want to know!  Watch for our Post Event Survey to be sent out shortly.  And thank you for contributing to a great 2017 SDC!

The use of cloud object storage is ramping up sharply especially in the public cloud, where its simplicity can significantly reduce capital budgets and operating expenses. And while it makes good economic sense, enterprises are challenged with legacy applications that do not support standard protocols to move data to and from the cloud. That’s why the SNIA Cloud Storage Initiative is hosting a live webcast on September 26th, “Cloud Object Storage and the Use of Gateways.” Object storage is a secure, simple, scalable, and cost-effective means of managing the explosive growth of unstructured data enterprises generate every day. Enterprises have developed data strategies specific to the public cloud; improved data protection, long term archive, application development, DevOps, big data analytics and cognitive artificial intelligence to name a few. However, these same organizations have legacy applications and infrastructure that are not object storage friendly, but use file protocols like NFS and SMB. Gateways enable SMB and NFS data transfers to be converted to Amazon’s S3 protocol while optimizing data with deduplication, providing QoS (quality of service), and efficiencies on the data path to the cloud. This webcast will highlight the market trends toward the adoption of object storage and the use of gateways to execute a cloud strategy, the benefits of object storage when gateways are deployed, and the use cases that are best suited to leverage this solution. You will learn:

• The benefits of object storage when gateways are deployed
• Primary use cases for using object storage and gateways in private, public or hybrid cloud
• How gateways can help achieve the goals of your cloud strategy without retooling your on-premise infrastructure and applications

We plan to share some pearls of wisdom on the challenges organizations are facing with object storage in the cloud from a vendor-neutral, SNIA perspective. If you need a firm background on cloud object storage before September 26^th, I encourage you to watch the SNIA Cloud on-demand webcast, “Cloud Object Storage 101.” It will provide you with a foundation to get even more out of this upcoming webcast. I hope you will join us on September 26^th. Register now to save your spot.

Whether storage is already a main focus of your career or may be advancing toward you, you'll definitely want to attend the flagship event for storage developers - and those involved in storage operations, decision making, and usage  - SNIA’s 19^th annual Storage Developer Conference (SDC), September 11-14, 2017 at the Hyatt Regency Santa Clara, California. The SNIA Technical Council has again put together a wide-ranging technical agenda featuring more than 125 industry experts from 60 companies and industry organizations, including Dell/EMC, Docker, FCIA, Google, Hitachi, HPE, IBM, Intel, Microsoft, NetApp, Oracle, Samsung, SAP, STA, and Toshiba. Over four days, network with fellow architects, developers, integrators, and users, and choose from 100+ sessions, three plugfests, and six Birds-of-a-Feather deep dives on a wide range of cutting edge technologies. Current General Session speakers are Sage Weil from Red Hat on Building a New Storage Backend for Ceph and Martin Petersen from Oracle on Recent Developments in the Linux I/O Stack. Among the 15+ topic areas featured at the conference are sessions on: * Flash and Persistent Memory * Big Data, Analytics, and the Internet-of-Things * Storage Resource Management * Storage Performance and Workloads * Containers * Object and Object Drive Storage * Cloud Storage * Storage Security and Identity Management * Data Performance and Capacity Optimization Network with our sponsors Intel, Cisco, IBM, Kalray, Radian, OpenSDS, Celestica, Chelsio, MemoScale, Newisys, SerNet, and Xilinx. Check out special demonstrations in our “Flash Community” area. If you’re a vendor wanting to test product interoperability, grab this chance to participate in one or more of the SDC plugfests underwritten by Microsoft, NetApp, SNIA Cloud Storage Initiative, and SNIA Storage Management Initiative (SMI): Cloud Interoperability, SMB3, and SMI Lab focused on SNIA Swordfish^TM  open to all with SNIA Swordfish^TM implementations. Find all the details here. Plan to attend our Plugfest open house on Monday evening, welcome reception on Tuesday evening, and a special SNIA 20^th anniversary celebration open to SDC attendees and invited guests on Wednesday, September 13. Registration is now open at storagedeveloper.org. where the agenda and speaker list are live. Don't know much about SDC?  Watch a conference overview here and listen to SDC podcasts here. See you in Santa Clara!

SNIA will be out and about in February in San Francisco and Santa Clara, CA, focused on their security, container, and file storage activities. February 14-17 2017, join SNIA in San Francisco at the RSA Conference in the OASIS Interop: KMIP & PKCS11 booth S2115. OASIS and SNIA member companies will be demonstrating OASIS Key Management Interoperability Protocol (KMIP) through live interoperability across all participants. SNIA representatives will be on hand in the booth to answer questions about the Storage Security Industry Forum KMIP Conformance Test Program, which enables organizations with KMIP implementations to validate the protocol conformance of those products and meet market requirements for secure, plug-and-play storage solutions. And Eric Hibbard, Chair of the SNIA Security Technical Work Group and CTO Security and Privacy, HDS Corporation, will participate in the “Can I Get a Witness? Technical Witness Bootcamp” session on February 17. The following week, February 21-23, join SNIA at Container World in Santa Clara CA. Enabling access to memory is an important concern to container designers, and Arthur Sainio, SNIA NVDIMM Special Interest Group Co-Chair from SMART Modular, will speak on Boosting Performance of Data Intensive Applications via Persistent Memory. Integrating containers into legacy solutions will be a focus of a panel where Mark Carlson, SNIA Technical Council Co-Chair from Toshiba, will speak on Container Adoption Paths into Legacy Infrastructure. SNIA experts will be joined by other leaders in the container ecosystem like Docker, Twitter, ADP, Google, and Expedia . The SNIA booth will feature cloud infrastructure and storage discussions and a demonstration of a multi-vendor persistent memory solution featuring NVDIMM!  (P.S. – Are you new to containers? Get a head start on conference discussions by checking out a December 2016 SNIA blog on Containers, Docker, and Storage.)   Closing out February, find SNIA at their booth at USENIX FAST from February 27-March 2 in Santa Clara, CA, where you can engage with SNIA Technical Council leaders on the latest activities in file and storage technologies. We look forward to seeing you at one (or more) of these events!  

SNIA's 18th Storage Developer Conference is officially a success, with 124 general and breakout sessions;  Cloud Interoperability, Kinetic Storage, and SMB3 plugfests; ten Birds-of-a-Feather Sessions, and amazing networking among 450+ attendees.  Sessions on NVMe over Fabrics won the title of most attended, but Persistent Memory, Object Storage, and Performance were right behind.  Many thanks to SDC 2016 Sponsors, who engaged attendees in exciting technology discussions. For those not familiar with SDC, this technical industry event is designed for a variety of storage technologists at various levels from developers to architects to product managers and more.  And, true to SNIA's commitment to educating the industry on current and future disruptive technologies, SDC content is now available to all - whether you attended or not - for download and viewing. You'll want to stream keynotes from Citigroup, Toshiba, DSSD, Los Alamos National Labs, Broadcom, Microsemi, and Intel - they're available now on demand on SNIA's YouTube channel, SNIAVideo. All SDC presentations are now available for download; and over the next few months, you can continue to download SDC podcasts which combine audio and slides. The first podcast from SDC 2016 - on hyperscaler (as well as all 2015 SDC Podcasts) are available here, and more will be available in the coming weeks. SNIA thanks all its members and colleagues who contributed to make SDC a success! A special thanks goes out to the SNIA Technical Council, a select group of acknowledged industry experts who work to guide SNIA technical efforts. In addition to driving the agenda and content for SDC, the Technical Council oversees and manages SNIA Technical Work Groups, reviews architectures submitted by Work Groups, and is the SNIA's technical liaison to standards organizations. Learn more about these visionary leaders at http://www.snia.org/about/organization/tech_council. And finally, don't forget to mark your calendars now for SDC 2017 - September 11-14, 2017, again at the Hyatt Regency Santa Clara. Watch for the Call for Presentations to open in February 2017.

Containers are the latest in what are new and innovative ways of packaging, managing and deploying distributed applications. On October 6^th, the SNIA Cloud Storage Initiative will host a live webcast, “Intro to Containers, Container Storage Challenges and Docker.” Together with our guest speaker from Docker, Keith Hudgins, we’ll begin by introducing the concept of containers. You’ll learn what they are and the advantages they bring illustrated by use cases, why you might want to consider them as an app deployment model, and how they differ from VMs or bare metal deployment environments.

We’ll follow up with a look at what is required from a storage perspective, specifically when supporting stateful applications, using Docker, one of the leading software containerization platforms that provides a lightweight, open and secure environment for the deployment and management of containers. Finally, we’ll round out our Docker introduction by presenting a few key takeaways from DockerCon, the industry-leading event for makers and operators of distributed applications built on Docker, that recently took place in Seattle in June of this year.

Join us for this discussion on:

• Application deployment history
• Containers vs. virtual machines vs. bare metal
• Factors driving containers and common use cases
• Storage ecosystem and features
• Container storage table stakes (focus on Enterprise-class storage services)
• Introduction to Docker
• Key takeaways from DockerCon 2016

This event is live, so we’ll be on hand to answer your questions. Please register today. We hope to see you on Oct. 6^th!

At our recent SNIA Webcast “Data Protection and OpenStack Mitaka,” Ben Swartzlander, Project Team Lead OpenStack Manila (NetApp), and Dr. Sam Fineberg, Distinguished Technologist (HPE), provided terrific insight into data protection capabilities surrounding OpenStack. If you missed the Webcast, I encourage you to watch it on-demand at your convenience. We did not have time to get to all of out attendees’ questions during the live event, so as promised, here are answers to the questions we received.

Q. Why are there NFS drivers for Cinder?

 A. It’s fairly common in the virtualization world to store virtual disks as files in filesystems. NFS is widely used to connect hypervisors to storage arrays for the purpose of storing virtual disks, which is Cinder’s main purpose.

 Q. What does “crash-consistent” mean?

 A. It means that data on disk is what would be there is the system “crashed” at that point in time. In other words, the data reflects the order of the writes, and if any writes are lost, they are the most recent writes. To avoid losing data with a crash consistent snapshot, one must force all recently written data and metadata to be flushed to disk prior to snapshotting, and prevent further changes during the snapshot operation.

Q. How do you recover from a Cinder replication failover?

 A. The system will continue to function after the failover, however, there is currently no mechanism to “fail-back” or “re-replicate” the volumes. This function is currently in development, and the OpenStack community will have a solution in a future release.

 Q. What is a Cinder volume type?

 A. Volume types are administrator-defined “menu choices” that users can select when creating new volumes. They contain hidden metadata, in the cinder.conf file, which Cinder uses to decide where to place them at creation time, and which drivers to use to configure them when created.

 Q. Can you replicate when multiple Cinder backends are in use?

 A. Yes

 Q. What makes a Cinder “backup” different from a Cinder “snapshot”?

 A. Snapshots are used for preserving the state of a volume from changes, allowing recovery from software or user errors, and also allowing a volume to remain stable long enough for it to be backed up. Snapshots are also very efficient to create, since many devices can create them without copying any data. However, snapshots are local to the primary data and typically have no additional protection from hardware failures. In other words, the snapshot is stored on the same storage devices and typically shares disk blocks with the original volume.

Backups are stored in a neutral format which can be restored anywhere and typically on separate (possibly remote) hardware, making them ideal for recovery from hardware failures.

 Q. Can you explain what “share types” are and how they work?

 A. They are Manila’s version of Cinder’s volume types. One key difference is that some of the metadata about them is not hidden and visible to end users. Certain APIs work with shares of types that have specific capabilities.

 Q. What’s the difference between Cinder’s multi-attached and Manila’s shared file system?

A. Multi-attached Cinder volumes require cluster-aware filesystems or similar technology to be used on top of them. Ordinary file systems cannot handle multi-attachment and will corrupt data quickly if attached more than one system. Therefore cinder’s multi-attach mechanism is only intended for fiesystems or database software that is specifically designed to use it.

Manilla’s shared filesystems use industry standard network protocols, like NFS and SMB, to provide filesystems to arbitrary numbers of clients where shared access is a fundamental part of the design.

 Q. Is it true that failover is automatic?

 A. No. Failover is not automatic, for Cinder or Manila

 Q. Follow-up on failure, my question was for the array-loss scenario described in the Block discussion. Once the admin decides the array has failed, does it need to perform failover on a “VM-by-VM basis’? How does the VM know to re-attach to another Fabric, etc.?

A. Failover is all at once, but VMs do need to be reattached one at a time.

 Q. What about Cinder? Is unified object storage on SHV server the future of storage?

 A. This is a matter of opinion. We can’t give an unbiased response.

 Q. What about a “global file share/file system view” of a lot of Manila “file shares” (i.e. a scalable global name space…)

 A. Shares have disjoint namespaces intentionally. This allows Manila to provide a simple interface which works with lots of implementations. A single large namespace could be more valuable but would preclude many implementations.