According to research from the Enterprise Strategy Group (ESG), IT organizations today are struggling to strike the right balance between public cloud and their on-premises infrastructure. Has hybrid cloud reached a tipping point? Find out on April 23, 2019 at our live webcast “The Hybrid Cloud Tipping Point” when the SNIA CSTI welcomes ESG senior analyst, Scott Sinclair, who will share research on current cloud trends, covering:

• Key drivers behind IT complexity
• IT spending priorities
• Multi-cloud & hybrid cloud adoption drivers
• When businesses are moving workloads from the cloud back on-premises
• Top security and cost challenges
• Future cloud projections

The research presentation will be followed by a panel discussion with Scott Sinclair and my SNIA cloud colleagues, Alex McDonald, Mike Jochimsen and Eric Lakin. We will be on-hand on the 23^rd to answer questions. Register today. We hope to see you there.

On February 13, 2019, the SNIA Cloud Storage Technologies Initiative (CSTI) presented a live webcast, Why Composable Infrastructure? Our goal was to clearly explain the reasoning behind, and the benefits of, composable infrastructure in an educational, vendor-neutral way. We believe our speakers, Philip Kufeldt and Mike Jochimsen, did just that. Now, as promised, Philip and Mike have answered the many interesting questions we received during the live event.

Q. Are composable infrastructure solutions incompatible with virtualized or containerized environments? Will these solutions only serve bare metal environments?

A. Composable infrastructure solutions will eventually work across any environment that supports the orchestration toolsets. There are no compatibility issues between virtualization/containerization and composable infrastructure, even if they fundamentally look at allocation of resources within a defined resource differently. For example, in a virtualized environment if the need for network bandwidth or storage capacity exceeds the capability of a given resource, a "larger" resource could be composed using the orchestration tools. It would then be managed within the virtualization layer like any other resource.

Q. Typically new technology adoption is slowed due to support within commercial operating systems. Are there changes needed in the major OS environments (Linux, Windows, VMware, etc.), that will need to be released before composable infrastructure solutions will be supported?

A. So the PCIe and Ethernet based fabrics are already well established and have great OS support. The storage world and networking worlds already deploy composable infrastructure. However, the newer standards such as Gen-Z, OpenCAPI, and CCIX will need both hardware and software support.   ARM SoCs are showing up with CCIX HW and OpenCAPI is in the Power architecture. But this is just the early stage; switches, enclosures and components that support these standards are still in the offing. Furthermore the OS support for these standards is also unavailable. And finally the management mechanisms and composability software is also undefined. So we still are a good distance from the larger composable infrastructure being available.

Q. Are the data center orchestration tools currently on the market capable of building a composable infrastructure solution like you described?

A. The tools on the market are still in the early stages of providing this capability. Some are purpose built for specific environments while others encompass a wider set of environments, but lack some of the dynamic/automation capabilities. Also, there is work going on, or starting up, in standards bodies to define the APIs needed for orchestration to work in truly heterogeneous application, operating system and hardware environments.

Q. In composable environments, how does security scale with it, specifically, encryption? Encrypt everything? Or some subset of jobs that truly are only jobs requiring encryption?

A. Fabrics can be configured to be local and private relieving the need for encrypted transfers. However there will be new issues to contend with. For example, consider memory that was previously used in one configuration that was disassembled and then reused in another. Ensuring that memory is cleaned before reuse will become required to prevent information leakage.

Q. For Gen-Z, Pooled Memory or Memory from different Racks, what about the Latency issues? Local memories don’t have issues with latency?

A. Although Gen-Z supports longer distance interconnects, it does not mean that only long distance configurations will be utilized. Think of it as a set of tools in a toolbox. Some memories will be close for lower latencies and others will be farther to provide for 4th or 5th level caching.

Q. Is it more about declarative mapping of the components? At this point software and hardware are decoupled, so the messaging and logic are really the requirement for orchestration.

A. The orchestration layer provides a translation between a declarative and imperative state in composable infrastructure. It is responsible for gathering the requirements from the application (declarative - "this is what I want"), then identifying the capabilities of the components on the network and logically mapping them to create a virtual infrastructure (imperative - "this is how to do it").

Q. As apps start to be built from microservices, which may run across different physical nodes, I would think this further raises performance challenges on disaggregated infrastructure. How this will impact things would be an interesting next topic.

A. Agreed. Although I believe microservices will actually be enhanced by composable infrastructure. Composable infrastructure in general will create smaller systems that more closely fit the needs of the service or classes of services that will run on them. Just as in a bin packing problem having smaller units tends to provide better utilization of the container.

Got more questions? Feel free to comment on this blog and we’ll answer.

On February 13, 2019, the SNIA Cloud Storage Technologies Initiative (CSTI) presented a live webcast, Why Composable Infrastructure? Our goal was to clearly explain the reasoning behind, and the benefits of, composable infrastructure in an educational, vendor-neutral way. We believe our speakers, Philip Kufeldt and Mike Jochimsen, did just that. Now, as promised, Philip and Mike have answered the many interesting questions we received during the live event. Q. Are composable infrastructure solutions incompatible with virtualized or containerized environments? Will these solutions only serve bare metal environments? A. Composable infrastructure solutions will eventually work across any environment that supports the orchestration toolsets. There are no compatibility issues between virtualization/containerization and composable infrastructure, even if they fundamentally look at allocation of resources within a defined resource differently. For example, in a virtualized environment if the need for network bandwidth or storage capacity exceeds the capability of a given resource, a “larger” resource could be composed using the orchestration tools. It would then be managed within the virtualization layer like any other resource. Q. Typically new technology adoption is slowed due to support within commercial operating systems. Are there changes needed in the major OS environments (Linux, Windows, VMware, etc.), that will need to be released before composable infrastructure solutions will be supported? A. So the PCIe and Ethernet based fabrics are already well established and have great OS support. The storage world and networking worlds already deploy composable infrastructure. However, the newer standards such as Gen-Z, OpenCAPI, and CCIX will need both hardware and software support.   ARM SoCs are showing up with CCIX HW and OpenCAPI is in the Power architecture. But this is just the early stage; switches, enclosures and components that support these standards are still in the offing. Furthermore the OS support for these standards is also unavailable. And finally the management mechanisms and composability software is also undefined. So we still are a good distance from the larger composable infrastructure being available. Q. Are the data center orchestration tools currently on the market capable of building a composable infrastructure solution like you described? A. The tools on the market are still in the early stages of providing this capability. Some are purpose built for specific environments while others encompass a wider set of environments, but lack some of the dynamic/automation capabilities. Also, there is work going on, or starting up, in standards bodies to define the APIs needed for orchestration to work in truly heterogeneous application, operating system and hardware environments. Q. In composable environments, how does security scale with it, specifically, encryption? Encrypt everything? Or some subset of jobs that truly are only jobs requiring encryption? A. Fabrics can be configured to be local and private relieving the need for encrypted transfers. However there will be new issues to contend with. For example, consider memory that was previously used in one configuration that was disassembled and then reused in another. Ensuring that memory is cleaned before reuse will become required to prevent information leakage. Q. For Gen-Z, Pooled Memory or Memory from different Racks, what about the Latency issues? Local memories don’t have issues with latency? A. Although Gen-Z supports longer distance interconnects, it does not mean that only long distance configurations will be utilized. Think of it as a set of tools in a toolbox. Some memories will be close for lower latencies and others will be farther to provide for 4th or 5th level caching. Q. Is it more about declarative mapping of the components? At this point software and hardware are decoupled, so the messaging and logic are really the requirement for orchestration. A. The orchestration layer provides a translation between a declarative and imperative state in composable infrastructure. It is responsible for gathering the requirements from the application (declarative – “this is what I want”), then identifying the capabilities of the components on the network and logically mapping them to create a virtual infrastructure (imperative – “this is how to do it”). Q. As apps start to be built from microservices, which may run across different physical nodes, I would think this further raises performance challenges on disaggregated infrastructure. How this will impact things would be an interesting next topic. A. Agreed. Although I believe microservices will actually be enhanced by composable infrastructure. Composable infrastructure in general will create smaller systems that more closely fit the needs of the service or classes of services that will run on them. Just as in a bin packing problem having smaller units tends to provide better utilization of the container. Got more questions? Feel free to comment on this blog and we’ll answer.

Cloud data centers are by definition very dynamic. The need for infrastructure availability in the right place at the right time for the right use case is not as predictable, nor as static, as it has been in traditional data centers. These cloud data centers need to rapidly construct virtual pools of compute, network and storage based on the needs of particular customers or applications, then have those resources dynamically and automatically flex as needs change. To accomplish this, many in the industry espouse composable infrastructure capabilities, which rely on heterogeneous resources with specific capabilities that can be discovered, managed, and automatically provisioned and re-provisioned through data center orchestration tools. The primary benefit of composable infrastructure results in a smaller grained sets of resources that are independently scalable and can be brought together as required. On February 13, 2019, The SNIA Cloud Storage Technologies Initiative is going to examine what’s happening with composable infrastructure in our live webcast, Why Composable Infrastructure? In this webcast, SNIA experts will discuss: What prompted the development of composable infrastructure?

• What is composable infrastructure?
• What are the enabling technologies and potential solutions
• Enabling technologies (not just what’s here, but what’s needed…)
• An update on the current status of composable infrastructure standards/products, and where we might be in two to five years

Our goal is to clearly explain the reasoning behind and the benefits of composable infrastructure in an educational, vendor-neutral way. We hope you’ll join us. Our experts will be on hand to answer your questions. Register today to save your spot.

Cloud data centers are by definition very dynamic. The need for infrastructure availability in the right place at the right time for the right use case is not as predictable, nor as static, as it has been in traditional data centers. These cloud data centers need to rapidly construct virtual pools of compute, network and storage based on the needs of particular customers or applications, then have those resources dynamically and automatically flex as needs change. To accomplish this, many in the industry espouse composable infrastructure capabilities, which rely on heterogeneous resources with specific capabilities that can be discovered, managed, and automatically provisioned and re-provisioned through data center orchestration tools. The primary benefit of composable infrastructure results in a smaller grained sets of resources that are independently scalable and can be brought together as required. On February 13, 2019, The SNIA Cloud Storage Technologies Initiative is going to examine what’s happening with composable infrastructure in our live webcast, Why Composable Infrastructure? In this webcast, SNIA experts will discuss:

• What prompted the development of composable infrastructure?
• What is composable infrastructure?
• What are the enabling technologies and potential solutions
• Enabling technologies (not just what’s here, but what’s needed…)
• An update on the current status of composable infrastructure standards/products, and where we might be in two to five years

Our goal is to clearly explain the reasoning behind and the benefits of composable infrastructure in an educational, vendor-neutral way. We hope you’ll join us. Our experts will be on hand to answer your questions. Register today to save your spot.

The SNIA Storage Developer Conference (SDC) is coming up September 24-27, 2018 at the Hyatt Regency Santa Clara CA.  The agenda is now live! SNIA on Storage is teaming up with the SNIA Technical Council to dive into major themes of the 2018 conference.  The SNIA Technical Council takes a leadership role to develop the content for each SDC, so SNIA on Storage spoke with Mark Carlson, SNIA Technical Council Co-Chair and Principal Engineer, Industry Standards, Toshiba Memory America, to understand why SDC is bringing Orchestration to conference attendees. SNIA On Storage (SOS):  When I think of “orchestration”, my first vision is of a conductor with a magnificent symphony.  Am I on the right track in thinking this way? Mark Carlson (MC):  If you think of the conductor as the “automator” of the symphony, you’re right on!  Orchestration for computing is the automated arrangement, coordination, and management of computer systems, middleware, and services.  For example, cloud orchestration technology helps to manage the interconnections and interactions among public and private cloud infrastructure workloads. SOS:  What are some examples of orchestration? MC:  One example is in using software containers, which are applications built on a small set of supported libraries instead of on the full operating system. This is an efficient way to use the underlying infrastructure instead of firing up multiple versions of the operating system as Virtual Machines.  Many organizations run microservices (small app components) in these containers, which can provide a simpler way to scale up a service in a much more fine-grained manner.  Here, orchestration is the conductor – the person up front – who makes sure the microservices and container-based infrastructure playing the music are in tune and on time.  There are many examples of container-based infrastructure, such as Kubernetes and Docker, evolving from the days when companies like Netflix moved to the cloud, created containers, and populated them with microservices. SOS:  Do companies and applications approach orchestration in the same way? MC:  Every container-based infrastructure has its own approach to orchestration and to the storage underlying the microservices.  SNIA member volunteers work with a variety of standards bodies and organizations to support how storage operations work in orchestration.  For example, Kubernetes is standardizing around an Application Programming Interface (API) for container storage interfaces (Container Storage Interface) that is an open source project.  SNIA is working on that with the Cloud Native Computing Foundation (CNCF) to ensure it works for storage vendors and applications. SOS: Why does the Storage Developer Conference agenda have a focus on orchestration?  MC: With all the applications and approaches out there, it’s important for developers to learn about orchestration and containers in a vendor neutral environment as a foundation for understanding and evaluation. That’s where SNIA’s Storage Developer Conference plays a big role.  Orchestration talks can be found in the agenda under the Cloud Storage, Storage Architecture, and Storage Management Tracks and feature speakers from Stanford University, HPE, DESY, Google, Portworx, Kasten, and NetApp. SOC:  How can I get up to speed on orchestration topics before attending SDC 2018? MC: SNIA has several great Brighttalk webcasts on containers that have thousands of views.  Check out these titles.

• Containers: Best Practices and Data Management Services 
• Containers and Persistent Memory

Also, check out this video on Containers and Persistent Memory from the 2018 SNIA Storage Developer Conference India. The 2017 SDC had a number of presentations on orchestration-related topics that are available via SNIA Video - and you can view the slides along with them.  Check under the Cloud Storage and Containers headings. Also check out the Storage Developer Conference Podcast Series, featuring 73 sessions from previous SDC events, with more arriving each month. SOS:  Does SNIA have a group focused on orchestration? MC:  SNIA’s Cloud Storage Technologies Initiative (CSTI) supports the evolving cloud business models and architectures, including OpenStack, software defined storage, Kubernetes, and object storage.  You can check out a new blog on CSTI here. SOC:  Great to chat with you, Mark, and look forward to our next dive – into NVMe!

The SNIA Cloud Storage Initiative (CSI) has had a rebrand; we’ve added a T for Technologies into our name, and we’re now officially the Cloud Storage Technologies Initiative (CSTI). That doesn’t seem like a significant change, but there’s a good reason. Our old name reflected the push to getting acceptance of cloud storage, and that specific cloud storage debate has been won, and big time. One relatively small cloud service provider is currently storing 400PB of clients’ data. Twitter alone consumes 300PB of data on Google’s cloud offering. Facebook, Amazon, AliBaba, Tencent – all have huge data storage numbers. Enterprises of every size are storing data in the cloud. That’s why we added the word “technologies.” The expanded charter and new name reflect the need to support the evolving cloud business models and architectures such as OpenStack, software defined storage, Kubernetes and object storage. It includes data services, orchestration and management, understanding hyperscale requirements and the role standards play. So what do we do? The CSTI is an active group that publishes articles and white papers, speaks at industry conferences and presents at highly-rated webcasts that have been viewed by thousands. You can learn more about the CSTI and check out the Infographic for highlights on cloud storage trends and CSTI activities. If you’re interested in cloud storage technologies, I encourage you to consider joining our group. We have multiple membership options for established vendors, startups, educational institutions, even individuals. Learn more about CSTI membership here.

We are increasingly living in a multi-cloud world, with potentially multiple private, public and hybrid cloud implementations supporting a single enterprise. Organizations want to leverage the agility of public cloud resources to run existing workloads without having to re-plumb or re-architect them and their processes. In many cases, applications and data have been moved individually to the public cloud. Over time, some applications and data might need to be moved back on premises, or moved partially or entirely, from one cloud to another. That means simplifying the movement of data from cloud to cloud. Data movement and data liberation – the seamless transfer of data from one cloud to another – has become a major requirement. On August 7, 2018, the SNIA Cloud Storage Technologies Initiative will tackle this issue in a live webcast, “Cloud Mobility and Data Movement.” We will explore some of these data movement and mobility issues and include real-world examples from the University of Michigan. We’ll discus:

• How do we secure data both at-rest and in-transit?
• What are the steps that can be followed to import data securely? What cloud processes and interfaces should we use to make data movement easier?
• How should we organize our data to simplify its mobility? Should we use block, file or object technologies?
• Should the application of the data influence how (and even if) we move the data?
• How can data in the cloud be leveraged for multiple use cases?

Register now for this live webcast. Our SNIA experts will be on-hand to answer you questions.  

We are increasingly living in a multi-cloud world, with potentially multiple private, public and hybrid cloud implementations supporting a single enterprise. Organizations want to leverage the agility of public cloud resources to run existing workloads without having to re-plumb or re-architect them and their processes. In many cases, applications and data have been moved individually to the public cloud. Over time, some applications and data might need to be moved back on premises, or moved partially or entirely, from one cloud to another. That means simplifying the movement of data from cloud to cloud. Data movement and data liberation – the seamless transfer of data from one cloud to another – has become a major requirement. On August 7, 2018, the SNIA Cloud Storage Technologies Initiative will tackle this issue in a live webcast, “Cloud Mobility and Data Movement.” We will explore some of these data movement and mobility issues and include real-world examples from the University of Michigan. We’ll discus:

• How do we secure data both at-rest and in-transit?
• Why is data so hard to move? What cloud processes and interfaces should we use to make data movement easier?
• How should we organize our data to simplify its mobility? Should we use block, file or object technologies?
• Should the application of the data influence how (and even if) we move the data?
• How can data in the cloud be leveraged for multiple use cases?

Register now for this live webcast. Our SNIA experts will be on-hand to answer you questions.    

In a recent survey of enterprise hybrid cloud users, the Evaluator Group saw that nearly 60% of respondents indicated that lack of interoperability is a significant technology issue that they must overcome in order to move forward. In fact, lack of interoperability was the number one issue, surpassing public cloud security and network security as significant inhibitors. The SNIA Cloud Storage Initiative (CSI) is pleased to have John Webster, Senior Partner at Evaluator Group, who will join us on December 12^th for a live webcast to dive into the findings of their research. In this webcast, Multi-Cloud Storage: Addressing the Need for Portability and Interoperability, my SNIA Cloud colleague, Mark Carlson, and John will discuss enterprise hybrid cloud objectives and barriers to adoption. John and Mark will focus on cloud interoperability within the storage domain and the CSI’s work that promotes interoperability and portability of data stored in the cloud. As moderator of this webcast, I’ll make sure we offer great insights on real-world cloud deployment challenges. As always, we will be available to answer your questions on the spot. I encourage you to register today. We hope to see you on the 12^th.