This talk describes work going on in three different organizations to enable scale out management of NVMe SSDs. The soon to be released NVME-MI 1.1 standard will allow management from host based agents as well as BMCs. This might be extended to allow support for Binary Encoded JSON (BEJ) in support of host agents and BMCs that want to support the Redfish Standard. We will also cover work going on in SNIA (Object Drive TWG) and DMTF in support. Learning Objectives: 1. Principles and limitations of scale out datacenter management 2. An understanding of the NVMe-MI standard 3.

One of the most significant challenges for software defined storage systems is to determine an appropriate tuning configuration considering diverse factors including: hardware, software stacks, and workload characteristics. Currently, the configuration of complex storage systems relies on the experience and skills of human experts. For storage companies, this creates bottlenecks on customer service time, since the number of experts is limited. Moreover, the tuning expertise can be diminished by continuous personal rotation and the lack of experience sharing.

More than a decade old data architecture isn’t enough for today’s data-driven businesses which are heavily dependent on AI/ML/DL. As the enterprises begin to operationalize these AI/ML workflows, they would need to optimize on storage I/O performance to feed to massively parallel GPU based compute. With growing IoT footprint, data management and AI/ML based compute challenges span across from edge, to the core and to the cloud.

Three emerging trends must be considered when assessing how storage should operate at extreme scale. First, continuing expansion in the volume of data to be stored is accompanied by increasing complexity in the metadata to be stored with it and queries to be executed on it. Second, ever increasing core and node counts require corresponding scaling of application concurrency while simultaneously increasing the frequency of hardware failure.

Mining interesting patterns through the knowledge discovery process is an area of growing interest. Incorporating knowledge and reasoning into existing network infrastructure could result in more efficient, smart and adaptive environments. Huge volumes of data collected on a large data center network presents interesting challenges for analyzing and identifying anomalies such as network misconfiguration and impending service outages.

Corporations and the global Open Source community have had a colorful relationship over the decades with each group struggling to understand the priorities, abilities and role of the other. Such relationships have ranged from hostile to prodigiously fruitful and have clearly resulted in the adoption of Open Source in virtually every aspect of computing. This talk will explore the qualities and precedences of strong Corporate/Open Source relationships and focus on the success of the OpenZFS enterprise file system project as a benchmark of contemporary success. I will explore:

There is a huge need to containerize the Stateful applications in today's world of Docker and Kubernetes. These stateful applications need a storage architecture that is truly cloud native. Container Attached Storage or CAS is a truly cloud native software architecture for applications running in containers. In CAS, the storage software itself is containerized and hence gets the advantages of being a micro service. In CAS architecture, each storage volume gets it's own storage controller running completely in user space and attains the maximum agility and policy granularity.

We are on the precipice of a new paradigm in the data center. Booming growth in both compute and storage have effectively changed how best to tie the traditional SW and HW layers together for scale, as well as which new SW and HW layers to use. The industry has answered the call with strategic changes in everything from storage device functionality, form factors and capacity points, to new compute platforms and the HW used to perform that compute. New offerings of open source HW and SW, tied together with new more functional and powerful networks help finish this new puzzle.

Persistent memory will significantly improve storage performance. But these benefits can be harder to realize in distributed storage systems such as Ceph or Gluster. In such architectures, several factors mitigate gains from faster storage. They include costly network overhead inherent for many operations, and deep software layers supporting necessary services. It is also likely that the high costs of persistent memory will limit deployments in the near term. This talk shall explore steps to tackle those problems.

The core activity within the newly created H2020 project “eXtreme DataCloud” is the policy-driven orchestrating of federated data management for data intensive sciences like High Energy Physics, Astronomy, Photon and Life Science. Well-known experts in the field are working on combining already established data management and orchestration tools to provide a highly scalable solution supporting the entire European Scientific Landscape.