Ozone on NVMe Wei-Chiu Chuang (jojochuang) Ritesh Shukla (kerneltime) Agenda • Overview of how Ozone and how it scales • Why NVME is important for Ozone for scaling • Benefits of using NVME • Impala Impala performance results from NVME clusters • Write path improvements results from NVME clusters • Summary • Questions Ozone Architecture Why does Ozone Scale? Separation of concerns Ozone Manager limitations, working set can be cached in memory and unused data can be destaged to disk • OM uses NVME to store RocksDBs • Future projects such as Snapshots leverage RocksDB to preserve simplicity of

bottlenecked by IO, meaning CPUs were constantly waiting on HDDs to respond. But when we replace HDDs with NVMe SSDs on a Dual Socket Best Practices for MySQL with SSDs Page 4 Server, the CPU becomes the Storage SAS HDD 2x SEAGATE ST600MP0005 15K rpm SATA SSD 2x Samsung 850 PRO NVMe SSD 2x Samsung XS1715 Quad‐socket (28 Core) Configuration Database Server (Client) 2x SEAGATE ST600MP0005 15K rpm SATA SSD 2x Samsung 850 Pro SAS SSD 2x Samsung PM1633 NVMe 2x Samsung PM1725 It is generally accepted that OLTP database applications have been I/O bound

Acceleration Kit (HSAK) improves the I/O performance of NVMe devices. The software library implements an I/O software stack of high-performance NVMe devices. It is advanced for its user mode, asynchronization lock-free, and polling features. Compared with the NVMe device I/O software stack in the native Linux kernel, the HSAK greatly reduces the latency of NVMe commands and improves the I/O processing capability capability (IOPS) of a single CPU. Challenges Evolving storage media such as NVMe solid state drives (SSDs) and storage class memory (SCM) have resulted in decreasing access latencies caused by the media

. . . . 81 5 Virtual storage 83 5.1 Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe . . . . . . 83 5.2 Disk image files (VDI, VMDK, VHD, HDD) . . . . . . . . . . . . . . . . . . . . Logic and Bus- Logic); see chapter 5.1, Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe, page 83 for details. Whereas providing one of these would be enough for VirtualBox by itself, this present initially. Please see chapter 5.1, Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe, page 83 for additional information. VirtualBox also provides a floppy controller, which is special:

. . . 81 5 Virtual storage 83 5.1 Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe . . . . . . 83 5.2 Disk image files (VDI, VMDK, VHD, HDD) . . . . . . . . . . . . . . . . . . . Logic and Bus- Logic); see chapter 5.1, Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe, page 83 for details. Whereas providing one of these would be enough for VirtualBox by itself, this present initially. Please see chapter 5.1, Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe, page 83 for additional information. VirtualBox also provides a floppy controller, which is special:

. . . 81 5 Virtual storage 83 5.1 Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe . . . . . . 83 5.2 Disk image files (VDI, VMDK, VHD, HDD) . . . . . . . . . . . . . . . . . . . Logic and Bus- Logic); see chapter 5.1, Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe, page 83 for details. Whereas providing one of these would be enough for VirtualBox by itself, this present initially. Please see chapter 5.1, Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe, page 83 for additional information. VirtualBox also provides a floppy controller, which is special:

. . . 81 5 Virtual storage 83 5.1 Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe . . . . . . 83 5.2 Disk image files (VDI, VMDK, VHD, HDD) . . . . . . . . . . . . . . . . . . . Logic and Bus- Logic); see chapter 5.1, Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe, page 83 for details. Whereas providing one of these would be enough for VirtualBox by itself, this present initially. Please see chapter 5.1, Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe, page 83 for additional information. VirtualBox also provides a floppy controller, which is special:

. . . 81 5 Virtual storage 83 5.1 Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe . . . . . . 83 5.2 Disk image files (VDI, VMDK, VHD, HDD) . . . . . . . . . . . . . . . . . . . Logic and Bus- Logic); see chapter 5.1, Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe, page 83 for details. Whereas providing one of these would be enough for VirtualBox by itself, this present initially. Please see chapter 5.1, Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe, page 83 for additional information. VirtualBox also provides a floppy controller, which is special:

. . . 81 5 Virtual storage 83 5.1 Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe . . . . . . 83 5.2 Disk image files (VDI, VMDK, VHD, HDD) . . . . . . . . . . . . . . . . . . . Logic and Bus- Logic); see chapter 5.1, Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe, page 83 for details. Whereas providing one of these would be enough for VirtualBox by itself, this present initially. Please see chapter 5.1, Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe, page 83 for additional information. VirtualBox also provides a floppy controller, which is special:

. . . 81 5 Virtual storage 83 5.1 Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe . . . . . . 83 5.2 Disk image files (VDI, VMDK, VHD, HDD) . . . . . . . . . . . . . . . . . . . Logic and Bus- Logic); see chapter 5.1, Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe, page 83 for details. Whereas providing one of these would be enough for VirtualBox by itself, this present initially. Please see chapter 5.1, Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe, page 83 for additional information. VirtualBox also provides a floppy controller, which is special: