1 基于SPDK的CurveBS PFS存储引擎10/17/22 2 Why ●为了减少使用cpu做内存copy，减少系统调用 ●发挥某些被操作系统屏蔽的功能，例如nvme write zero ●根据阿里《When Cloud Storage Meets RDMA》的说法 ●在100Gbps网络带宽时，内存带宽成为瓶颈 ●Intel Memory Latency Checker (MLC)测试得到的CPU内存带宽是 ●读写内存都由网卡进行offload ●应用程序不再通过系统调用在内核和用户态来回切换10/17/22 4 磁盘的读写 ●基于EXT4的存储引擎，依然需要通过系统调用来回切换 ●读写都需要CPU拷贝数据 ●不能发挥某些NVME的功能，例如write zero10/17/22 5 为什么用PFS ●对代码比较熟悉 ●找一个能管理裸盘，具有产品级可靠性的代码挺难的 ●PFS支持类POSIX文件的接口，与使用EXT4的存储引擎代码很像， ●直接DMA读写，要求的内存必须是DPDK的hugetlb内存 ●必须符合NVME 内存读写地址对齐要求 ●offset 512对齐 ●为零copy提供接口10/17/22 10 BRPC IOBuf DMA ●修改BRPC，允许使用dpdk内存作为IOBuf的内存分配器 ●BRPC接收到的数据在IOBuf中，IOBuf直接使用于NVME DMA传输 ●使用IOBuf内存读nvme，避免自己写PRP页面对齐内存分配代码10/17/22

. . . . 81 5 Virtual storage 83 5.1 Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe . . . . . . 83 5.2 Disk image files (VDI, VMDK, VHD, HDD) . . . . . . . . . . . . . . . . . . . . Logic and Bus- Logic); see chapter 5.1, Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe, page 83 for details. Whereas providing one of these would be enough for VirtualBox by itself, this present initially. Please see chapter 5.1, Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe, page 83 for additional information. VirtualBox also provides a floppy controller, which is special:

. . . . 81 5 Virtual storage 83 5.1 Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe . . . . . . 83 5.2 Disk image files (VDI, VMDK, VHD, HDD) . . . . . . . . . . . . . . . . . . . . Logic and Bus- Logic); see chapter 5.1, Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe, page 83 for details. Whereas providing one of these would be enough for VirtualBox by itself, this present initially. Please see chapter 5.1, Hard disk controllers: IDE, SATA (AHCI), SCSI, SAS, USB MSD, NVMe, page 83 for additional information. VirtualBox also provides a floppy controller, which is special:

Polardb的方法 共享盘使用阿里云自带的高性能Nvme盘，注意使用Nvme磁盘对可用 区有要求： • 华东1（杭州）可用区I • 华东2（上海）可用区B • 华北2（北京）可用区K • 华南1（深圳）可用区F。 只有某些规格的虚拟机可以挂载Nvme共享盘： • g7se • c7se • r7se 虚拟机要求是按量付费才可以挂载Nvme共享盘 阿里云的VIP功能目前还在内侧阶段，需要申

基于在架构上的选择和优秀的工程实践，Curve 在性能、运维、稳定性、工程实践质量上都优于Ceph主要亮点 — 高性能 测试环境：3台服务器*8块NVME， Intel(R) Xeon(R) Gold 5318Y CPU @ 2.10GHz ，256G，3副本，使用自带fio 高性能 NVME 块存储场景，Curve随机读写性能远优于Ceph 单卷 多卷主要亮点 — 易运维 运维场景 Curve Ceph

命令卸载，从编辑 /etc/fstab 文件步骤开始执行，添加挂载参数重新挂载 即可。 1. # umount /dev/nvme0n1 下面以 /dev/nvme0n1 数据盘为例： 查看数据盘 1. # fdisk -l 2. Disk /dev/nvme0n1: 1000 GB 创建分区表 在部署目标机器上安装 NTP 服务 在部署目标机器上添加数据盘 ext4 文件系统挂载参数 Ansible 书栈(BookStack.CN) 构建 1. # parted -s -a optimal /dev/nvme0n1 mklabel gpt -- mkpart primary ext4 1 -1 格式化文件系统 1. # mkfs.ext4 /dev/nvme0n1 查看数据盘分区 UUID，本例中 nvme0n1 的 UUID 为 c51eb23b-195c-4061-92a9-3fad812cc12f。 f414c5c0-f823-4bb1-8fdf-e531173a72ed 6. └─sda3 ext4 547909c1-398d-4696-94c6-03e43e317b60 / 7. sr0 8. nvme0n1 ext4 c51eb23b-195c-4061-92a9-3fad812cc12f 编辑 /etc/fstab 文件，添加 nodelalloc 挂载参数 1.

to guarantee the correctness of the test result. • For the TiKV server, it is recommended to use NVMe SSDs to ensure faster reads and writes. • If you only want to test and verify the features, follow options on the target machines that deploy TiKV For production deployments, it is recommended to use NVMe SSD of EXT4 filesystem to store TiKV data. This configuration is the best practice, whose reliability running the umount /dev/nvme0n1p1 com- mand, skip directly to the fifth step below to edit the /etc/fstab file, and add the options again to the filesystem. Take the /dev/nvme0n1 data disk as an example:

to guarantee the correctness of the test result. • For the TiKV server, it is recommended to use NVMe SSDs to ensure faster reads and writes. • If you only want to test and verify the features, follow options on the target machines that deploy TiKV For production deployments, it is recommended to use NVMe SSD of EXT4 filesystem to store TiKV data. This configuration is the best practice, whose reliability running the umount /dev/nvme0n1p1 com- mand, skip directly to the fifth step below to edit the /etc/fstab file, and add the options again to the filesystem. Take the /dev/nvme0n1 data disk as an example:

to guarantee the correctness of the test result. • For the TiKV server, it is recommended to use NVMe SSDs to ensure faster reads and writes. • If you only want to test and verify the features, follow options on the target machines that deploy TiKV For production deployments, it is recommended to use NVMe SSD of EXT4 filesystem to store TiKV data. This configuration is the best practice, whose reliability running the umount /dev/nvme0n1p1 com- mand, skip directly to the fifth step below to edit the /etc/fstab file, and add the options again to the filesystem. Take the /dev/nvme0n1 data disk as an example:

to guarantee the correctness of the test result. • For the TiKV server, it is recommended to use NVMe SSDs to ensure faster reads and writes. • If you only want to test and verify the features, follow options on the target machines that deploy TiKV For production deployments, it is recommended to use NVMe SSD of EXT4 filesystem to store TiKV data. This configuration is the best practice, whose reliability running the umount /dev/nvme0n1p1 com- mand, skip directly to the fifth step below to edit the /etc/fstab file, and add the options again to the filesystem. Take the /dev/nvme0n1 data disk as an example: