 Performance of Apache Ozone on NVMerestart (minimal caching) • Warp test to measure network saturation when using S3 • Impala TPCDS benchmark • Ratis streaming performance tests How much does disk read cost with NVME? Impala TPCDS Why datanode instead of the Leader ● Netty zero buffer copy ○ No gRPC buffer problem Ratis streaming Benchmark – Observation ● V2 Streaming multiple-client cases can be ~3x of V1 Async ○ Streaming can use0 码力 | 34 页 | 2.21 MB | 1 年前3 Performance of Apache Ozone on NVMerestart (minimal caching) • Warp test to measure network saturation when using S3 • Impala TPCDS benchmark • Ratis streaming performance tests How much does disk read cost with NVME? Impala TPCDS Why datanode instead of the Leader ● Netty zero buffer copy ○ No gRPC buffer problem Ratis streaming Benchmark – Observation ● V2 Streaming multiple-client cases can be ~3x of V1 Async ○ Streaming can use0 码力 | 34 页 | 2.21 MB | 1 年前3
 What's New In Apache Ozone 1.3所有已存在的桶,升级后变成LEGACY 版本,以⽀持向后兼容 存储Key格式基本同OBS, 通过配置项区分偏向⽂件,还是偏向S3对象的⽀持 在Bucket 创建时指定Layout,后续不⽀持更改 17 Benchmark Query Details: Dropped “catelog_sales” table with sub- paths(files/dirs) count = 5K Query Completion ~= 16 million 数据块 ⾜够! • 16TB HDD, 极⼩⽂件场景,数据块平均1KB, 总共 16 billion 数据块 有挑战! 21 Micro Benchmark Container 数量上升 I. V2 性能明显下降 II. V3 性能波动不⼤(除Container删除操作) Single node(96C 250G), 10 disks(3TB)0 码力 | 24 页 | 2.41 MB | 1 年前3 What's New In Apache Ozone 1.3所有已存在的桶,升级后变成LEGACY 版本,以⽀持向后兼容 存储Key格式基本同OBS, 通过配置项区分偏向⽂件,还是偏向S3对象的⽀持 在Bucket 创建时指定Layout,后续不⽀持更改 17 Benchmark Query Details: Dropped “catelog_sales” table with sub- paths(files/dirs) count = 5K Query Completion ~= 16 million 数据块 ⾜够! • 16TB HDD, 极⼩⽂件场景,数据块平均1KB, 总共 16 billion 数据块 有挑战! 21 Micro Benchmark Container 数量上升 I. V2 性能明显下降 II. V3 性能波动不⼤(除Container删除操作) Single node(96C 250G), 10 disks(3TB)0 码力 | 24 页 | 2.41 MB | 1 年前3
 Ozone meetup Nov 10, 2022 Ozone User Group Summitrestart (minimal caching) • Warp test to measure network saturation when using S3 • Impala TPCDS benchmark • Ratis streaming performance tests Software under test CDP Private Cloud Base 7.1.8 + • IMPALA-114570 码力 | 78 页 | 6.87 MB | 1 年前3 Ozone meetup Nov 10, 2022 Ozone User Group Summitrestart (minimal caching) • Warp test to measure network saturation when using S3 • Impala TPCDS benchmark • Ratis streaming performance tests Software under test CDP Private Cloud Base 7.1.8 + • IMPALA-114570 码力 | 78 页 | 6.87 MB | 1 年前3
共 3 条
- 1













