result: cs1 [copyset-1, copyset-7] cs2 [copyset-2, copyset-5] cs3 [copyset-3, copyset-6] cs4 [copyset-1, copyset-6] cs5 [copyset-2, copyset-8] cs6 [copyset-4, copyset-5] cs7 [copyset-1, copyset-5] cs8 [copyset-3 copyset-8] cs9 [copyset-4, copyset-7] cs10[copyset-2, copyset-7] cs11[copyset-3, copyset-6] cs12[copyset-4, copyset-8] round2: zone1 zone2 zone3 zone4 ------------------------ cs2 cs6 cs7 cs11 cs11 cs3 cs4 cs9 cs10 cs1 cs5 cs8 cs12 ------------------------ copyset-5: (cs2, cs6, cs7) copyset-6: (cs11, cs3, cs4) copyset-7: (cs9, cs10, cs1) copyset-8: (cs5, cs8, cs12)

ode 坏盘（CS1对应的盘）后的迁移流程 初始状态，copyset1，copyset2，copyset3的三个副本分别在 CS1,CS3,CS4上，完成迁移后，CS1上的副本迁移到CS2上 ① CS1超时未向MDS上报心跳（默认半小时） ② MDS标记CS1状态为offline ③ MDS的recover scheduler发现copyset1, 2, 3的副本CS1 offline， from CS1 to CS2的operator给这三个copyset ④ MDS通过RPC在CS2上创建copyset1,2,3这三个copyset ⑤ 假定三个copyset的leader都是CS3，在CS3的下一次心跳的 response中，下发第三步生成的三个operator ⑥ CS3收到change peer from CS1 to CS2的operator，给CS2同步 r raft日志，当CS2成功赶上进度时，本次raft成员变更成功完成， CS2成为了复制组的一员，CS1不再属于这个复制组。 ⑦ CS3在下一次心跳中向MDS报告本次raft成员变更已完成 ⑧ MDS在得知CS1上的所有copyset都成功迁移后，把CS1设置为 retired，CS1下线完毕。ChunkServer核心模块-CopysetNode 换盘（CS1对应的盘）后重新上线的流程

物理池：poolIO流程 client MDS leader Chunk server 1、发起请求 2、查询元数据 5、返回结果 5、返回结果 user 3、查询leader节点 4、向leader cs发起请求 1. 用户发起请求； 2. Client 向 mds 查询请求的元数据， 并缓存到本地，请求转换为对 chunk 的请求 3. Client 向 chunkserver 查询 chunk 精确的流量控制，对io几乎无影响自治 • 集群负载和资源均衡 • leader copyset scatter-width • 无需人工干预 • 对io影响几乎无影响易运维 • 升级秒级影响 • 客户端采用CS架构 • NEBD Client: 对接上层业务 • NEBD Server: 接受请求 调用Curve Client处理 • 升级只需重启Server 秒级影响易运维 • 丰富的metric体系

community Our awesome community members! Alibaba for hosting the meetupTVM Conference 2019 https://sampl.cs.washington.edu/tvmconf/ Thu, December 5th 2019, Seattle. 2 days before NeurIPS, Vancouver is 3h

热升级之前，QEMU是直接链接curve-client， 所以client版本需要升级时，需要对QEMU进 程进行重启。NEBD 整体介绍 在QEMU和Curve Client中间加入热升级模块，避 免直接依赖 热升级模块是CS结构：  NEBD Client(part1)：只包含轻量的业务逻辑， 以链接库的形式提供给QEMU使用  NEBD Server(part2)：将NEBD Client的请求转 发到Curve

• 如果源卷在s3上，则location格式为objectName@s3，例如: objectxxx@s3 • 如果源卷在curve内部，则location格式为 fileName:offset@cs，例如: /test1:0@csCHUNKSERVER端克隆实现-读时复制原理 • 使用chunkfile的bitmap来标记写过的Page， 一个Page大小为4KB • 读请求到来时，根据bitmap中的信息，

25/33背景 01 02 03 04 Curve质量控制 Curve监控体系 Curve运维体系Curve运维特性 易部署 • 批量配置、批量部署 • 操作简单 易升级 • 客户端采用CS架构，升级只需重启服务，秒级影响 • MDS与ChunkServer支持滚动升级 自治 • 自动均衡 • 自动故障恢复 27/33易部署 准备安装 包 配置用户 配置SSH 免密 安装

%rax # imm = 0x9502F9 addq $-2441406, %rax # imm = 0xFFDABF42 Source line: 5 popq %rbp retq nopw %cs:(%rax,%rax) Because the compiler knows that integer addition and multiplication are associative and Broadcast.combine_axes(1, 1, 1) () source Base.Broadcast.combine_styles – Function. combine_styles(cs...) -> BroadcastStyle Decides which BroadcastStyle to use for any number of value arguments. Uses RandomDevice: for OS-provided entropy. This may be used for cryptographically secure random numbers (CS(P)RNG). • MersenneTwister: an alternate high-quality PRNG which was the default in older versions

%rax # imm = 0x9502F9 addq $-2441406, %rax # imm = 0xFFDABF42 Source line: 5 popq %rbp retq nopw %cs:(%rax,%rax) Because the compiler knows that integer addition and multiplication are associative and Broadcast.combine_axes(1, 1, 1) () source Base.Broadcast.combine_styles – Function. combine_styles(cs...) -> BroadcastStyle Decides which BroadcastStyle to use for any number of value arguments. Uses RandomDevice: for OS-provided entropy. This may be used for cryptographically secure random numbers (CS(P)RNG). • MersenneTwister: an alternate high-quality PRNG which was the default in older versions

%rax # imm = 0x9502F9 addq $-2441406, %rax # imm = 0xFFDABF42 Source line: 5 popq %rbp retq nopw %cs:(%rax,%rax) Because the compiler knows that integer addition and multiplication are associative and Broadcast.combine_axes(1, 1, 1) () source Base.Broadcast.combine_styles – Function. combine_styles(cs...) -> BroadcastStyle Decides which BroadcastStyle to use for any number of value arguments. Uses RandomDevice: for OS-provided entropy. This may be used for cryptographically secure random numbers (CS(P)RNG). • MersenneTwister: an alternate high-quality PRNG which was the default in older versions