4M/1G（待定），chunk内部会划分为多个block，每个block最大4M，每个block对应s3上一个object。 s3上对象已chunkid_indexblock_version进行命名，元数据则已S3ChunkInfo（见数据结构）的方式存储在inode中。对于文件顺序写场景，文件0~4M的s3对象必然为chunkid_0_0,4M~8M为chunkid_1_0，以此类推， 还有一种情况是文件先写了0~2M，然后在写2M~4M，这里会采用append到同一个对象的方式进行写，而不是额外upload到一个新的对象；元数据则为{2,0,0,8M}。对于覆盖写，为了区分新老数据，则会对version进 行++，比如覆盖写了0~4M，则数据会写到chunkid_0_1的对象，则元数据包含了2个S3Chunkinfo{2,0,0,8M}和{2,1,0,4M}。 接口和关键数据结构 common of 11 // inodes3chunk message S3ChunkInfo { required uint64 chunkId = 1; required uint64 version = 2; required uint64 offset = 3; required uint64 len = 4; // file logic length required

This is the first request sent by the kernel to the daemon. It is used to negotiate the protocol version and other filesystem parameters. struct fuse_init_in { uint32_t major; uint32_t minor; uint32_t file operations and as a parameter to the * destroy() method. * * Introduced in version 2.3 * Changed in version 2.6 */ void *(*init) (struct fuse_conn_info *conn); // libfuse include/fuse_common // unsigned want; struct fuse_conn_info { /** * Major version of the protocol (read-only) */ unsigned proto_major; /** * Minor version of the protocol (read-only) */ unsigned proto_minor;

[s3chunkinfo(s)] s3chunkinfo { chunkid version // write always 0, compact will increase it offset len } s3 object命名: chunkid_version_index (index为obj在chunk内的index) 执行步骤 数据整理作为一个后台服务(线程池) ，compaction+1，chunkid为上一步获取的chunkid，为需要新增的obj - 老的obj为全部需要删除的部分 应用变更 - 先读写新增的s3 objects列表, 由于新增了version字段, 不会涉及到覆盖老的对象 - 加锁, 增量的更新inode的s3chunkinfolist, 保证原子更新, 更新失败回退新增数据 - 等待N秒, 保证mds已经告知client缓存失效,

proto mds.proto /* * Copyright (c) 2020 NetEase Inc. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You metaserver.proto /* * Copyright (c) 2020 NetEase Inc. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You proto space.proto /* * Copyright (c) 2020 NetEase Inc. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You

Support Operator capability level 5 (in Plan) • now support helmCurrent Status • Release 2 major version on CurveBS • v1.2 supporting QOS, Discard, data silent check • v1.3 some performance optimization capability level 2: automated application provisioning and configuration management and patch and minor version upgrads supported • File meta data preallocate • RAFT optimization • ParallelRaft for write

SnapFile的命名方式为“chunk_” + ChunkId + “_snap_”+ seqNum的形式，以区别于 ChunkFile。CHUNKSERVER端快照实现-SNAPFILE 字段 类型 说明 version uint8_t 文件格式协议 版本号 demaged bool 损坏标记 sn uint64_t 快照版本号 bits uint32_t 位图的位数 bitmap char[] 位图 安装元数据时即分配好chunk。 无额外接口： 无需Flatten接口。 适用场景： 适用于从云主机或快照创建镜像CHUNKSERVER端克隆实现-CHUNKFILE 字段 类型 说明 version uint8_t 文件格式协议版本号 sn uint64_t chunk文件的版本号 correntSn uint64_t chunk的修正版本号 locationSi ze size_t

12 Raft Snapshot +---------+---------+------+-----------------+-----+-----------+ | CURVEFS | version | size | key_value_pairs | EOF | check_sum | +---------+---------+------+-----------------+-----+-----------+ ---------+ 持久化文件 字段 字节数 说明 CURVEFS 7 magic number（常量字符 "CURVEFS"），用于标识该文件为 curvefs 元数据持久化文件 version 4 文件版本号（当文件格式变化时，可以 100% 向后兼容加载旧版持久化文件） size 8 键值对数量 key_value_pairs / 键值对（当 size 为 0 时，该字段为空）

tag=harbor.cloud.netease.com/curve/curvefs:chengyi01-debian113 修改 & 单元测试 curvefs/test/tools/curvefs_version_tool_test.cpp # 使用 clang 编译 CC=clang CXX=clang++ bazel build curvefs/...THANK YOU

application provisioning and configuration management SEAMLESS UPGRADES Y (by Helm) patch and minor version upgrads supported FULL LIFECYCLE Plan to Support app lifecycle, storage lifecycle(backup, failure

S3ChunkInfoList> s3ChunkInfoMap = 14; // TYPE_S3 only, first is chunk index optional uint64 version = 15; } class ClientS3Adaptor { public: ClientS3Adaptor () {} void Init(const S3ClientAdaptorOption