Curve for CNCF Mainqueue design • Memory zero copy design • Cloud native supportCloud native for CurveBS • CSI plugin for CurveBS • Deploy CurveBS as container service (in Plan) • Config CurveBS by (Cluster and data consistency • POSIX-compatiable • Cloud native supportCloud native plan for CurveFS • CSI plugin for CurveFS (in Plan) • Deploy CurveFS as container service (in Plan) • Config CurveFS by (cluster POSIX-compatible and mountable • Cache support on CurveFS • CurveFS cloud native support • csi plugin for CurveFS • support operator capability level 2: automated application provisioning and configuration0 码力 | 21 页 | 4.56 MB | 6 月前3
Curve Cloud Nativeand ETCD server • Replication between CurveFS configured with CRDsFeature list for CurveBS • CSI plugin for CurveBS • CurveBS (Cluster and Pool CRDs) • chunk server pods can run on arbitrary PVs(local chunkserver on PVCs to automatically grow when the cluster is nearly fullFeature list for CurveFS • CSI plugin for CurveFS • CurveFS (cluster and storage pools) CRDs • Dynamic Volume Provisioning for CurveFS0 码力 | 9 页 | 2.85 MB | 6 月前3
Open Flags 调研O_TMPFILE libfuse open void(* fuse_lowlevel_ops::open)(fuse_req_t req, fuse_ino_t ino, struct fuse_file_info *fi)© XXX Page 6 of 23 Open a file Open flags are available in fi->flags. The following rules apply opened. See fuse_file_info structure infor more details. If this request is answered with an error code of ENOSYS and FUSE_CAP_NO_OPEN_SUPPORT is set in fuse_conn_info.capable, this is treated FastCFS处理方式是自定义FileInfo保存文件打开的状态信息,在create()、open()、opendir()操作时填充进 fuse_file_info结构中,在后续操作中直接使用: struct fuse_file_info { /** Open flags. Available in open() and release() */ int flags; /** In case 0 码力 | 23 页 | 524.47 KB | 6 月前3
CurveFS对接S3方案设计4M}。 接口和关键数据结构 common.proto enum FSType { TYPE_VOLUME = 1; TYPE_S3 = 2; } message S3Info { required string ak = 1; required string sk = 2; required string endpoint = 3; required 2; required FSType fsType = 3; optional common.Volume volume = 4; optional common.S3Info s3Info = 5; } message FsInfo { required uint32 fsId = 1; required string fsName = 2; required required FSType fsType = 9; optional common.Volume volume = 10; optional common.S3Info s3Info = 11; } client端数据结构© XXX Page 5 of 11 // clients3 class S3ClientAdaptor { public: S3ClientAdaptor()0 码力 | 11 页 | 145.77 KB | 6 月前3
CurveFS Client 概要设计+readdirplus +copy_file_range +lseek 关键接口分析 init void (*init) (void *userdata, struct fuse_conn_info *conn); 根据挂载信息,从mds获取文件系统信息(或superblock),块分配器(bitmap)和root inode所在的copyset、 metaserver ip等信息 去me (*write) (fuse_req_t req, fuse_ino_t ino, const char *buf, size_t size, off_t off, struct fuse_file_info *fi); 首先根据inode id 从缓存中查找到对应inode结构; 如果inode缓存中不存在对应的inode,则从mds获取inode所在copyset,metaserver ip等信 of 11 read void (*read) (fuse_req_t req, fuse_ino_t ino, size_t size, off_t off, struct fuse_file_info *fi); 首先根据inode id 从缓存中查找到对应inode结构; 如果inode缓存中不存在对应的inode,则从mds获取inode所在copyset,metaserver ip等信0 码力 | 11 页 | 487.92 KB | 6 月前3
CurveFS S3本地缓存盘方案1024*4; LOG(INFO) << "whs start write."; for(int i = 0; i < 100000; i++) { char str[5]; sprintf(str, " %d" , i); Write(str, buf, length); } LOG(INFO) << "whs end write 如上,写10万个4k文件,最终耗时5秒左右,如下: hzwuhongsong@pubbeta1-nostest2:/mnt$ sudo cat /data/log/curve/libcurve-cb6f5cfd.log.INFO.20210825-171813.1930722 | grep whs | egrep "start|end" I 2021-08-25T17:18:13.642961+0800 19307220 码力 | 9 页 | 150.46 KB | 6 月前3
CurveFs 用户权限系统调研*(*init) (struct fuse_conn_info *conn); // libfuse include/fuse_common.h // Capability flags that the filesystem wants to enable. // unsigned want; struct fuse_conn_info { /** * Major version of tmp/fsmount# setfacl -m u:wanghai01:rw- file setfacl: file: Operation not supported # mount debug info ...© XXX Page 32 of 33 unique: 69, opcode: GETXATTR (22), nodeid: 2, insize: 72, pid: 2081159© XXX0 码力 | 33 页 | 732.13 KB | 6 月前3
BRPC与UCX集成指南开源项目 ●支持RDMA,TCP,Shared memory等 ●能透明支持多个链路传输,例如多网卡bond ●编译成.so或lib的方式,可以集成到应用程序里 ●有完善的配置功能,ucx_info可以dump配置信息 ●有性能测试工具 ●比较详细的文档2223 UCS ●是一些工具代码,例如 –链表 –hash table –epoll event loop – memory0 码力 | 66 页 | 16.29 MB | 6 月前3
CurveFS Copyset与FS对应关系+ 256 = 464B; volume file类型的inode,按照1w条extent估算,占用内存208 + 10000*56 = 560208B; s3 类型的inode,按照1w条s3info估算,占用内存208 + 10000*64 = 640208B; 8.1 一台机器上能存放多少个inode和dentry 由于元数据全部缓存在本地,而且磁盘空间远大于内存空间,所以一台机0 码力 | 19 页 | 383.29 KB | 6 月前3
Curve元数据节点高可用leaderElection->CampaginLeader() || false == leaderElection->LeaderKeyExist()) { LOG(INFO) << leaderElectionOp.leaderUniqueName << " campaign for leader agin"; } leader0 码力 | 30 页 | 2.42 MB | 6 月前3
共 10 条
- 1













