Curve Cloud Nativeabnormal detection, schedule tuningCloud native Feature list • Features for Cluster • Features for CurveBS • Features for CurveFS • Features for monitorFeature list for cluster • Install / uninstall/ run on arbitrary PVs(local storage) as an alternative to host path • Support automatically detect new nodes, adding / removing nodes and disk drives • Support dynamic volume resizing • chunkserver Dynamic Volume Provisioning for CurveFS • Dynamice volume resizing • Support automatically detect new nodes, adding / removing nodes • Allow CurveBS and S3 automatically grow underneath CurveFS when0 码力 | 9 页 | 2.85 MB | 6 月前3
Curve for CNCF Mainstorage system https://www.opencurve.io/Agenda • What is Curve • Use Cases • CurveBS • Key Features • Comparing to Ceph • CurveFS • Comparing to Ceph • Current Status • RoadmapWhat is Curve class for kubernete, nbd for HOST/VMPerformance (vs. Ceph RBD)Performance (vs. Ceph RBD)CurveBS Features • RAFT for data consistency • minor impaction when chunk server fails • Precreated chunk file meta overhead increase read/write magnification PERFORMANCE High Need to optimize rocksdbCurveFS Features • CurveFS can manage storages (open cloud storage and on-prem storage) and expose unified file0 码力 | 21 页 | 4.56 MB | 6 月前3
CurveFs 用户权限系统调研maximum size of read requests© XXX Page 11 of 33 * must be set both here *and* passed to fuse_session_new() * using the ``-o max_read=`` mount option. At some point * in the future, specifying the mount the default umask value for pam_umask and is used by # useradd and newusers to set the mode of the new home directories. # 022 is the "historical" value in Debian for UMASK # 027, or even 077, could be -b7ba0ef33531 Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery extent 64bit flex_bg sparse_super 0 码力 | 33 页 | 732.13 KB | 6 月前3
OID CND Asia Slide: CurveFSdistributed shared storage services ● Polardb for PostgreSQL provides database services Solution features ● seperation between storage and computation ● The call method is the same as that of the standalone Key designs used by CURVE � CURVE Roadmap � CURVE Community � You can take awayCURVE Roadmap ● Features ○ Supports RDMA and SPDK ○ Further reduce I/O latency and improve throughput ○ Support data lifecycle0 码力 | 24 页 | 3.47 MB | 6 月前3
Curve支持S3 数据缓存方案缓存层级分为fs->file->chunk->datacache 4层,通过inodeId找到file,通过index找到chunk,然后通过offset~len找到是否有合适的datacache或者new datacache。 对外接口 流程上对于读写缓存有影响的接口包括:write,read,releaseCache,flush,fssync。后面会详细介绍这些接口流程。这里不需要提供trun Adaptor), chunkCacheManager_(chunkCacheManager), chunkPos_(chunkPos), len_(len) { data_ = new char[len]; memcpy(data_, data, len); } virtual ~DataCache() { delete data_; 则一并获取 5. 如果有可写的DataCache,则调用Write接口将数据合并到DataCache中; ,加入到ChunkCacheManager的Map中。 如果没有可写的DataCache则new一个 5.完成后返回成功。 Read流程 1.根据请求offset,计算出对应的chunk index和chunkPos。将请求拆分成多个chunk的ReadChunk调用。 2.在ReadC0 码力 | 9 页 | 179.72 KB | 6 月前3
Raft在Curve存储中的工程实践集群的配置不可避免会发生变更,比如替换宕机的机器。 直接配置变更可能出现双主问题 • 共同一致(joint consensus) • 集群先切换到一个过渡的配置(old + new),一旦共同一 致已经被提交,系统切换到新的配置(new)。RAFT协议简介 日志压缩 • 日志会不断增长,占用空间 • 采用快照的方式压缩日志 • 在某个时间点,整个系统的状态都以快照的形式写入 到稳定的持久化存储中 done); void remove_peer(const PeerId& peer, Closure* done); void change_peers(const Configuration& new_peers, Closure* done); StateMachine void on_apply(::raft::Iterator& iter); void on_snapshot_save(SnapshotWriter*0 码力 | 29 页 | 2.20 MB | 6 月前3
Bazelcd /curve/ # 使用 gcc 编译 bazel build –config=gcc7-later … # use '-faligned-new' to enable C++17 over-aligned new support # 使用 clang 编译 CC=clang CXX=clang++ bazel build …2 制作镜像 docker run -v0 码力 | 6 页 | 4.69 MB | 6 月前3
CurveBS IO Processing Flowin advance. When users write data, they only need to select an appropriate CopySet and create a new chunk. Why use a CopySet to manage data? 1. Reduce metadata. If the replication group membership segments are allocated with a default size of 1GB (configurable >=1GB), so client requests MDS for new segment infrequently. For random write scenarios, it is possible to allocate space frequently at0 码力 | 13 页 | 2.03 MB | 6 月前3
CurveFS Client 概要设计*newname, unsigned int flags); rename有两种做法: 一是,向metaserver发起inode和dentry迁移,从parent迁移到new parent,并修改name为newname。 二是,在new parent创建新的inode和dentry,然后删除旧的parent下的inode和dentry 两者都涉及到rename的事务性的问题?( ) 这里可能还需0 码力 | 11 页 | 487.92 KB | 6 月前3
Curve Detail Introduction for CNCFRoadmap • MultiRaft optimization • ParallelRaft for write • Reduce write magnification for file new write • Cooperate with Alibaba to support high-performance polardb for postgresql using CurveBS0 码力 | 23 页 | 6.53 MB | 6 月前3
共 11 条
- 1
- 2













