HBase Read Path openinx@apache.org Abstract ❏ Client Side ❏ Server Side ❏ Tuning Part-1 Client Side HBase Client ClientScanner ClientScanner cache(queue) scanner.next() RegionServer-0 RegionServer-1 (old generation) ● Less mixed GC(s) and shorter STW time. End-to-end offheap on the read-path (HBASE-11425) BucketCache StoreFileScanner Copy the Block from BucketCache(offheap) to onheap. Rpc Handler accumulate multiple results until reach max result size even if reach batch limit ○ Related issue: HBASE-21206 ● BlockSize ? Part-3 Tuning Tuning ● Read Distribution ● Locality ● Short Circuit Read

HBase Practice At Xiaomi huzheng@xiaomi.com About This Talk ● Async HBase Client ○ Why Async HBase Client ○ Implementation ○ Performance ● How do we tuning G1GC for HBase ○ CMS vs G1 ○ Tuning Tuning G1GC ○ G1GC in XiaoMi HBase Cluster Part-1 Async HBase Client Why Async HBase Client ? Request-1 Response-1 Request-2 Response-2 Request-3 Response-3 Request-4 Response-4 Request-1 66% Availability: 0% Why Async HBase Client ? ● Region Server / Master STW GC ● Slow RPC to HDFS ● Region Server Crash ● High Load ● Network Failure BTW: HBase may also suffer from fault amplification

HBase Practice At XiaoMi tianjy1990@gmail.com openinx@apache.org Part-1 Problems In Practice Problems in XiaoMi ❏ Problem 1. How to satisfy the regular demand of scanning table without affecting analysis need to scan a large number of data from hbase ❏ They are executed by mapreduce or spark, that put a heavy burden on HBase Scan snapshot directly ❏ HBase already provides this feature: TableSnapshotInputFormat TableSnapshotInputFormat (ClientSideRegionScanner) ❏ Construct regions by snapshot files ❏ Read data without any HBase RPC requests ❏ Required READ access to reference files and HFiles Snapshot ACL ❏ HDFS ACL could

HBase基本介绍 ⽥田志鹏 20190714 上次分位点估算当时没解决的两个问题已更更新ppt. 今天讲的内容⽐比较基础, ⽽而且偏理理论, 因为我个⼈人也没有太多实际使⽤用经验, 纸上谈兵. Apache HBase™ is the Hadoop database, a distributed, scalable, big data store. Use Apache HBase™ clusters of commodity hardware. Apache HBase is an open-source, distributed, versioned, non-relational database modeled after Google's Bigtable … 先来⼀一段HBase官⽹网的⾃自我介绍. blabla翻译⼀一下 重点看其中的红字, 什什么hadoop数据库 像redis是存kv结构的数据, MongoDB是存储⽂文档型数据, 那HBase存什什么样的数据? • ’表/⾏行行/列列’ • Row Key • ColumnFamily列列族 : ColumnQualifier列列限定名 • Version/Timestamp 分数:语⽂文 数据模型 逻辑视图 整个HBase和关系数据库很像, 但⼜又要时时注意两者的区别. 右⾯面我继续以⼀一次考试学⽣生分数距离

Postgres Conference China 2016 中国用户大会 HBase最佳实践及优化 陈飚 cb@cloudera.com Cloudera Postgres Conference China 2016 中国用户大会 关于我… 陈飚 Cloudera售前技术经理、资深方案架构师 http://biaobean.pro 原Intel Hadoop发行版核心开发人员, 成功实施并运维多 产品开发及方案顾问，先后负责Hadoop 产品 化、HBase 性能调优，以及行业解决方案顾问 2 Postgres Conference China 2016 中国用户大会 HBase的历史 2006年 Google发表 了BigTable 论文 2006年底由 PowerSet 的 Chad Walters和 Jim Kellerman 发起了HBase 项目，依据 BigTable的论文 重构关系数据 重构关系数据 库 2007年2月建立 了HBase的原型 版本 2007年10月建立 了第一个可用的 HBase版本 2008年成为 Apache Hadoop 的一个子项目 3 HBase是Google BigTable的开源实现 • BigTable利用GFS作为其文件存储系统 • HBase使用HDFS作为其文件存储系统 Postgres Conference China 2016

HBASE-21879 Read HFile ’s Block into ByteBuffer directly. 1. Background For reducing the Java GC impact to p99/p999 RPC latency, HBase 2.x has made an offheap read and write path. The KV are allocated Case In above pictures, the p999 latency is almost the same as G1GC STW cost (~100ms). After HBASE-11425​ , almost all memory allocations should be in the offheap, there should be rarely heap allocation As the basic idea part said, the first thing is to design a global ByteBuffAllocator. In ​HBASE-11425​ , we have introduced an offheap memory management policy as following: 1. Set a max memory