Service Mesh的延伸 之论道Database Mesh 分享人：张亮 日期：2018年07月25日Service Mesh风头正劲Service Mesh产品多样化Service Mesh的优势 云原生 零入侵 可观察性 面向运维服务化之后，数据库怎么办？ 服务 • 无状态 • 根据规则路由 • 业务方处理事务 数据库 • 有状态 • 根据SQL路由 • 数据库自动处理事务数据库的进化趋势 • SQL • ACID+BASE • 分布式 NewSQLNewSQL的分类 New Architecture Transparent Sharding Middleware Database-as-a-Service What's Really New with NewSQL?数据库中间层的优势 系统 •事务 运维 • DBA 开发 • SQL数据库中间层应具备的能力 Sidecar 数据库 任意 单一 单一 连接数 高 低 高 异构语言 仅Java 任意 任意 性能 损耗低 损耗略高 损耗低 无中心化 是 否 是 静态入口 无 有 无 Sidecar的优势Database Mesh架构图Sharding-Sphere 核心功能 数据分片 分布式事务 数据库治理 弹性伸缩 管控界面 实现方案 Sharding-JDBC Sharding-Proxy Sharding-Sidecar

traditional data processing applications, we know the entire dataset in advance, e.g. tables stored in a database. A data stream is a data set that is produced incrementally over time, rather than being available not know when the stream ends. 3 Vasiliki Kalavri | Boston University 2020 DW DBMS SDW DSMS Database Management System • ad-hoc queries, data manipulation tasks • insertions, updates, deletions develop space-efficient and time-efficient algorithms Vasiliki Kalavri | Boston University 2020 Relational Streaming Model Vasiliki Kalavri | Boston University 2020 Streams as evolving relations • A

objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 11.2 Database-style DataFrame joining/merging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 package providing fast, flexible, and expressive data structures designed to make working with “relational” or “labeled” data both easy and intuitive. It aims to be the fundamental high-level building block 1.2.1 New features • New unified merge function for efficiently performing full gamut of database / relational-algebra operations. Refactored existing join methods to use the new infrastructure, resulting

objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 11.2 Database-style DataFrame joining/merging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 package providing fast, flexible, and expressive data structures designed to make working with “relational” or “labeled” data both easy and intuitive. It aims to be the fundamental high-level building block 1.3.1 New features • New unified merge function for efficiently performing full gamut of database / relational-algebra operations. Refactored existing join methods to use the new infrastructure, resulting

objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 11.2 Database-style DataFrame joining/merging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 package providing fast, flexible, and expressive data structures designed to make working with “relational” or “labeled” data both easy and intuitive. It aims to be the fundamental high-level building block 1.4.1 New features • New unified merge function for efficiently performing full gamut of database / relational-algebra operations. Refactored existing join methods to use the new infrastructure, resulting

CHAPTER TWO EASE OF USE You only need to be familiar with Structured Query Language (SQL) and Java Database Connectivity (JDBC) to handle massive data. It helps you focus on the design and implementation Started With Kyuubi and DBeaver What is DBeaver DBeaver is a free multi-platform database tool for developers, database administrators, analysts and all people who need to work with databases. Supports DBeaver If you have successfully installed DBeaver, just hit the button to launch it. Select a database Substantially, this step is to choose a JDBC Driver type to use later. We can choose Apache Hive

CHAPTER TWO EASE OF USE You only need to be familiar with Structured Query Language (SQL) and Java Database Connectivity (JDBC) to handle massive data. It helps you focus on the design and implementation Started With Kyuubi and DBeaver What is DBeaver DBeaver is a free multi-platform database tool for developers, database administrators, analysts and all people who need to work with databases. Supports DBeaver If you have successfully installed DBeaver, just hit the button to launch it. Select a database Substantially, this step is to choose a JDBC Driver type to use later. We can choose Apache Hive

thrift client(cross-language support, both tcp and http), a Java Database Connectivity(JDBC) interface over thrift, or an Open Database Connectivity (ODBC) interface over a JDBC-to-ODBC bridge to communicate table storage layer via Hudi, Iceberg, or/and Delta Lake. • Logical data warehouse – Provide a relational abstraction on top of disparate data without ETL jobs, from collecting to connect- ing. 2.2 Run 968 INFO metastore.HiveMetaStore: 2: get_database: default 2020-11-16 23:50:52.968 INFO HiveMetaStore.audit: ugi=kentyao ip=unknown-ip- ˓→addr cmd=get_database: default 2020-11-16 23:50:52.970 INFO metastore

thrift client(cross-language support, both tcp and http), a Java Database Connectivity(JDBC) interface over thrift, or an Open Database Connectivity (ODBC) interface over a JDBC-to-ODBC bridge to communicate table storage layer via Hudi, Iceberg, or/and Delta Lake. • Logical data warehouse – Provide a relational abstraction on top of disparate data without ETL jobs, from collecting to connect- ing. 2.2 Run 968 INFO metastore.HiveMetaStore: 2: get_database: default 2020-11-16 23:50:52.968 INFO HiveMetaStore.audit: ugi=kentyao ip=unknown-ip- ˓→addr cmd=get_database: default 2020-11-16 23:50:52.970 INFO metastore

thrift client(cross-language support, both tcp and http), a Java Database Connectivity(JDBC) interface over thrift, or an Open Database Connectivity (ODBC) interface over a JDBC-to-ODBC bridge to communicate table storage layer via Hudi, Iceberg, or/and Delta Lake. • Logical data warehouse – Provide a relational abstraction on top of disparate data without ETL jobs, from collecting to connect- ing. 2.2 Run 968 INFO metastore.HiveMetaStore: 2: get_database: default 2020-11-16 23:50:52.968 INFO HiveMetaStore.audit: ugi=kentyao ip=unknown-ip- ˓→addr cmd=get_database: default 2020-11-16 23:50:52.970 INFO metastore