JDBC, OBBC SQL ANSI SQL USERS FLEXIBLE DEPLOYMENT Local Storage Other RDBMSes Spark GemFire Cloud Object Storage HDFS JSON, Apache AVRO, Apache Parquet and XML Teradata SQL (Hyper-Q) 2017.thegiac.com Greenplum ⼤大数据平台 • 一次打包，到处运行：裸机、私有云、公有云 • 各种数据源：Hadoop、S3、数据库、文件、Spark、Ka,a • 各种数据格式：结构化、半结构化（JSON/XML/Hstore)、非结构化 • 强大内核: MPP、优化器、多态存储、灵活分区、高速加载、PG内核 • 强大的灵活性、 (100s) (1s) (10K s) (1M s) 可扩展性 – PageRank 性能 2017.thegiac.com MADlib vs. Spark: 不不同的产品，侧重点不不同 MADlib Spark 算法库 易用性 需要编程 查询优化 成熟度稍差 内存和流处理 通过 Gemfire SQL 语法支持 需要提升 磁盘数据 不是核心焦点

出来，几乎成为当前 Hadoop 开发使用的一个技术热点趋势。 这 些 技 术 包 括：Hive、Pivotal HAWQ、SPARK SQL、Impala、 Prest、Drill、Tajo 等等很多，这些技术有些是在 Mapreduce 上做 了优化。例如 Spark 采用内存中的 Mapreduce 技术，号称性能比 基于文件的的 Mapreduce 提高 10 倍；有的则采用 C/C++ 测试集 ,99 个 SQL）为例，包括 SPARK、Impala、Hive，只能支 持其中的 1/3 左右。 Big Date2.indd 15 16-11-22 下午3:38 16 由于 HADOOP 本身 Append-only 的特性，SQL-On-Hadoop 大多不 支持数据局部更新和删除功能 (update/delete)；例如 Spark 计算时， 需要预先将数据装载到 DataFrames 的架构更加灵活，存储节点和计算节点的无关 性，这体现在以下 2 个方面： ·扩展性方面 Hadoop 架构支持单独增加数据节点或计算节点，依托于 Hadoop 的 SQL-On-Hadoop 系统，例如 HAWQ、SPARK 均可单独增加 计算层的节点或数据层的 HDFS 存储节点，HDFS 数据存储对计算 层来说是透明的； MPP 数据库扩展时，一般情况下是计算节点和数据节点一起增加 的，在增加节点后，需要对数据做重分布才能保证数据与节点的紧

Structured Data JDBC, ODBC SQL ANSI SQL FLEXIBLE DEPLOYMENT Local Storage Other RDBMSes Spark GemFire Cloud Object Storage HDFS JSON, Apache AVRO, Apache Parquet and XML Teradata SQL Other

私有云 云数据库实现方案 ● 全新数据库 ○ Snowflake ● 原有数据库架构升级 ○ Vertica Eon Mode ● 容器化数据库+Kubernetes ○ Apache Spark ○ CockroachDB ○ Apache HAWQ 云数据库存储方案 ● 块存储 ○ 文件系统接口 ● 对象存储 ○ 成本低 ○ 扩展性强 ○ 访问延迟高 Greenplum

lock-in; this also impacts business agility • Incompatible with new big-data technologies (EMR, Spark, etc..) Added value from HVR • Rich replication (next slide) • Multiple SCD models (slides after

IT 人员 开发 人员 业务 分析师 数据 科学家 灵活 部署 数据源和数据管道 Spring Cloud Data Flow ETL 本地存储 HDFSS 云对象 存储 GemFire Spark 其他 RDBMSes 多结构数据 PIVOTAL GREENPLUM 平台 原生接口 分析应用 用户 JDBC、OBBC Teradata SQL Apache MADlib Python

version of Greenplum Database due to licensing restrictions. Support for data connectors: Greenplum-Spark Connector Greenplum-Informatica Connector Greenplum-Kafka Integration Greenplum Streaming Server Center Tanzu Greenplum Text Tanzu Greenplum Streaming Server Tanzu Greenplum Connector for Apache Spark Tanzu Greenplum Connector for Apache NiFi Tanzu Greenplum Connector for Informatica VMware Greenplum Connector for Apache Spark v1.6.2 - The Tanzu Greenplum Connector for Apache Spark supports high speed, parallel data transfer between Greenplum and an Apache Spark cluster using Spark’s Scala API. Greenplum

version of Greenplum Database due to licensing restrictions. Support for data connectors: Greenplum-Spark Connector Greenplum-Informatica Connector Greenplum-Kafka Integration Greenplum Streaming Server Center Tanzu Greenplum Text Tanzu Greenplum Streaming Server Tanzu Greenplum Connector for Apache Spark Tanzu Greenplum Connector for Apache NiFi Tanzu Greenplum Connector for Informatica VMware Greenplum Connector for Apache Spark v1.6.2 - The Tanzu Greenplum Connector for Apache Spark supports high speed, parallel data transfer between Greenplum and an Apache Spark cluster using Spark’s Scala API. Greenplum

due to licensing restrictions. Support for data connectors: Greenplum-NiFi Connector Greenplum-Spark Connector Greenplum-Informatica Connector Greenplum-Kafka Integration Greenplum Streaming Server 131 Tanzu Greenplum Text Tanzu Greenplum Streaming Server Tanzu Greenplum Connector for Apache Spark Tanzu Greenplum Connector for Apache NiFi Tanzu Greenplum Connector for Informatica VMware Tanzu Connector for Apache Spark v1.6.2 - The Tanzu Greenplum Connector for Apache Spark supports high speed, parallel data transfer between Greenplum and an Apache Spark cluster using Spark’s Scala API. Greenplum

Center Tanzu Greenplum Text Tanzu Greenplum Streaming Server Tanzu Greenplum Connector for Apache Spark Tanzu Greenplum Connector for Apache NiFi Tanzu Greenplum Connector for Informatica VMware Greenplum Connector for Apache Spark v1.6.2 - The Tanzu Greenplum Connector for Apache Spark supports high speed, parallel data transfer between Greenplum and an Apache Spark cluster using Spark’s Scala API. Greenplum recreate and repopulate the affected tables after restore. N/A Spark Connector This version of Greenplum is not compatible with Greenplum-Spark Connector versions earlier than version 1.7.0, due to a change