up the live stream into batches of X seconds. • Treats each batch as RDDs and processes them using RDD operations. • Discretized Stream Processing (DStream) 7 / 79 Spark Streaming ▶ Run a streaming up the live stream into batches of X seconds. • Treats each batch as RDDs and processes them using RDD operations. • Discretized Stream Processing (DStream) 7 / 79 Spark Streaming ▶ Run a streaming up the live stream into batches of X seconds. • Treats each batch as RDDs and processes them using RDD operations. • Discretized Stream Processing (DStream) 7 / 79 DStream (1/2) ▶ DStream: sequence

吞吐 TB-PB TB-PB 未经⼤大规模⽣生产验证 性能 ⼀一般(分钟⼩小时级别) 快(秒级) 优秀 x2 稳定性 好 ⼀一般 已在阿⾥里里内部验证 API 差(MR) 最丰富 (RDD/DataSet/DataFrame) Python/Scala/R/Java 丰富 (TableAPI) Scala/Java SQL HiveSQL SparkSQL ANSI SQL