)ELETE WHEN MAT(HE) AN) +LA,<>H)H THEN UP)ATE a<EE = chan>=E.a<EE=E WHEN NOT MAT(HE) THEN .NSERT (GE=D.<, a<EE VALUES (chan>=E.GE=D.<, chan>=E.a<EE S1a2k + D+/4a CaCDC数据 1、仅依t S1a2k+D+/4a，架构简e。

O1: Scala Dependency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 1.3.1.2 O2: Java gateway process exited before sending its port number . . . . . . . . . . . 22 1.3.2 Usage issues InaccessibleObjectException: Unable to make field private final byte[] java.lang.String.value accessible: module java.base does not “opens java.lang” to unnamed module @4e4aea35 . . . . . . . . . . . . . . . 20:43 flink-shaded-zookeeper-3.5.9.jar # -rw-r--r-- 1 dianfu staff 15M 10 18 20:43 flink-table-api-java-uber-1.15.2.jar # -rw-r--r-- 1 dianfu staff 35M 10 18 20:43 flink-table-planner-loader-1.15.2.jar

O1: Scala Dependency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 1.3.1.2 O2: Java gateway process exited before sending its port number . . . . . . . . . . . 22 1.3.2 Usage issues InaccessibleObjectException: Unable to make field private final byte[] java.lang.String.value accessible: module java.base does not “opens java.lang” to unnamed module @4e4aea35 . . . . . . . . . . . . . . . 20:43 flink-shaded-zookeeper-3.5.9.jar # -rw-r--r-- 1 dianfu staff 15M 10 18 20:43 flink-table-api-java-uber-1.15.2.jar # -rw-r--r-- 1 dianfu staff 35M 10 18 20:43 flink-table-planner-loader-1.15.2.jar

Flink in a UNIX environment. • A Java 8.x installation. To develop Flink applications and use its DataStream API in Java or Scala you will need a Java JDK. A Java JRE is not sufficient! • Apache Maven Maven 3.x. • An IDE for Java and/or Scala development, such as IntelliJ IDEA (preferred), Eclipse, or Netbeans with appropriate plugins installed. • gsutil for accessing datasets in Google Cloud

T • ListState.add(value: T) • ListState.addAll(values: java.util.List[T]). • List State.get(): Iterable[T] • ListState.update(values: java.util.List[T]) Flink’s state primitives 13 Vasiliki Kalavri DataStream> connectedRides = rides
 .connect(fares)
 .flatMap(new MatchFunction());
 Java example 20 Vasiliki Kalavri | Boston University 2020 public static class EnrichmentFunction extends TaxiFare>> out) throws Exception {
 // similar logic for processing fare events
 }
 }
 } Java example (cont.) 21 Vasiliki Kalavri | Boston University 2020 List snapshotState(long checkpointId

x2 稳定性 好 ⼀一般 已在阿⾥里里内部验证 API 差(MR) 最丰富 (RDD/DataSet/DataFrame) Python/Scala/R/Java 丰富 (TableAPI) Scala/Java SQL HiveSQL SparkSQL ANSI SQL 易易⽤用性 ⼀一般 易易⽤用 ⼀一般 ⼯工具/⽣生态 ⼀一般 丰富 ⼀一般 Flink Batch应⽤用

execute("Compute max sensor temperature”)
 }
 } Flink programs are defined in regular Scala/Java methods Set up the execution environment: local, cluster, I/O, time semantics, parallelism t_out Run with a class entry point and arguments:
 ./bin/flink run -c org.apache.flink.examples.java.wordcount.WordCount \
 ./examples/batch/WordCount.jar \


types Type Examples Tuples Tuple1 through Tuple25 types. POJOs A POJO (plain old Java object) is any Java class that • has an empty default constructor • all fields are either ◦public, or ◦have

possibly different type A series of transformations on streams in Stream SQL, Scala, Python, Rust, Java… 40 Vasiliki Kalavri | Boston University 2020 Stateful operators Logic State <#Brexit Relational Dataflow Input in-order out-of-order Results approximate exact Language SQL extensions, CQL Java, Scala, Python, SQL Execution centralized distributed Parallelism pipeline pipeline, task, data

different type A series of transformations on streams in Stream SQL, Scala, Python, Rust, Java… ??? Vasiliki Kalavri | Boston University 2020 Logic State <#Brexit, 521> <#WorldCup