beneficial? ??? Vasiliki Kalavri | Boston University 2020 • Use equivalence transformation rules if the language allows • selection operations are commutative • theta-join operations are commutative • natural | Boston University 2020 B 21 Profitability • Running two applications together on a single core, one with operators B and C, the other with operators B and D. Redundancy elimination B A B C Chromium/25.0.1364.160 Chrome/ 25.0.1364.160 Safari/537.22 Referer: https://www.google.be/ Accept-Language: en-US,en;q=0.8 Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3 GET /dumprequest HTTP/1.1 Host:

dianfu staff 295K 10 18 20:43 log4j-api-2.17.1.jar # -rw-r--r-- 1 dianfu staff 1.7M 10 18 20:43 log4j-core-2.17.1.jar # -rw-r--r-- 1 dianfu staff 24K 10 18 20:43 log4j-slf4j-impl-2.17.1.jar Please make sure [2]: Table Creation Table is a core component of the Python Table API. A Table object describes a pipeline of data transformations. It 1.1. Getting Started 19 pyflink-docs, Release release-1.15 DataStream Creation DataStream is a core component of the Python DataStream API. A DataStream object describes a pipeline of data transformations

dianfu staff 295K 10 18 20:43 log4j-api-2.17.1.jar # -rw-r--r-- 1 dianfu staff 1.7M 10 18 20:43 log4j-core-2.17.1.jar # -rw-r--r-- 1 dianfu staff 24K 10 18 20:43 log4j-slf4j-impl-2.17.1.jar Please make sure [2]: Table Creation Table is a core component of the Python Table API. A Table object describes a pipeline of data transformations. It 1.1. Getting Started 19 pyflink-docs, Release release-1.16 DataStream Creation DataStream is a core component of the Python DataStream API. A DataStream object describes a pipeline of data transformations

type, content, timing constraints. • Actions define how to produce results from the matches. Language Types 3 Vasiliki Kalavri | Boston University 2020 Three classes of operators: • relation-to-relation: portions of a stream. • relation-to-stream: create streams through querying tables Declarative language: CQL 4 Vasiliki Kalavri | Boston University 2020 Select IStream(*) From S1 [Rows 5], S2 [Rows τ> whenever tuple s is in R at time τ. 6 Vasiliki Kalavri | Boston University 2020 Imperative language: Aurora SQuAl Queries are represented in graphical representation using boxes and arrows Tumble

Boston University 2020 1. Process events online without storing them 2. Support a high-level language (e.g. StreamSQL) 3. Handle missing, out-of-order, delayed data 4. Guarantee deterministic (on Dataflow Streaming Relational Dataflow Input in-order out-of-order Results approximate exact Language SQL extensions, CQL Java, Scala, Python, SQL Execution centralized distributed Parallelism pipeline

2020 Apache Flink • An open-source, distributed data analysis framework • True streaming at its core • Streaming & Batch API Historic data Kafka, RabbitMQ, ... HDFS, JDBC, ... Event logs ETL,

attributes or meta-data. • Consumers subscribe to events by specifying filters in a subscription language. • Filters define constraints in the form of name-value pairs and basic comparison operators

challenging? 28 Vasiliki Kalavri | Boston University 2020 Using pseudocode (or the programming language of your choice), write a program that reads a stream of integers and computes: 29 1. the maximum