doc 12 Chapter 1. How to build docs locally pyflink-docs, Release release-1.15 TableEnvironment Creation TableEnvironment is the entry point and central context for creating Table and SQL API programs API to create a TableEnvironment. TableEnvironment is responsible for: • Table management: Table Creation, listing Tables, Conversion between Table and DataStream, etc. • User-defined function management: create(env_settings) table_env [2]: Table Creation Table is a core component of the Python Table API. A Table object describes a pipeline of data

doc 12 Chapter 1. How to build docs locally pyflink-docs, Release release-1.16 TableEnvironment Creation TableEnvironment is the entry point and central context for creating Table and SQL API programs API to create a TableEnvironment. TableEnvironment is responsible for: • Table management: Table Creation, listing Tables, Conversion between Table and DataStream, etc. • User-defined function management: create(env_settings) table_env [2]: Table Creation Table is a core component of the Python Table API. A Table object describes a pipeline of data

runtime • It dynamically routes data after measuring which ordering is the most profitable Dynamic re-ordering with Eddy B A D C Eddy C D A B ??? Vasiliki Kalavri | Boston University 2020 18 Data-parallel streaming languages enable fission by construction • Elastic scaling techniques enable dynamic operator fission by adjusting the number of parallel operator instances according to data rates constraints: what are the trusted hosts for each operator? • Ensure state migration: if placement is dynamic and the operator is stateful, its state must be moved in a consistent manner Operator placement

millisBehindLatest > threshold 4.12 Monitoring Latency Generally speaking, latency is the delay between the creation of an event and the time at which results based on this event become visible. Once the event is practice, it has proven invaluable to add timestamps to your events at multiple stages (at least at creation, persistence, ingestion by Flink, publication by Flink; possibly sampling those to save bandwidth)

0, we receive one event: • Insert-only edge stream: events indicate edge additions • Fully-dynamic edge stream: events indicate edge additions or deletions A t+1, the graph is obtained by inserting nton.pdf • Stefani, Lorenzo De, et al. Triest: Counting local and global triangles in fully dynamic streams with fixed memory size. TKDD 2017. https://www.kdd.org/ kdd2016/papers/files/rfp0465-de-stefaniA

Stream Models Vasiliki Kalavri | Boston University 2020 A stream can be viewed as a massive, dynamic, one-dimensional vector A[1…N]. The size N of the streaming vector is defined as the product of negative. Events can be continuously inserted and deleted from the stream. It can model fully dynamic situations: • Monitoring active IP network connections is a Turnstile stream, as connections can

data when input rates increase beyond system capacity. • Load shedding techniques operate in a dynamic fashion: the system detects an overload situation during runtime and selectively drops tuples

load is Θ(ln n/ln ln n), with high probability ??? Vasiliki Kalavri | Boston University 2020 Dynamic resource allocation • Choose one among n workers • check the load of each worker and send the