Lowest-cost plan ??? Vasiliki Kalavri | Boston University 2020 12 • What does efficient mean in the context of streaming? • queries run continuously • streams are unbounded • In traditional ad-hoc database resource availability: the host must have enough resources for all assigned operators • Ensure security constraints: what are the trusted hosts for each operator? • Ensure state migration: if placement

// window type @Override public void process( String key, Context context, Iterable events, Collector> out) if (event.value > max) max = event.value; } out.collect(new Tuple3<>(key, context.window().getEnd(), max)); } } Buffers all the events DataStream input = SensorReading>, String, TimeWindow> { @Override public void process( String key, Context context, Iterable maxReading, Collector>

window, an Iterable to access the elements of the window, and a Collector to emit results. • A Context gives access to the metadata of the window (start and end timestamps in the case of a time window) Evaluates the window void process( KEY key, Context ctx, Iterable vals, Collector out) throws Exception; public abstract class Context implements Serializable { // Returns processElement(v: IN, ctx: Context, out: Collector[OUT]) is called for each record of the stream. Result records are emitted by passing them to the Collector. The Context object gives access to the

. . . . 29 1.3.4.3 O3: NoSuchMethodError: org.apache.flink.table.factories.DynamicTableFactory$Context.getCatalogTable()Lorg/apache/flink/table/catalog/CatalogTable 30 1.3.5 Runtime issues . . . . . Release release-1.15 TableEnvironment Creation TableEnvironment is the entry point and central context for creating Table and SQL API programs. Flink is an unified streaming and batch computing engine DynamicTableFactory$Context.getCatalogTable()Lorg/apache/flink/table/catalog/CatalogTable java.lang.NoSuchMethodError: org.apache.flink.table.factories.DynamicTableFactory ˓→$Context.getCatalogTable()

. . . . 29 1.3.4.3 O3: NoSuchMethodError: org.apache.flink.table.factories.DynamicTableFactory$Context.getCatalogTable()Lorg/apache/flink/table/catalog/CatalogTable 30 1.3.5 Runtime issues . . . . . Release release-1.16 TableEnvironment Creation TableEnvironment is the entry point and central context for creating Table and SQL API programs. Flink is an unified streaming and batch computing engine DynamicTableFactory$Context.getCatalogTable()Lorg/apache/flink/table/catalog/CatalogTable java.lang.NoSuchMethodError: org.apache.flink.table.factories.DynamicTableFactory ˓→$Context.getCatalogTable()

: “SAQL: A Stream-based Query System for Real- Time Abnormal System Behavior Detection”, USENIX Security '18 12 Interested in a more research-oriented project?
 Let’s discuss it during office hours

id) Vasiliki Kalavri | Boston University 2020 Use keyed state to store and access state in the context of a key attribute: • For each distinct value of the key attribute, Flink maintains one state instance is called, Flink’s runtime automatically puts all keyed state objects of the function into the context of the key of the record that is passed by the function call. • A function can only access the