I1 O1 N’i update checkpoint send state ??? Vasiliki Kalavri | Boston University 2020 How can we make sure that checkpoints are meaningful and coherent? 7 ??? Vasiliki Kalavri | Boston University MHL4tF8FQSEdRb0YvHCsYWmhA20m7dLMJuxuhFz8K148qHj1b3jz35ikOWjrg4G3782wMy9IOFPasr6NpeWV1bX1xkZ zc2t7Z9fc239QcSopODTmsewHRAFnAhzNId+IoFEAYdeMLkp/d4jSMVica+nCXgRGQkWMkp0IfnmocuJGHAkPiZyLErq2fTN1tW26qAF4ldkxaq MHL4tF8FQSEdRb0YvHCsYWmhA20m7dLMJuxuhFz8K148qHj1b3jz35ikOWjrg4G3782wMy9IOFPasr6NpeWV1bX1xkZ zc2t7Z9fc239QcSopODTmsewHRAFnAhzNId+IoFEAYdeMLkp/d4jSMVica+nCXgRGQkWMkp0IfnmocuJGHAkPiZyLErq2fTN1tW26qAF4ldkxaq

. . . . . . . . . . . . . . . . . . . . . 25 1.3.3.1 O1: InaccessibleObjectException: Unable to make field private final byte[] java.lang.String.value accessible: module java.base does not “opens java refer to Build PyFlink. Check the installed package You could then perform the following checks to make sure that the installed PyFlink package is ready for use: curl -L https://raw.githubusercontent. log4j-core-2.17.1.jar # -rw-r--r-- 1 dianfu staff 24K 10 18 20:43 log4j-slf4j-impl-2.17.1.jar Please make sure that the versions of all the Flink jar packages are consistent, e.g. 1.15.2 in the above example

. . . . . . . . . . . . . . . . . . . . . 25 1.3.3.1 O1: InaccessibleObjectException: Unable to make field private final byte[] java.lang.String.value accessible: module java.base does not “opens java refer to Build PyFlink. Check the installed package You could then perform the following checks to make sure that the installed PyFlink package is ready for use: curl -L https://raw.githubusercontent. log4j-core-2.17.1.jar # -rw-r--r-- 1 dianfu staff 24K 10 18 20:43 log4j-slf4j-impl-2.17.1.jar Please make sure that the versions of all the Flink jar packages are consistent, e.g. 1.15.2 in the above example

https://www.da-platform.com/blog/manage-rocksdb-memory-size-apache-flink? __hstc=216506377.c9dc814ddd168ffc714fc8d2bf20623f. 1550652804788.1550652804788.1550652804788.1&__hssc=216506377.3.1551426921706&__hsfp=3017175250

2/25: Last Day to DROP Clases (without a ‘W’ grade) 4/3: Last Day to DROP Classes (with a ‘W’ grade) Make sure to check the Official Semester Dates 11 Vasiliki Kalavri | Boston University 2020 Final Project machines) Google cluster • https://github.com/google/cluster-data/blob/master/ ClusterData2011_2.md Make sure to read and become familiar with the format and schema document: • https://drive.google.com

not shed load • might be producing too slow or become idle • stream processor should be able to make progress • might fail (or seem as if they failed) Streaming sources… 3 Producers and consumers

for (Long s : state) offset = s; } } A stateful source 23 get a lock to make output and state update atomic Vasiliki Kalavri | Boston University 2020 • Working with State: https://ci

∈ rj • as a serialization of r1 followed by a series of delta tuples that indicate updates to make to obtain r2, r3, ..., etc. • as a replacement sequence where some attribute A denotes a key and

Boston University 2020 23 Ensure the combination of A1, A2 is equivalent to A: Given a stream s, make sure A2(A1(s)) = A(s), e.g., • if A is a selection operator and the selection predicate uses logical

fine-grained, impractical for high-rate streams Sampling degrades accuracy Simplified models make strong assumptions Unsuitable for complex operators, e.g. sliding windows, joins ??? Vasiliki