FLINK_HOME instead of the JAR packages under PyFlink installation package. You can refer to PyFlink documentation for more details. 1.3.1.2 O2: Java gateway process exited before sending its port number The connector JAR in PyFlink jobs, you can refer to the dependency management page of official PyFlink documentation. 1.3.4.2 O2: ClassNotFoundException: com.mysql.cj.jdbc.Driver py4j.protocol.Py4JJavaError: An

FLINK_HOME instead of the JAR packages under PyFlink installation package. You can refer to PyFlink documentation for more details. 1.3.1.2 O2: Java gateway process exited before sending its port number The connector JAR in PyFlink jobs, you can refer to the dependency management page of official PyFlink documentation. 1.3.4.2 O2: ClassNotFoundException: com.mysql.cj.jdbc.Driver py4j.protocol.Py4JJavaError: An

the progress of upstream stages • minimum of output watermarks of all upstream tasks • The output watermark captures the progress of the stage itself • minimum of input watermarks and event-times

problem 5 Given a logical dataflow with sources S1, S2, … Sn and rates λ1, λ2, … λn identify the minimum parallelism πi per operator i, such that the physical dataflow can sustain all source rates. S1

overestimate real frequencies. The best approximation is not the average of all counters, but the minimum. let f: array of length p for j=1 to p do i = hj(x) f[j] = ci,j return min(f[1], f[2], …, f[p])

file:///home/user/wordcount_out 19 Flink commands Vasiliki Kalavri | Boston University 2020 Resources • Documentation • https://flink.apache.org/ • Community • https://flink.apache.org/community.html#mailing-lists losing any records committed to the log. Vasiliki Kalavri | Boston University 2020 Resources • Documentation • https://kafka.apache.org/ • Community • https://kafka.apache.org/contact • Conference

additional dependencies on the classpath. Please check out the Flink system resource metrics documentation9 for additional guidance and details. System resource monitoring in Flink can be very helpful metrics and monitoring capabilities of Apache Flink. I highly recommend going over Flink’s metrics documentation10 for a full reference of Flink’s metrics system.

pre-processing, deployment, and testing): (40%) • code deliverables must be accompanied by documentation 8 Vasiliki Kalavri | Boston University 2020 Schedule 9 vasia.github.io/dspa20/ lectures