for use: curl -L https://raw.githubusercontent.com/apache/flink/master/flink-python/pyflink/ ˓→examples/table/word_count.py -o word_count.py python3 word_count.py # You will see outputs as following: following: curl -L https://raw.githubusercontent.com/apache/flink/master/flink-python/pyflink/ ˓→examples/table/word_count.py -o word_count.py python3 word_count.py If there any any problems, you could -Dkubernetes.container.image= \ --pyModule word_count \ --pyFiles /opt/flink/examples/python/table/word_count.py Execute PyFlink jobs in session mode with Native Kubernetes You could

for use: curl -L https://raw.githubusercontent.com/apache/flink/master/flink-python/pyflink/ ˓→examples/table/word_count.py -o word_count.py python3 word_count.py # You will see outputs as following: following: curl -L https://raw.githubusercontent.com/apache/flink/master/flink-python/pyflink/ ˓→examples/table/word_count.py -o word_count.py python3 word_count.py If there any any problems, you could -Dkubernetes.container.image= \ --pyModule word_count \ --pyFiles /opt/flink/examples/python/table/word_count.py Execute PyFlink jobs in session mode with Native Kubernetes You could

Deliverables • One (1) written report of maximum 5 pages (10%). • Code (including pre-processing, deployment, and testing): (40%) • code deliverables must be accompanied by documentation 8 Vasiliki Kalavri less than 15% of all data. 18 Vasiliki Kalavri | Boston University 2020 Can you give me some examples of streaming data sources? 19 Vasiliki Kalavri | Boston University 2020 20 Location-based services applications • Complex filtering and alarm activation • Aggregation of multiple sensors and joins • Examples • Real-time statistics, e.g. weather maps • Monitor conditions to adjust resources, e.g. power

application with no arguments:
 ./bin/flink run ./examples/batch/WordCount.jar Run an application with input and output arguments:
 ./bin/flink run ./examples/batch/WordCount.jar \
 --input class entry point and arguments:
 ./bin/flink run -c org.apache.flink.examples.java.wordcount.WordCount \
 ./examples/batch/WordCount.jar \
 --input file:///home/user/hamlet

Array • composite types: Tuples, POJOs, and Scala case classes • Kryo for unknown types Type Examples Tuples Tuple1 through Tuple25 types. POJOs A POJO (plain old Java object) is any Java class

comparing them with yesterday’s frequencies provides an indication of what is “trending” Motivating examples ??? Vasiliki Kalavri | Boston University 2020 19 • Expand the classical BF with an array of m

function is executed in the driver process. 31 / 79 Output Operations (2/4) ▶ What’s wrong with this code? ▶ This requires the connection object to be serialized and sent from the driver to the worker. send(record) // executed at the worker } } 32 / 79 Output Operations (2/4) ▶ What’s wrong with this code? ▶ This requires the connection object to be serialized and sent from the driver to the worker. send(record) // executed at the worker } } 32 / 79 Output Operations (3/4) ▶ What’s wrong with this code? ▶ Creating a connection object has time and resource overheads. ▶ Creating and destroying a connection

Continuous Processing & Streaming Analytics Event-driven Applications ✔ ✔ ✔ 扫码加入社群 与志同道合的码友一起 Code Up 阿里云开发者社区 Apache Flink China 2群 粘贴二维码 谢谢！

• Retries simply replay the output that has been checkpointed, i.e. the user’s non- deterministic code is not re-executed Bloom filters for performance • Maintaining a catalog of all IDs ever seen

time window) for functional reasons. 4. Each computation in your Flink topology (framework or user code), as well as each network shuffle, takes time and adds to latency. 5. If the application emits