message - many-to-many communication - message content / structure matters for delivery 8 MB architecture advantages • Multiple producers/consumers as concurrent clients • Effective failure handling If consumers are slow, throughput might degrade. • DBs support secondary indexes for efficient search while MBs only offer topic-based subscription. • DB query results depend on a snapshot and clients

about? The design and architecture of modern distributed streaming 4 Fundamental for representing, summarizing, and analyzing data streams Systems Algorithms Architecture and design Scheduling

(live,1) (and,1) (let,1) (live,2) 4 Vasiliki Kalavri | Boston University 2020 Distributed architecture client Flink program JobManager web dashboard TaskManager TaskManager TaskManager 5

can store a fixed proportion of the stream, e.g. 1/10th 7 search engine query stream Example use-case: Web search user behavior study Q: How many queries did users repeat last

way to reach Zurich from London through Berlin? These are the top-10 relevant results for the search term “graph” ??? Vasiliki Kalavri | Boston University 2020 Basics 1 5 4 3 2 “node” or “vertex”

• The number of distinct users who have visited a website? • The top-10 queries inserted in a search engine? • The connected components of accounts in a stream of financial transactions? What synopsis

page in the official Flink documen- tation. For example, you can open the Kafka connector page and search keyword “SQL Client JAR” which is a fat JAR of Kafka connector. • It should be noted that you should

page in the official Flink documen- tation. For example, you can open the Kafka connector page and search keyword “SQL Client JAR” which is a fat JAR of Kafka connector. • It should be noted that you should