(Vasia) Kalavri
 vkalavri@bu.edu Spring 2020 4/28: Graph Streaming ??? Vasiliki Kalavri | Boston University 2020 Modeling the world as a graph 2 Social networks friend follows The web Actor-movie results for the search term “graph” ??? Vasiliki Kalavri | Boston University 2020 Basics 1 5 4 3 2 “node” or “vertex” “edge” 1 5 4 3 2 undirected graph directed graph 4 ??? Vasiliki Kalavri Kalavri | Boston University 2020 Graph streams Graph streams model interactions as events that update an underlying graph structure 5 Edge events: A purchase, a movie rating, a like on an online post

up an environment to develop Flink programs • Implement streaming data processing pipelines • Flink managed state • Event time Streaming in Apache Flink • Streams are natural • Events of any type

[tile] DB libraries, and large-scale graph analytics. Open-source software projects resulting from his work include the Matrix Template Library, the Boost Graph Library and Open MPI. 。Phil Ratzloff *。Distinguished relationships between elements of a data set *，Without regard to what the data set actually is *。 Graph theoretical (abstract) results can be applied to many different practical (concrete) problems -theory cvMuNKCATIOWS 有二有 灿瑟必帮 和一是/第 手太则光大 A，训四 昱 站 -本| 厅 是，主 半一司、|全之体 ~ 对 /人-看 奋才全 和 全原 和 刁曾 The Future ls Big Graph: on meetalThngs soneaiadman MD] Basic Principles *。The C++ standard library (nee STL) provides

Scalable Stream Processing - Spark Streaming and Flink Amir H. Payberah payberah@kth.se 05/10/2018 The Course Web Page https://id2221kth.github.io 1 / 79 Where Are We? 2 / 79 Stream Processing Systems Spark streaming ▶ Flink 4 / 79 Spark Streaming 5 / 79 Contribution ▶ Design issues • Continuous vs. micro-batch processing • Record-at-a-Time vs. declarative APIs 6 / 79 Spark Streaming ▶ Run Run a streaming computation as a series of very small, deterministic batch jobs. • Chops up the live stream into batches of X seconds. • Treats each batch as RDDs and processes them using RDD operations

Selection ✓ Order by ❏ Group by ✓ Query ✓ Limit ✓ Expression Compatibility ✓ Join ✓ Subquery ✓ Graph traverse ✓ Out direction ✓ In direction ✓ Bidirection ✓ Edge traverse ✓ Distinct Index 01.

Scott, Principal Engineer at CMU SEI Graph/ML/AI algorithms for large- and small- scale parallel systems. Working on GBTL, a linear algebra-based C++ library for graph analytics.[DISTRIBUTION STATEMENT GraphBLAS community, C API Overview of our draft C++ API How might this interoperate with standard C++, graph library proposal? 4[DISTRIBUTION STATEMENT A] This material has been approved for public release GraphBLAS community, C API Overview of our draft C++ API How might this interoperate with standard C++, graph library proposal? 5[DISTRIBUTION STATEMENT A] This material has been approved for public release

Using the Microsoft Graph API to get Office 365 data in your mobile apps Alex Ziskind Technical Director @digitalix www.nuvious.com Hi, I’m Alex From + 750 Million 50 Million per month Problems

top-down task graph What is Task Graph-based Programming System (TGPS) Code 4• TGPS encapsulates function calls and their dependencies in a top-down task graph What is Task Graph-based Programming task graph What is Task Graph-based Programming System (TGPS) Code A B C D B A C D 6• TGPS encapsulates function calls and their dependencies in a top-down task graph What is Task Graph-based precede(task_d); 19 20 sched.schedule(); 21 sched.wait(); B A C D 1. Easy to write and express a task graph 2. Allow to implement irregular parallel decomposition strategies 8Existing TGPSs on Heterogenous

4/14: Stream processing optimizations ??? Vasiliki Kalavri | Boston University 2020 2 • Costs of streaming operator execution • state, parallelism, selectivity • Dataflow optimizations • plan translation basics 3 source sink input port output port dataflow graph ??? Vasiliki Kalavri | Boston University 2020 Revisiting the basics 4 Dataflow graph • operators are nodes, data channels are edges • ??? Vasiliki Kalavri | Boston University 2020 12 • What does efficient mean in the context of streaming? • queries run continuously • streams are unbounded • In traditional ad-hoc database queries

Kalavri
 vkalavri@bu.edu CS 591 K1: Data Stream Processing and Analytics Spring 2020 2/04: Streaming languages and operator semantics Vasiliki Kalavri | Boston University 2020 Vasiliki Kalavri | Boston interval of 5–15 s) by an item of type C with Z < 5. 8 Vasiliki Kalavri | Boston University 2020 Streaming Operators 9 Vasiliki Kalavri | Boston University 2020 Operator types (I) • Single-Item Operators println!("seen: {:?}", x))
 .connect_loop(handle);
 }); t (t, l1) (t, (l1, l2)) Streaming Iteration Example Terminate after 100 iterations Create the feedback loop 13 Vasiliki Kalavri