GPU Tile 1 Tile 0 Xe LinkProject Goals - Offer high-level, standard C++ distributed data structures - Support distributed algorithms - Achieve high performance for both multi-GPU, NUMA, and multi-node reduce(par_unseq, z, 0, std::plus()); }Outline - Background (Ranges, Parallelism, Distributed Data Structures) - Distributed Ranges (Concepts) - Implementation (Algorithms and views) - Complex sparse matrices) - Lessons learnedOutline - Background (Ranges, Parallelism, Distributed Data Structures) - Distributed Ranges (Concepts) - Implementation (Algorithms and views) - Complex

program for a supercomputer? Introduce PGAS Model, RDMA Building Remote Pointer Types Building Distributed Data Structures Extending to GPUsThis Talk Background: how do we write a program for a supercomputer supercomputer? Introduce PGAS Model, RDMA Building Remote Pointer Types Building Distributed Data Structures Extending to GPUsThis Talk Background: how do we write a program for a supercomputer? Introduce Introduce PGAS Model, RDMA Building Remote Pointer Types Building Distributed Data Structures Extending to GPUsThis Talk Background: how do we write a program for a supercomputer? Introduce PGAS Model

forms of multithreading Hardware vs Software multithreading Data parallelism, Task parallelism, Dataflow parallelism 2 13 Hardware-level multithreading ILP - Instruction-level Parallelism 1 CPU, IO-tasks futures) - Synchronization - Scheduling overhead - Thread-safe memory management 16 Dataflow parallelism - Alternative names - Pipeline parallelism - Graph parallelism - Stream parallelism keywords to expose different requirements Synchronization: - Channels / Shared memory for data - Dataflow parallelism for dependency - Or Barriers with “async/finish” model of Habanero Java - OpenMP barriers

has been (informally) Component Based Software Engineering ● Multi-process and multi-nodes, distributed systems. ● Lot of inter-process communication. ● Publish-Subscribe, Request-Reply. ● Each node Dataflow between Nodes ● We need to share data between Nodes. ● We want to make this explicit and expose

Alvarez-Martinez, D. Jimenez-Gonzalez and Y. Etsion, "Hybrid Dataflow/von-Neumann Architectures," in IEEE Transactions on Parallel and Distributed Systems OOO ExecutionAlex Dathskovsky | alex.dathskovsky@speedata

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327 26 Multi-processing and Distributed Computing 329 26.1 Code Availability and Loading Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1301 68 Delimited Files 1332 69 Distributed Computing 1337 69.1 Cluster Manager Interface . . . . . . . . . . . . . . . . . . . . . . . . need to vectorize code for performance; devectorized code is fast • Designed for parallelism and distributed computation • Lightweight "green" threading (coroutines) • Unobtrusive yet powerful type system

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327 26 Multi-processing and Distributed Computing 329 26.1 Code Availability and Loading Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1301 68 Delimited Files 1332 69 Distributed Computing 1337 69.1 Cluster Manager Interface . . . . . . . . . . . . . . . . . . . . . . . . need to vectorize code for performance; devectorized code is fast • Designed for parallelism and distributed computation • Lightweight "green" threading (coroutines) • Unobtrusive yet powerful type system

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327 26 Multi-processing and Distributed Computing 329 26.1 Code Availability and Loading Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1301 68 Delimited Files 1332 69 Distributed Computing 1337 69.1 Cluster Manager Interface . . . . . . . . . . . . . . . . . . . . . . . . need to vectorize code for performance; devectorized code is fast • Designed for parallelism and distributed computation • Lightweight "green" threading (coroutines) • Unobtrusive yet powerful type system

上游发出来的订单状态机顺序因为⽹络问题会发⽣乱序，影 响我的计算流程 • 上游的业务团队会调整发出来的业务消息，经常导致下游发 ⽣故障 Deprecation 章节 Dependa bot Dataflow Event time Window Watermark Schema Registry Data Valuation for machine learning 还有这篇 paper

SolverPerformance Precision ESP: Path-Sensitive Program Verification in Polynomial Time Path-Sensitive Dataflow Analysis with Iterative RefinementMSVC has both Path-sensitive • Use after move • Concurrency checks