GPU Tile 1 Tile 0 Xe LinkProject Goals - Offer high-level, standard C++ distributed data structures - Support distributed algorithms - Achieve high performance for both multi-GPU, NUMA, and multi-node reduce(par_unseq, z, 0, std::plus()); }Outline - Background (Ranges, Parallelism, Distributed Data Structures) - Distributed Ranges (Concepts) - Implementation (Algorithms and views) - Complex sparse matrices) - Lessons learnedOutline - Background (Ranges, Parallelism, Distributed Data Structures) - Distributed Ranges (Concepts) - Implementation (Algorithms and views) - Complex

program for a supercomputer? Introduce PGAS Model, RDMA Building Remote Pointer Types Building Distributed Data Structures Extending to GPUsThis Talk Background: how do we write a program for a supercomputer supercomputer? Introduce PGAS Model, RDMA Building Remote Pointer Types Building Distributed Data Structures Extending to GPUsThis Talk Background: how do we write a program for a supercomputer? Introduce Introduce PGAS Model, RDMA Building Remote Pointer Types Building Distributed Data Structures Extending to GPUsThis Talk Background: how do we write a program for a supercomputer? Introduce PGAS Model

has been (informally) Component Based Software Engineering ● Multi-process and multi-nodes, distributed systems. ● Lot of inter-process communication. ● Publish-Subscribe, Request-Reply. ● Each node Dataflow between Nodes ● We need to share data between Nodes. ● We want to make this explicit and expose

Alvarez-Martinez, D. Jimenez-Gonzalez and Y. Etsion, "Hybrid Dataflow/von-Neumann Architectures," in IEEE Transactions on Parallel and Distributed Systems OOO ExecutionAlex Dathskovsky | alex.dathskovsky@speedata

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327 26 Multi-processing and Distributed Computing 329 26.1 Code Availability and Loading Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1301 68 Delimited Files 1332 69 Distributed Computing 1337 69.1 Cluster Manager Interface . . . . . . . . . . . . . . . . . . . . . . . . need to vectorize code for performance; devectorized code is fast • Designed for parallelism and distributed computation • Lightweight "green" threading (coroutines) • Unobtrusive yet powerful type system

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327 26 Multi-processing and Distributed Computing 329 26.1 Code Availability and Loading Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1301 68 Delimited Files 1332 69 Distributed Computing 1337 69.1 Cluster Manager Interface . . . . . . . . . . . . . . . . . . . . . . . . need to vectorize code for performance; devectorized code is fast • Designed for parallelism and distributed computation • Lightweight "green" threading (coroutines) • Unobtrusive yet powerful type system

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327 26 Multi-processing and Distributed Computing 329 26.1 Code Availability and Loading Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1301 68 Delimited Files 1332 69 Distributed Computing 1337 69.1 Cluster Manager Interface . . . . . . . . . . . . . . . . . . . . . . . . need to vectorize code for performance; devectorized code is fast • Designed for parallelism and distributed computation • Lightweight "green" threading (coroutines) • Unobtrusive yet powerful type system

SolverPerformance Precision ESP: Path-Sensitive Program Verification in Polynomial Time Path-Sensitive Dataflow Analysis with Iterative RefinementMSVC has both Path-sensitive • Use after move • Concurrency checks

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331 26 Multi-processing and Distributed Computing 332 26.1 Code Availability and Loading Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1354 69 Delimited Files 1385 70 Distributed Computing 1390 70.1 Cluster Manager Interface . . . . . . . . . . . . . . . . . . . . . . . . need to vectorize code for performance; devectorized code is fast • Designed for parallelism and distributed computation • Lightweight "green" threading (coroutines) • Unobtrusive yet powerful type system

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331 26 Multi-processing and Distributed Computing 333 26.1 Code Availability and Loading Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1352 69 Delimited Files 1383 70 Distributed Computing 1388 70.1 Cluster Manager Interface . . . . . . . . . . . . . . . . . . . . . . . . need to vectorize code for performance; devectorized code is fast • Designed for parallelism and distributed computation • Lightweight "green" threading (coroutines) • Unobtrusive yet powerful type system