Pepabo, Inc. 2019.07.25 GopherCon 2019 Optimization for number of goroutines using feedback control Principal engineer Yusuke MIYAKE @monochromegane Pepabo R&D Institute, GMO Pepabo, Inc. https://blog Issues to solve for the realization �27 1. Selection of performance metrics 2. Finding how to control rapidly and continuously Performance metrics Performance metrics �29 • Independent resource type number of goroutines based on the metrics • Continuously • Rapidly • Accurately • Feedback control meets these conditions. • Tracking a given set-point using errors from the set-point. • Applying

Persistent Hash Map Persistent Transactional Data Structures Live Demonstration ReferencesIntroduction Persistent Hash Map Persistent Transactional Data Structures Live Demonstration References Overview Introduction Data Structures Design Goals Methodology Performance Results Live Demonstration A Persistent Hash Map for Graph Processing Workloads and a Methodology for Persistent Transactional Data Structures 2In 2IntroductionIntroduction Persistent Hash Map Persistent Transactional Data Structures Live Demonstration References Introduction Persistent Memory ▶ Persistent Memory is positioned as a new tier in the

search is now O (log N)! Awesome!!!Problems (Not really, more like caveats …) Like all tree data structures, not really cache friendly. ● Can always represent the tree as a heap but you still jump around

List-like data structures std::vector std::list std::dequestd::vector C++ version of the array-list data structure Backed by a C-style array Automatically allocates a new backing array when inserting

supercomputer? Introduce PGAS Model, RDMA Building Remote Pointer Types Building Distributed Data Structures Extending to GPUsThis Talk Background: how do we write a program for a supercomputer? Introduce Introduce PGAS Model, RDMA Building Remote Pointer Types Building Distributed Data Structures Extending to GPUsThis Talk Background: how do we write a program for a supercomputer? Introduce PGAS Model Model, RDMA Building Remote Pointer Types Building Distributed Data Structures Extending to GPUsThis Talk Background: how do we write a program for a supercomputer? Introduce PGAS Model, RDMA Building

0 GPU Tile 1 Tile 0 Xe LinkProject Goals - Offer high-level, standard C++ distributed data structures - Support distributed algorithms - Achieve high performance for both multi-GPU, NUMA, and (Ranges, Parallelism, Distributed Data Structures) - Distributed Ranges (Concepts) - Implementation (Algorithms and views) - Complex Data Structures (Dense and sparse matrices) - Lessons (Ranges, Parallelism, Distributed Data Structures) - Distributed Ranges (Concepts) - Implementation (Algorithms and views) - Complex Data Structures (Dense and sparse matrices) - Lessons

Stack/Queue Live Demonstration Design and Implementation of Highly Scalable Quantifiable Data Structures in C++ 2MotivationMotivation Quantifiability Vector Space Entropy Measurement Design and Implementation of atoms in the universe is 1082 Design and Implementation of Highly Scalable Quantifiable Data Structures in C++ 4Motivation Quantifiability Vector Space Entropy Measurement Design and Implementation of takes effect in real-time order. Design and Implementation of Highly Scalable Quantifiable Data Structures in C++ 5Motivation Quantifiability Vector Space Entropy Measurement Design and Implementation of

Stream Processing and Analytics Vasiliki (Vasia) Kalavri
 vkalavri@bu.edu Spring 2020 4/09: Flow control and load shedding ??? Vasiliki Kalavri | Boston University 2020 Keeping up with the producers what if the queue grows larger than available memory? • block the producer (back-pressure, flow control) 2 ??? Vasiliki Kalavri | Boston University 2020 Load management approaches 3 ! Load shedder runtime and selectively drops tuples according to a QoS specification. • Similar to congestion control or video streaming in a lower quality. 4 ??? Vasiliki Kalavri | Boston University 2020 https://commons