producer: one producer (aka writer) thread ● Single consumer: one consumer (aka reader) thread ● Lock-free: it doesn’t use mutex locks. At any point of time, some thread will make progress. ● Wait-free: connected end-to-end. The oldest entry is processed first. 4 Single Producer Single Consumer Lock-Free Wait-Free Fifo5 Read message from network Handle message Write response to network SPSC Pikus Taking a Byte Out of C++ - Avoiding Punning by Starting Lifetimes - Robert Leahy Awesome Lock-Free - Erik Rigtorp What is Low Latency C++? (Part 1) - Timur Doumler - CppNow 2023 What is Low Latency

Deferred reclamation, i.e., garbage collection in C++ Some assumed knowledge • You’ve heard of lock-free programming and know what a compare_exchange is • You know roughly what atomic does and what it Deferred reclamation, i.e., garbage collection in C++ Some assumed knowledge • You’ve heard of lock-free programming and know what a compare_exchange is • You know roughly what atomic does and what it -- danielanderson.net std::atomic How many lines does it take to implement a lock-free & memory-safe stack? template struct Stack { struct Node { T t; shared_ptr

net3 What we’ll learn today • Very quick review of concurrency and lock-free programming • Review the “bread and butter” of lock-free design patterns • Define wait-free algorithms, understand the definition assumed knowledge • You know what std::atomic does and what it is used for • You’ve heard of lock-free programming and know what a compare_exchange is Daniel Anderson -- danielanderson.net4 Our motivating concurrent operations • Guaranteed bounded completion time for every individual operation.11 A lock-free solution Daniel Anderson -- danielanderson.net struct Counter { bool increment_if_not_zero()

event_handler(){ auto handle=spawn_background_task(); handle.wait(); // no benefit }Aside: Non-Blocking vs Lock-free In lots of cases, short-term blocking (e.g. with short-lived std::mutex locks) is OK. ⇒ for those those cases, Non-Blocking means Not waiting for a lengthy task to run.Aside: Non-Blocking vs Lock-free In lots of cases, short-term blocking (e.g. with short-lived std::mutex locks) is OK. ⇒ for those but one thread at any point, that one thread will complete its task. ⇒ for those cases, you need Lock-free allocators, message queues, etc.How do we move work off the current thread?How do we move work off

Structures Live Demonstration References Persistent concurrent hash Map (PMap) ▶ Non-volatile ▶ Lock-free ▶ Guaranteed system-wide progress ▶ Scales up with multiple threads ▶ Open addressing ▶ In-place Structures Live Demonstration References Resizing ▶ Adapted from Cliff Click’s hash map [14] ▶ Lock-free resizing is challenging ▶ Keys and values are separate atomics ▶ Partial operations are possible Structures Live Demonstration References Related Works Compared ▶ Concurrent level hashing (clevel) ▶ Lock-free ▶ Open addressing (of pointers) ▶ Resize (but only expansion) ▶ OneFile hash map (OneFile) ▶ Wait-free

System/Subsystem Design (Defines API boundaries and behavior) • Implementation details (e.g., “lock-free” and “wait-free” algorithms provide guarantees for system-wide or per-thread progress) highest System/Subsystem Design (Defines API boundaries and behavior) • Implementation details (e.g., “lock-free” and “wait-free” algorithms provide guarantees for system-wide or per-thread progress) highest System/Subsystem Design (Defines API boundaries and behavior) • Implementation details (e.g., “lock-free” and “wait-free” algorithms provide guarantees for system-wide or per-thread progress) highest

integrity. Efficient and safe concurrent updates to shared variables. Enable the implementation of lock-free data structures, leading to improved performance and scalability. Useful in various applications in data-parallel algorithms, statistics collection, and optimization processes. Useful for: ● Lock-free data structures ● Parallel reductions (OpenMP) ● Optimization algorithms ● Statistics collectionProposed

other processes made it safely through Local work that might throwOut-of-band in C++: Atomics • (Lock-free) atomic updates don’t block – ➔ can use them anywhere in parallel algorithms or tasks – e.g., write

C++ for Concurrency Version 2 (wg21.link/n4895).  Hazard Pointers: Safe Memory Reclamation for Lock-Free Objects, IEEE Transactions on Parallel and Distributed Systems. 15 (8): 491–504, June 2004. Hazard

Problem #1 - Task Queues Do Not Scale Well: ● Contention: ○ Even the most meticulously designed lock-free queues experience a significant drop in performance as the number of threads increases. ● Multiple