Go compile time instrumentation Przemysław Delewski February 18, 2025 Warsaw, PolandAgenda 1. Short bio 2. Observability domain 3. OpenTelemetry 4. History of OpenTelemetry go compile time instrumentation member of the OpenTelemetry Go compile-time instrumentation project https://github.com/open-telemetry/community/blo b/main/projects/go-compile-instrumentation.mdSoftware complexity Today's software, especially information carried together with spanHistory of OpenTelemetry go instrumentationBeginnings of go instrumentation Everything started with two proposals: - https://github.com/open-telemetry/opentelemetry-go-instrumen

macOS, iOS, Android, WASM*) Hybrid profiling capabilities (sampling and/or instrumentation) (CPU and GPU instrumentation) Tracing capabilities (values, messages, plots, allocations, …) Hassle-free (Windows, Linux, macOS, iOS, Android, WASM*) Hybrid profiling capabilities (sampling and/or instrumentation) Tracing capabilities (values, messages, plots, allocations, …) Free and Open source Kudos (Windows, Linux, macOS, iOS, Android, WASM*) Hybrid profiling capabilities (sampling and/or instrumentation) Tracing capabilities (values, messages, plots, allocations, …) Free and Open source Kudos

macOS, iOS, Android, WASM*) Hybrid profiling capabilities (sampling and/or instrumentation) (CPU and GPU instrumentation) Tracing capabilities (values, messages, plots, allocations, …) Hassle-free (Windows, Linux, macOS, iOS, Android, WASM*) Hybrid profiling capabilities (sampling and/or instrumentation) Tracing capabilities (values, messages, plots, allocations, …) Free and Open source Kudos (Windows, Linux, macOS, iOS, Android, WASM*) Hybrid profiling capabilities (sampling and/or instrumentation) Tracing capabilities (values, messages, plots, allocations, …) Free and Open source Kudos

see how AddressSanitizer works behind the scenes (compiler and ASan runtime) and analyze the instrumentation impact, both in perf and memory footprint. We’ll examine a handful of examples diagnosed by impact: depending on tool, from 2x up to 10x) extra-memory usage (for memory related tools/instrumentation), 2x or more sometimes difficult to map error reports into source code for Release/optimized impact: depending on tool, from 2x up to 10x) extra-memory usage (for memory related tools/instrumentation), 2x or more sometimes difficult to map error reports into source code for Release/optimized

} – 1994 A system for building customized program analysis tools, called “dynamic binary instrumentation” elsewhere. This simple ATOM program instruments all of an (already compiled) program’s load and so are paravirtualization techniques.Valgrind A Framework for Heavyweight Dynamic Binary Instrumentation We focus on Valgrind’s unique support for shadow values—a powerful but previously little-studied register and memory value with another value that describes it. — 2007 Let’s build on the instrumentation capabilities of tools such as ATOM, and go extremely far in the analysis capabilities. In particular

AddressSanitizer Memory error detector compile & link -fsanitize=address google/sanitizers a compiler instrumentation module a runtime library which replaces the malloc functionMemory Leaks: Tools Tool What How AddressSanitizer Memory error detector compile & link -fsanitize=address google/sanitizers a compiler instrumentation module a runtime library which replaces the malloc function 😃 ● no code change required ●

flow, as well as the call path which led to these allocations © 2020 Apex.AI, Inc.Injecting Instrumentation Code pointer allocate(size_type n, const void * = 0) { instrument::type_reg(); ○ -g to have the file:line information for call graphs ○ -O0 to prevent optimizing out our instrumentation calls ○ -emit-llvm to compile into the bitcode ○ Probably -DNDEBUG to get the Release versions

in its grasp” 119 Source: ChatGPTClang Xray Instrumentation 120 __xray_patch_function(id); __xray_set_handler(XRayRCon::Profile);Clang Xray Instrumentation 121 __xray_patch_function(id); __xray_s

Profilers used in Game Development Tracy, Intel VTune, Microso� PIX, Custom • Need profiler with instrumentation • Need to see whole process and all threads Using Intel’s VTune in this presentation 13Key or 3rd party - many integrated into engine Used to measure CPU, GPU, etc Need profiler with instrumentation Cannot rely on sampling - everything is in jobs Need to see all threads Looking for gaps, long

@msarahan • @rrigdon 4.4.57 4.6.4 (2019-02-13) Improvements • allow configuring location of instrumentation records (#7849) • prepend conda-env pip commands with env activation to fix library loading #6243 mark root as not writable when sys.prefix is not a conda environment (#6274) • add timing instrumentation (#6458) 4.4.92 4.3.30 (2017-10-17) Improvements • address #6056 add additional proxy variables default_return]) Enforce a maximum time for a callable to complete. get_instrumentation_record_file() print_instrumentation_data() 4.6. Developer guide 503 conda, Release 23.10.1.dev3 Attributes