Zero instrumentation monitoring with your first steps in eBPF Beatriz Martínez, Isovalent - @beatrizmrg October 28, 2020 2 What will we be doing? 3 How are we going to do it? 4 Demo time 5 Tracing Tracing Go function with uprobes 6 Demo time 7 Conclusions ● eBPF programs can be attached to different events: ○ Kprobes ○ uprobes ○ Tracepoints ○ network packets… ● Frameworks, Go bindings make it easier: ○ First approach: C, LLVM/CLANG, tc ○ Second approach: gobpf, bcc ● Non-instrumentation monitoring http://34.70.147.68 $curl 34.70.147.68:8083 Let’s represent the attendees in

eBPF Summit October 28, 2020 Debugging Go in production using eBPF ABOUT ME ? i’m Zain @zainasgar Co-Founder/CEO Pixie (@pixie_run) & Adjunct Professor of CS @ Stanford DEVELOPER PROBLEM You’re fmt.Printf("iterations: %d\n”, iterations) YOUR OPTIONS Option 1: Add a log to your program, re-compile and re-deploy. ○ This can be simple log statements, or ○ More comprehensive like Open tracing. ing/trace_example DEMO: Go Argument Tracer ● Utilizing tracepoints for dynamic logging allows for easy instrumentation of production binaries ● The complexities of the Go ABI make it difficult to do

How to ship BPF with your Go project Lorenz Bauer October 28, 2020 Goal ● Count the number of packets on lo ● Print the count to the console ● From a single Go binary Use the source, Luke github github.com/lmb/ship-bpf-with-go github.com/cilium/ebpf

assembly language with a stable instruction set. eBPF programs can be loaded and upgraded in real time without the need to restart the kernel. System calls Bees of various talents took many roles in soon heard of possible improvement options. eBee’s fellows were zealous bees, aiming for speed: “We’d go faster, for sure, with buzz code instructions.” “This would be much simpler for us to parse and read the lead. SLOW 0 0 0 0 1 1 1 1 0 1 1 1 0 0 1 1 0 0 0 0 1 0 0 1 1 0 1 0 0 1 1 0 The Just-in-Time (JIT) compilation step translates the generic bytecode of the program into the machine-specific instruction

Paper Innovation Projects Community-certified openEuler Distributions ( Sorted by certification time ) Partner System xFusion Digital Technologies Co., Ltd. FusionOS 22 UnionTech Software Technology support Scenario collaboration Scenario enablement Linux kernel x86, Arm, RISC-V GPU NPU Real-time kernel Security kernel Database Big data Resource orchestration Desktop system Industrial application configuration file. You only need to compile commands for compilation, environment, running, and batch running in the configuration file to automatically compile and run applications. HPC Deployment

Kubernetes SIG-Instrumentation lead Prometheus team member Richard Hartmann & Frederic Branczyk @TwitchiH & @fredbrancz Intro to Prometheus Introduction Background Operations & observability Outro Time split observability Outro Prometheus 101 Inspired by Google’s Borgmon Time series database unit64 millisecond timestamp, float64 value Instrumentation & exporters Not for event logging Dashboarding via Grafana server answer to HTTP requests?) White-box monitoring: Instrumention code from the inside (How much time does this subroutine take?) Every service should have its own metrics endpoint Hard API commitments

Frederic Branczyk Red Hat (previously CoreOS) All things Prometheus / Kubernetes Kubernetes SIG-Instrumentation lead Prometheus team member Richard Hartmann & Frederic Branczyk @TwitchiH & @fredbrancz Prometheus Beyond Outro Prometheus 101 Inspired by Google’s Borgmon Time series database int64 timestamp, float64 value Ecosystem of instrumentation & exporters Not for events Logging Tracing (more on that later) - 2.6 Beyond Outro Storage Prometheus 1.x We used to have one file per time series ..and one common index for all of time Relatively easy to implement Pretty efficient Why change? Richard Hartmann

Using Istio Docker Cilium with Docker & libnetwork Mesos Cilium with Mesos/Marathon Envoy Envoy Go Extensions Administration System Requirements Summary Linux Distribution Compatibility Matrix Linux demand. This results in a large number of application containers to be started in a short period of time. Typical container firewalls secure workloads by filtering on source IP addresses and destination In order for the entire system to come up, the following components have to be running at the same time: kube-dns or coredns cilium-xxx cilium-etcd-operator etcd-operator etcd-xxx All timeouts are configured

Using Istio Docker Cilium with Docker & libnetwork Mesos Cilium with Mesos/Marathon Envoy Envoy Go Extensions Administration System Requirements Summary Linux Distribution Compatibility Matrix Linux demand. This results in a large number of application containers to be started in a short period of time. Typical container firewalls secure workloads by filtering on source IP addresses and destination Cluster Mesh With Kind we can simulate Cluster Mesh in a sandbox too. Kind Configuration This time we need to create (2) config.yaml, one for each kubernetes cluster. We will explicitly configure their

demand. This results in a large number of application containers to be started in a short period of time. Typical container firewalls secure workloads by filtering on source IP addresses and destination Cluster Mesh With Kind we can simulate Cluster Mesh in a sandbox too. Kind Configuration This time we need to create (2) config.yaml, one for each kubernetes cluster. We will explicitly configure their In order for the entire system to come up, the following components have to be running at the same time: kube-dns or coredns cilium-xxx cilium-operator-xxx cilium-etcd-operator etcd-operator cilium-etcd-xxx