eBPF Summit October 28, 2020 Debugging Go in production using eBPF ABOUT ME ? i’m Zain @zainasgar Co-Founder/CEO Pixie (@pixie_run) & Adjunct Professor of CS @ Stanford DEVELOPER PROBLEM You’re Argument Tracer ● Utilizing tracepoints for dynamic logging allows for easy instrumentation of production binaries ● The complexities of the Go ABI make it difficult to do. Especially when you consider:

features enabled. These features may include unreleased or beta features that are not considered production-ready. While it provides a convenient way to try out experimental features, It should only be used Cilium on K3s [https://k3s.io/], a highly available, certified Kubernetes distribution designed for production workloads in unattended, resource-constrained, remote locations or inside IoT appliances. This with ARM support planned [https://github.com/cilium/cilium/issues/9898] for a future release. Install a Master Node The first step is to install a K3s master node making sure to disable support for the

Kubernetes Integration Multi-Cluster (Cluster Mesh) Getting Help FAQ Slack GitHub Training Enterprise support Security Bugs Operations System Requirements Summary Linux Distribution Compatibility Matrix Linux environment Self-Managed Kubernetes Managed Kubernetes Installer Integrations CNI Chaining Setting up Support for External Workloads (beta) Network Policy Security Tutorials Identity-Aware and HTTP-Aware Policy features enabled. These features may include unreleased or beta features that are not considered production-ready. While it provides a convenient way to try out experimental features, It should only be used

Mesh) Getting Help FAQ Slack GitHub Training Enterprise support Security Bugs Operations System Requirements Summary Architecture Support Linux Distribution Compatibility & Considerations Linux Kernel Container Images Developer images Official release images Experimental Docker BuildKit and Buildx support Official Cilium repositories Update cilium-builder and cilium-runtime images Nightly Docker image Cluster Mesh Setting up Cluster Mesh Load-balancing & Service Discovery Network Policy Setting up Support for External Workloads (beta) Operations Running Prometheus & Grafana Istio Getting Started Using

Kubernetes Integration Multi-Cluster (Cluster Mesh) Getting Help FAQ Slack GitHub Training Enterprise support Security Bugs Operations System Requirements Summary Linux Distribution Compatibility & Considerations Container Images Developer images Official release images Experimental Docker BuildKit and Buildx support Official Cilium repositories Update cilium-builder and cilium-runtime images Nightly Docker image Cluster Mesh Setting up Cluster Mesh Load-balancing & Service Discovery Network Policy Setting up Support for External Workloads (beta) Operations Running Prometheus & Grafana Istio Getting Started Using

Cilium on K3s [http://k3s.io], a highly available, certified Kubernetes distribution designed for production workloads in unattended, resource-constrained, remote locations or inside IoT appliances. This with ARM support planned [https://github.com/cilium/cilium/issues/9898] for a future release. Install a Master Node The first step is to install a K3s master node making sure to disable support for the in the same cluster, there should be no issue as both version are mutually compatible in order to support gradual migration [https://helm.sh/docs/topics/v2_v3_migration/]. Cilium chart is targeting Helm

--override "cluster.spec.etcdClusters[*].version=3.1.11" --kubernetes-version 1.10.3 --cloud-labels "Team=Dev,Owner=Admin" ${NAME} You may be prompted to create a ssh public-private key pair. ssh-keygen that is to be installed. Please note [Kops 1.9 officially supports k8s version 1.9] --cloud-labels "Team=Dev,Owner=Admin" : Labels for your cluster ${NAME} : Name of the cluster. Make sure the name ends documentation [https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/#support-hostport]. Note Before using HostPort, read the Kubernetes Configuration Best Practices [https://kubernetes

updates to the kernel can take years to reach end users running stable long-term releases (LTS) in production. It is possible to extend the kernel’s functionalities by writing and loading kernel modules, as BCC, bpftrace, Hubble, Parca, or Pyroscope use eBPF for tracing and monitoring purposes. The team flew and traveled, becoming so prosperous That the news spread around, even faster than light, Making their skills were for hire. And the united bees formed the Hive Alliance. After years of proven production experience, eBPF has been adopted for Windows and other privileged execution contexts. The eBPF

that is to be installed. Please note [Kops 1.9 officially supports k8s version 1.9] --cloud-labels "Team=Dev,Owner=Admin" : Labels for your cluster ${NAME} : Name of the cluster. Make sure the name ends s empire-hq service only has search access for both the indices Fortunately, the Empire DevOps team is using Cilium for their Kubernetes cluster. Cilium provides L7 visibility and security policies environment running on your machine. It is designed to take 15-30 minutes. NOTE: Cassandra-aware policy support is s�ll in beta phase. It is not yet ready for produc�on use. Addi�onally, the Cassandra-specific

exposes total run_time_ns and run_cnt – Use cases: – Benchmarking + CI/CD – Sampling profiler in production How does it work? – Adds ~20ns of overhead per run Two ways to enable kernel eBPF stats sysctl G6LOCAL ✔ ✔ ✔ ✔ ✔ BPF_PROG_TYPE_XDP ✔ ? ✔ ? ✔ BPF_PROG_TYPE_FLOW_DISSECTOR ✔ ✔ ✔ ✔ ✔ Feature Support ebpfbench - Go library for eBPF benchmarking https://github.com/DataDog/ebpfbench ebpfbench