North-South Load Balancing of Kubernetes Services with eBPF/XDP Martynas Pumputis (Isovalent) October 28, 2020 10.0.0.1 10.0.0.2 10.0.0.3 httpd httpd “httpd” service 10.0.0.1:30000 10.0.0.2:30000 KUBE-FORWARD -m comment --comment "kubernetes forwarding rules" -m mark --mark 0x4000/0x4000 -j ACCEPT -A KUBE-FORWARD -s 10.217.0.0/16 -m comment --comment "kubernetes forwarding conntrack pod source rule" --ctstate RELATED,ESTABLISHED -j ACCEPT -A KUBE-FORWARD -d 10.217.0.0/16 -m comment --comment "kubernetes forwarding conntrack pod destination rule" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

网络组研发负责人 演讲人 cilium介绍 https://cilium.io https://github.com/cilium cilium是 kubernetes 的 CNI 网络解决方案，创新采用了 eBPF datapath，为 kubernetes网络和 linux 社区的 eBPF 发展，启动了 最要的推动作用。 截止 2021.10 ，cilium github 项目已有 9.3K star，Contributors star，Contributors 316位 cilium的特色功能： • 网络功能 • 负载均衡 • 网络安全 • 可观察性 • 多集群连通 注：本 PPT 基于 cilium v1.10.4 进行分析 ��������������� ��������������� �������������������� �������������������� �������������������� 、 sendmsg、 recvmsg 、getpeername 、 bind等系统调用，实现 service 的地址解 析，并且伪装通信目的地址，让上层应用 无感知 DNAT 的发生 效果： • 集群内访问nodePort、LoadBalancer 的service时，能够减少数据包转发跳 数，极大提高网络性能 • 相比传统 iptables 等 技术，降低了访 问延时。例如在相同环境下，service

Concepts Component Overview Terminology Networking Network Security eBPF Datapath Observability Kubernetes Integration Multi-Cluster (Cluster Mesh) Getting Help FAQ Slack GitHub Training Enterprise support Layer 4 Examples Layer 7 Examples Deny Policies Host Policies Layer 7 Protocol Visibility Using Kubernetes constructs in policy Endpoint Lifecycle Troubleshooting Monitoring & Metrics Cilium Metrics Hubble Local Development in Vagrant Box Making Changes Add/update a golang dependency Add/update a new Kubernetes version Optional: Docker and IPv6 Debugging Building Container Images Developer images Official

the Cilium architecture and how these components integrate with existing architectures, such as Kubernetes. Installation : Details instructions for installing, configuring, and troubleshooting Cilium in Datapath Scale Kubernetes Integration Getting Help FAQ Slack GitHub Security Bugs Integrations Kubernetes Introduction Concepts Requirements Configuration Network Policy Endpoint CRD Kubernetes Compatibility Policy Policy Enforcement Modes Rule Basics Layer 3 Examples Layer 4 Examples Layer 7 Examples Kubernetes Endpoint Lifecycle Troubleshooting Monitoring & Metrics Installation cilium-agent cilium-operator

Concepts Component Overview Terminology Networking Network Security eBPF Datapath Observability Kubernetes Integration Multi-Cluster (Cluster Mesh) Getting Help FAQ Slack GitHub Training Enterprise support Layer 4 Examples Layer 7 Examples Deny Policies Host Policies Layer 7 Protocol Visibility Using Kubernetes constructs in policy Endpoint Lifecycle Troubleshooting Monitoring & Metrics Cilium Metrics Hubble Local Development in Vagrant Box Making Changes Add/update a golang dependency Add/update a new Kubernetes version Optional: Docker and IPv6 Debugging Building Container Images Developer images Official

Concepts Component Overview Terminology Networking Network Security eBPF Datapath Observability Kubernetes Integration Multi-Cluster (Cluster Mesh) Getting Help FAQ Slack GitHub Training Enterprise support Layer 4 Examples Layer 7 Examples Deny Policies Host Policies Layer 7 Protocol Visibility Using Kubernetes constructs in policy Endpoint Lifecycle Troubleshooting Monitoring & Metrics Cilium Metrics Hubble Local Development in Vagrant Box Making Changes Add/update a golang dependency Add/update a new Kubernetes version Optional: Docker and IPv6 Debugging Building Container Images Developer images Official

the Cilium architecture and how these components integrate with existing architectures, such as Kubernetes. Installation : Details instructions for installing, configuring, and troubleshooting Cilium in Datapath Scale Kubernetes Integration Getting Help FAQ Slack GitHub Security Bugs Integrations Kubernetes Introduction Concepts Requirements Configuration Network Policy Endpoint CRD Kubernetes Compatibility Policy Policy Enforcement Modes Rule Basics Layer 3 Examples Layer 4 Examples Layer 7 Examples Kubernetes Endpoint Lifecycle Troubleshooting L7 Protocol Visibility API Rate Limiting Default Rate Limits

Orchestrators Concepts Component Overview Terminology Networking Network Security eBPF Datapath Kubernetes Integration Multi-Cluster (Cluster Mesh) Getting Help FAQ Slack GitHub Security Bugs Operations Layer 3 Examples Layer 4 Examples Layer 7 Examples Host Policies Layer 7 Protocol Visibility Using Kubernetes constructs in policy Endpoint Lifecycle Troubleshooting Monitoring & Metrics Cilium Metrics Hubble server Hubble Relay Reference Command Cheatsheet Command utilities: Command examples: Kubernetes examples: Command Reference cilium-agent cilium cilium-health cilium-operator cilium-operator-aws

the Cilium architecture and how these components integrate with exis�ng architectures, such as Kubernetes. Installa�on : Details instruc�ons for installing, configuring, and troubleshoo�ng Cilium in different Datapath Scale Kubernetes Integra�on Ge�ng Help FAQ Slack GitHub Security Bugs Integra�ons Kubernetes Introduc�on Concepts Requirements Configura�on Network Policy Endpoint CRD Kubernetes Compa�bility Troubleshoo�ng Layer 3 Examples Layer 4 Examples Layer 7 Examples Kubernetes Endpoint Lifecycle Troubleshoo�ng Monitoring & Metrics Exported Metrics Cilium as a Kubernetes pod Cilium as a host-agent on a node Troubleshoo�ng

挑战1：微服务、多语言、多协议环境下，端到端观测 复杂度上升，埋点成本居高不下 Kubernetes 容器 网络、操作系统、硬件 基础设施层复杂度日益增加 如何关联? 挑战3：数据散落，工具多， 缺少上下文，排查效率低下 业务应用 应用框架 容器虚拟化 系统调用 内核 应用性能监控（APM） Kubernetes监控 Kubernetes组件异常： Scheduler, KCM， etcd，api-server 内核异常：进程调度， 内存管理，文件管理， 夯机宕机，资源异 常… 应用组件异常：线程池满，数据库连接无法获取， OOM，文件读取错误… 无法自顶向下端到端 串联导致棘手问题频 发。 Kubernetes下的可观测 Golang + eBPF实现数据采 集 第四部分 eBPF在可观测领域的优势 无侵入 多语言/多协议/多框架 全栈覆盖 无侵入性 • 无需修改代码 • 无需重启应用