LINE’s Private Cloud Yutaro Hayakawa October 28, 2020 • Messaging & many family services • 185 million global MAU • 3Tbps+ network traffic in total LINE Verda: LINE’s Private Cloud Service IaaS LB LB NAT … PaaS FaaS … Verda and XDP Based L4 Load Balancer Service • Part of our private cloud service since 2017 • 5100 private, 760 public VIPs • k8s CCM integration (Type: LoadBalancer) L4LB

infrastructure as the only requirement is IP connectivity between hosts which is typically already given. Native Routing: Use of the regular routing table of the Linux host. The network is required to be capable awareness of the underlying networking infrastructure. This mode works well with: Native IPv6 networks In conjunction with cloud network routers If you are already running routing daemons Load Balancing Cilium Kubernetes cluster using Google Kubernetes Engine [https://cloud.google.com/kubernetes-engine]. See Installing Google Cloud SDK [https://cloud.google.com/sdk/install] for instructions on how to install

infrastructure as the only requirement is IP connectivity between hosts which is typically already given. Native Routing: Use of the regular routing table of the Linux host. The network is required to be capable awareness of the underlying networking infrastructure. This mode works well with: Native IPv6 networks In conjunction with cloud network routers If you are already running routing daemons Load balancing Distributed Install Hubble Hubble is a fully distributed networking and security observability platform for cloud native workloads. It is built on top of Cilium and eBPF to enable deep visibility into the communication

infrastructure as the only requirement is IP connectivity between hosts which is typically already given. Native Routing: Use of the regular routing table of the Linux host. The network is required to be capable awareness of the underlying networking infrastructure. This mode works well with: Native IPv6 networks In conjunction with cloud network routers If you are already running routing daemons Load balancing Distributed Enable Hubble Hubble is a fully distributed networking and security observability platform for cloud native workloads. It is built on top of Cilium and eBPF to enable deep visibility into the communication

infrastructure as the only requirement is IP connectivity between hosts which is typically already given. Native Routing: Use of the regular routing table of the Linux host. The network is required to be capable awareness of the underlying networking infrastructure. This mode works well with: Native IPv6 networks In conjunction with cloud network routers If you are already running routing daemons Load Balancing Cilium Kubernetes cluster using Google Kubernetes Engine [https://cloud.google.com/kubernetes-engine]. See Installing Google Cloud SDK [https://cloud.google.com/sdk/install] for instructions on how to install

infrastructure as the only requirement is IP connectivity between hosts which is typically already given. Native Routing: Use of the regular routing table of the Linux host. The network is required to be capable awareness of the underlying networking infrastructure. This mode works well with: Native IPv6 networks In conjunction with cloud network routers If you are already running routing daemons Load Balancing Cilium Observability section. Installation on OpenShift OKD OpenShift Requirements 1. Choose preferred cloud provider. This guide was tested in AWS, Azure & GCP. 2. Read OpenShift documentation [https://docs

infrastructure as the only requirement is IP connectivity between hosts which is typically already given. Native Routing: Use of the regular routing table of the Linux host. The network is required to be capable awareness of the underlying networking infrastructure. This mode works well with: Native IPv6 networks In conjunction with cloud network routers If you are already running routing daemons Load balancing Distributed The following guides cover the installation steps for managed Kubernetes environments as offered by cloud providers. If a particular offering is not covered, the guide Installation with managed etcd has a

the networking stack. This enhances network performance and flexibility making it ready for the cloud-native world. Projects using eBPF for networking include Cilium and Katran, for example. System calls Phippy (the giraffe), and Tai (the elephant) are copy- right The Linux Foundation, on behalf of the Cloud Native Computing Foundation. They are licensed under Creative Commons Attribution 4.0 International

underlying networking infrastructure. This mode works well with: Na�ve IPv6 networks In conjunc�on with cloud network routers If you are already running rou�ng daemons Load balancing Distributed load balancing The following guides cover the installa�on steps for managed Kubernetes environments as offered by cloud providers. If a par�cular offering is not covered, the guide Standard Installa�on has a good chance Installation on Google GKE GKE Requirements Install the Google Cloud SDK ( gcloud ) see [Installing Google Cloud SDK] (h�ps://cloud.google.com/sdk/install) Create a project or use an exis�ng one export

NAT XDP eBPF NAT DSR 加速南北向 nodePort 访问 传统的 nodePort 转发，伴随着 SNAT的发生。而 Cilium 为 nodePort 提供了 native 和 IPIP 等方式的 DSR （direct server return）实现，有效减 少了网络转发的跳数，极大提升了 nodePort的转发性能，降低访问延时。 相关测试表明： • kube redirect_peer redirect_neigh step1 client -> node1 : nodePort step3 client -> pod2 : targetPort native DSR DNAT and No SNAT step4 pod2:targetPort -> client step6 node2 : nodePort -> client client ipv4 tunnel case: soure identity->vxlan VNI field tc eBPF look up identiy by source ip for ipv4 native-routing case tc eBPF insert source identity to skb->mark worker node2 implement policy selecting