Scaling a Multi-Tenant k8s Cluster in a Telco Pablo Moncada October 28, 2020 About MasMovil group ● 4th telecom company in Spain ● Provides voice and broadband services to +12M customers ● Several

contact k8s api-server In the Cilium agent logs you will see: level=info msg="Establishing connection to apiserver" host="https://10.96.0.1:443" subsys=k8s level=error msg="Unable to contact k8s api-server" ipAddr="https://10.96.0.1:443" subsys=k8s level=fatal msg="Unable to initialize Kubernetes subsystem" error="unable to create k8s client: unable to create k8s client: Get https://10.96.0.1:443/api/ Important Please ensure that you are running version 1.7.9 [https://github.com/aws/amazon-vpc-cni- k8s/releases/tag/v1.7.9] or newer of the AWS VPC CNI plugin to guarantee compatibility with Cilium. The

The DaemonSet will automa�cally install itself as Kubernetes CNI plugin. K8s 1.15 K8s 1.14 K8s 1.13 K8s 1.12 K8s 1.11 K8s 1.10 kubectl create -f https://raw.githubusercontent.com/cilium/cilium/v1 The DaemonSet will automa�cally install itself as Kubernetes CNI plugin. K8s 1.15 K8s 1.14 K8s 1.13 K8s 1.12 K8s 1.11 K8s 1.10 kubectl create -f https://raw.githubusercontent.com/cilium/cilium/v1 during the bootstrapping phase of Cilium. For Docker as container runtime: K8s 1.15 K8s 1.14 K8s 1.13 K8s 1.12 K8s 1.11 K8s 1.10 kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/v1

contact k8s api-server In the Cilum agent logs you will see: level=info msg="Establishing connection to apiserver" host="https://10.96.0.1:443" subsys=k8s level=error msg="Unable to contact k8s api-server" ipAddr="https://10.96.0.1:443" subsys=k8s level=fatal msg="Unable to initialize Kubernetes subsystem" error="unable to create k8s client: unable to create k8s client: Get https://10.96.0.1:443/api/ "ipam": { "type": "calico-ipam" }, "policy": { "type": "k8s" }, "kubernetes": { "kubeconfig": "/etc/cni/net.d/calico-kubeconfig"

contact k8s api-server In the Cilum agent logs you will see: level=info msg="Establishing connection to apiserver" host="https://10.96.0.1:443" subsys=k8s level=error msg="Unable to contact k8s api-server" ipAddr="https://10.96.0.1:443" subsys=k8s level=fatal msg="Unable to initialize Kubernetes subsystem" error="unable to create k8s client: unable to create k8s client: Get https://10.96.0.1:443/api/ "ipam": { "type": "calico-ipam" }, "policy": { "type": "k8s" }, "kubernetes": { "kubeconfig": "/etc/cni/net.d/calico-kubeconfig"

--kubernetes-version 1.10.3 : Kubernetes version that is to be installed. Please note [Kops 1.9 officially supports k8s version 1.9] --cloud-labels "Team=Dev,Owner=Admin" : Labels for your cluster ${NAME} : Name of the "ipam": { "type": "calico-ipam" }, "policy": { "type": "k8s" }, "kubernetes": { "kubeconfig": "/etc/cni/net.d/calico-kubeconfig" Disabled 104 k8s:io.cilium.k8s.policy.cluster=default 10.15.233.139 ready k8s:io.cilium.k8s.policy.serviceaccount=coredns

contact k8s api-server In the Cilum agent logs you will see: level=info msg="Establishing connection to apiserver" host="https://10.96.0.1:443" subsys=k8s level=error msg="Unable to contact k8s api-server" ipAddr="https://10.96.0.1:443" subsys=k8s level=fatal msg="Unable to initialize Kubernetes subsystem" error="unable to create k8s client: unable to create k8s client: Get https://10.96.0.1:443/api/ "ipam": { "type": "calico-ipam" }, "policy": { "type": "k8s" }, "kubernetes": { "kubeconfig": "/etc/cni/net.d/calico-kubeconfig"

contact k8s api-server In the Cilium agent logs you will see: level=info msg="Establishing connection to apiserver" host="https://10.96.0.1:443" subsys=k8s level=error msg="Unable to contact k8s api-server" ipAddr="https://10.96.0.1:443" subsys=k8s level=fatal msg="Unable to initialize Kubernetes subsystem" error="unable to create k8s client: unable to create k8s client: Get https://10.96.0.1:443/api/ Important Please ensure that you are running version 1.7.9 [https://github.com/aws/amazon-vpc-cni- k8s/releases/tag/v1.7.9] or newer of the AWS VPC CNI plugin to guarantee compatibility with Cilium. $

from source to deploy in one click, on any language or stack ● Every developer gets to their own k8s namespace About Okteto A few months ago... GCP was not thrilled Open Source to the rescue! ● Installed automatically reload falco on rule changes ● Rules: monitor well known IPs, binary names, forbidden k8s actions ● Action: Notify to slack for human banhammer ● Use Ubuntu instead of ContainerOS Current

Balancer Service • Part of our private cloud service since 2017 • 5100 private, 760 public VIPs • k8s CCM integration (Type: LoadBalancer) L4LB Node L4LB Architecture XDP DPlane L3DSR with IPIP, Magrev Advertise VIP with eBGP Configure with RPC Health check daemon etc… Service Discovery Per-flow ECMP k8s CCM Frontend (dash board) To Backends User For More Information • Our motivation, detailed architecture