Scaling a Multi-Tenant k8s Cluster in a Telco Pablo Moncada October 28, 2020 About MasMovil group ● 4th telecom company in Spain ● Provides voice and broadband services to +12M customers ● Several

transparently insert security visibility + enforcement, but does so in a way that is based on service / pod / container identity (in contrast to IP address identification in traditional systems) and can filter this: ♻ Restarted unmanaged pod kube-system/event-exporter-gke-564fb97f9- rv8hg ♻ Restarted unmanaged pod kube-system/kube-dns-6465f78586-hlcrz ♻ Restarted unmanaged pod kube-system/kube-dns-autoscaler- Restarted unmanaged pod kube-system/l7-default-backend-7fd66b8b88- qqhh5 ♻ Restarted unmanaged pod kube-system/metrics-server-v0.3.6- 7b5cdbcbb8-kjl65 ♻ Restarted unmanaged pod kube-system/stackdr

transparently insert security visibility + enforcement, but does so in a way that is based on service / pod / container identity (in contrast to IP address identification in traditional systems) and can filter paths include with and without service load- balancing and various network policy combinations. The pod name indicates the connectivity variant and the readiness and liveness gate indicates success or failure 65s pod-to-a-79546bc469-rl2qq 1/1 Running 0 66s pod-to-a-allowed-cnp-58b7f7fb8f-lkq7p 1/1 Running 0 66s pod-to-a-de

transparently insert security visibility + enforcement, but does so in a way that is based on service / pod / container identity (in contrast to IP address identification in traditional systems) and can filter paths include with and without service load- balancing and various network policy combinations. The pod name indicates the connectivity variant and the readiness and liveness gate indicates success or failure 67s pod-to-a-allowed-cnp-87b5895c8-bfw4x 1/1 Running 0 68s pod-to-a-b76ddb6b4-2v4kb 1/1 Running 0 68s pod-to-a-denie

transparently insert security visibility + enforcement, but does so in a way that is based on service / pod / container identity (in contrast to IP address identification in traditional systems) and can filter paths include with and without service load- balancing and various network policy combinations. The pod name indicates the connectivity variant and the readiness and liveness gate indicates success or failure 4m50s pod-to-a-59b5fcb7f6-gq4hd 1/1 Running 0 4m50s pod-to-a-allowed-cnp-55f885bf8b-5lxzz 1/1 Running 0 4m50s pod-to-a-ext

transparently insert security visibility + enforcement, but does so in a way that is based on service / pod / container identity (in contrast to IP address identification in traditional systems) and can filter this: ♻ Restarted unmanaged pod kube-system/event-exporter-gke-564fb97f9- rv8hg ♻ Restarted unmanaged pod kube-system/kube-dns-6465f78586-hlcrz ♻ Restarted unmanaged pod kube-system/kube-dns-autoscaler- Restarted unmanaged pod kube-system/l7-default-backend-7fd66b8b88- qqhh5 ♻ Restarted unmanaged pod kube-system/metrics-server-v0.3.6- 7b5cdbcbb8-kjl65 ♻ Restarted unmanaged pod kube-system/stackdr

transparently insert security visibility + enforcement, but does so in a way that is based on service / pod / container identity (in contrast to IP address identification in traditional systems) and can filter for the TLS certificates between etcd peers to work correctly, a DNS reverse lookup on a pod IP must map back to pod name. If you are using CoreDNS, check the CoreDNS ConfigMap and validate that in-addr.arpa listed as wildcards next to cluster.local. You can validate this by looking up a pod IP with the host utility from any pod: host 10.60.20.86 86.20.60.10.in-addr.arpa domain name pointer cilium-etcd- 972nprv9dp

Kubernetes Endpoint Lifecycle Troubleshoo�ng Monitoring & Metrics Exported Metrics Cilium as a Kubernetes pod Cilium as a host-agent on a node Troubleshoo�ng Component & Cluster Health Connec�vity Problems Policy transparently insert security visibility + enforcement, but does so in a way that is based on service / pod / container iden�ty (in contrast to IP address iden�fica�on in tradi�onal systems) and can filter on The DaemonSet will automa�cally install itself as Kubernetes CNI plugin. K8s 1.15 K8s 1.14 K8s 1.13 K8s 1.12 K8s 1.11 K8s 1.10 kubectl create -f https://raw.githubusercontent.com/cilium/cilium/v1

何实现集群内服务间的高效互通、满足应用 SLA 诉求已成为数据中心面临的关键问题，对云基础设施提出了很高的要求。 基于 K8S 的云基础设施能够帮助应用实现敏捷的部署管理，但在应用流量编排方面有所欠缺，服务网格的出现很好的 弥补了 K8S 流量编排的缺陷，与 K8S 互补，真正实现敏捷的云应用开发运维。但随着对服务网格应用的逐步深入，当前服 务网格的代理架构，数据面引入了额外的时延底噪开销，已成为业界共识的性能问题。 网格加速能力：以典型的 service mesh 场景为例，使能 sockmap 网格加速能力之后，业务容器和 envoy 容器之间的通信将被 ebpf 程序短接，通过缩短通信路径从而达到加速效果，对于同节点上 Pod 间通信也能通过 ebpf 程序进行加速。 功能描述 OS (ipstack + iptables) 服务 A 服务 B 服务 A 服务 B 服务治理 流量治理 流量治理 服务治理 OS 服务网格场景：优化云原生服务网格内服务通信性能。例如电商、计费、金融、物流、短视频、在线会议、云游戏等 对服务时延敏感的应用。 相关使用方式请参考 Kmesh 使用介绍。 应用场景 Pod1 Pod2 socket socket socket 业务程序 envoy socket socket socket envoy server Node 特性增强 21 openEuler

互联的基础。 面向未来，社区将持续创新、社区共建、繁荣生态，夯实数字基座。 夯实云化基座 • 容器操作系统 KubeOS：云原生场景，实现 OS 容器化部署、运维，提供与业务容器一致的基于 K8S 的管理体验。 • 安全容器方案：iSulad+shimv2+StratoVirt 安全容器方案，相比传统 Docker+QEMU 方案，底噪和启动时间优化 40%。 • 双平面部署工具 eggo：Arm/x86 社区孵化的云底座操作系统，集成了 rpm-ostree 支持、ignition 配置等技术。采用双根文件系统、原 子化更新的设计思路，使用 nestos-assembler 快速集成构建，并针对 K8S、OpenStack 等平台进行适配，优化容器运行底噪，使 系统具备十分便捷的集群组建能力，可以更安全的运行大规模的容器化工作负载。 1. 开箱即用的容器平台：NestOS 集成适配了 iSulad、Docker、Podman • 精细化性能 Profiling：提供多维度（包括系统、进程、容器、Pod 等多个维度）、高精度（10ms 采样周期）的性能（包括 CPU 性能、 内存占用、资源占用、系统调用等类型）火焰图、时间线图，可实时在线持续性采集。 • K8S Pod 全栈可观测及诊断：提供 K8S 视角的 Pod 集群业务流实时拓扑能力，Pod 性能观测能力、DNS 观测能力、SQL 观测 能力等。 A-Ops