Scaling a Multi-Tenant k8s Cluster in a Telco Pablo Moncada October 28, 2020 About MasMovil group ● 4th telecom company in Spain ● Provides voice and broadband services to +12M customers ● Several Services +3k CPU +2k Mem +5TB Nodes +300 kube-proxy replacement NetworkPolicy logging Multi-cluster DNS Aware NetworkPolicy Increased Istio security External Services TLS visibility Performance

org, home of the Network Time Protocol project16 • The pool.ntp.org projecti, being a big virtual cluster of timeservers.17 • Freedesktop.org info on timedatectl18 • Freedesktop.org info on systemd-timesyncd Section, Multipaths Device Configuration Attributes. 2.2. Consistent Multipath Device Names in a Cluster When the user_friendly_names configuration option is set to yes, the name of the multipath device multipath.conf configuration file, the name is not automatically consistent across all nodes in the cluster. This should not cause any difficulties if you use LVM to create logical devices from the multipath

your datacentre, public or private. Whether you want to deploy an OpenStack cloud, a Kubernetes cluster or a 50,000-node render farm, Ubuntu Server delivers the best value scale-out performance available create -f qcow2 disk-image.qcow2 10G Formatting 'disk-image.qcow2', fmt=qcow2 size=10737418240 cluster_size=65536 lazy_refcounts=off refcount_bits=16 qemu-img info disk-image.qcow2 image: disk-image disk-image.qcow2 file format: qcow2 virtual size: 10 GiB (10737418240 bytes) disk size: 196 KiB cluster_size: 65536 Format specific information: 78 compat: 1.1 lazy refcounts: false refcount bits: 16

33 - 本文档使用 书栈(BookStack.CN) 构建 较好一点！因此，对于主机的安全要求就需要严格的要求啦！就鸟哥的观点来看， 如果你的主机是用来替你赚钱的， 例如某些研究单位的大型 Cluster 运算主机， 那么即使架设一个甚至让你觉得很不方便的防火墙系统，都是合理 的手段！因为主机被入侵就算了，若数据被窃取，呵呵！ 那可不是闹着玩的！ 由上面的整个架站流程来看，由规划到安装、主机 会主动的帮你重组而进行传送，差一点的可能就直接回报这个封包无效而丢弃了～这个时候可 就糗大啰～ 所以， MTU 设定为 9000 这种事情，大概仅能在内部网络的环境中作～举例来说，很多的内部丛集系 统 (cluster) 就将他们的内部网络环境 MTU 设定为 9000，但是对外的适配卡可还是原本的标准 1500 喔！ ^_^ 2.2.5 MTU 最大传输单位 4.2. 2.2 TCP/IP 的链结层相关协议 switch 啦！因为 10/100/1000Mbps 的 switch 要比 10/100Mbps 的设备快上十倍，速度可是差很多的啊！如果你的 设备还需要更快时， 例如鸟哥之前服务的实验室内部的 cluster (丛集式计算机群) ，则购买的 switch 甚至需 要支持 Jumbo frame 这种支持大讯框的硬件架构才行，否则速度上不来啊！ 网络线：考虑与速度相配的等级、线材形状、施工配线等

过程中长时间等待的问题。 更多信息详见文档：高可用。 • 高性能 相对 MySQL 及 Percona Server For MySQL 的性能表现更稳定优异，支持高性能的内存查询加速 AP 引擎、InnoDB 并行查询、 并行 LOAD DATA、事务无锁化、线程池等特性，在 TPC-C 测试中相对 MySQL 性能提升超过 30%，在 TPC-H 测试中的性能 表现是 MySQL 的十几倍甚至上百倍。 HeatWave 的大规模并行、高性能的内存查询加速 AP 引擎，可将 GreatSQL 的数据分析性能提升几个数量级。 - 支持 InnoDB 并行查询，适用于轻量级 OLAP 应用场景，在 TPC-H 测试中平均提升 15 倍，最高提升 40+ 倍。 - 优化 InnoDB 事务系统，实现了大锁拆分及无锁化等多种优化方案，OLTP 场景整体性能提升约 20%。 - 支持并行 LOAD DATA，

Advanced Networking Cluster Mesh Operations Istio Concepts Component Overview Terminology Networking Network Security eBPF Datapath Observability Kubernetes Integration Multi-Cluster (Cluster Mesh) Getting Troubleshooting Component & Cluster Health Observing Flows with Hubble Observing flows with Hubble Relay Connectivity Problems Policy Troubleshooting etcd (kvstore) Cluster Mesh Troubleshooting Symptom clusters? What is the 95th and 99th percentile latency between HTTP requests and responses in my cluster? Which services are performing the worst? What is the latency between two services? Security observability

Advanced Networking Cluster Mesh Operations Istio Concepts Component Overview Terminology Networking Network Security eBPF Datapath Observability Kubernetes Integration Multi-Cluster (Cluster Mesh) Getting Troubleshooting Component & Cluster Health Observing Flows with Hubble Observing flows with Hubble Relay Connectivity Problems Policy Troubleshooting etcd (kvstore) Cluster Mesh Troubleshooting Symptom clusters? What is the 95th and 99th percentile latency between HTTP requests and responses in my cluster? Which services are performing the worst? What is the latency between two services? Security observability

Terminology Networking Network Security eBPF Datapath Observability Kubernetes Integration Multi-Cluster (Cluster Mesh) Getting Help FAQ Slack GitHub Training Enterprise support Security Bugs Operations Scalability report Performance Evaluation Setup Evaluation Results Tuning Troubleshooting Component & Cluster Health Observing Flows with Hubble Observing flows with Hubble Relay Connectivity Problems Policy clusters? What is the 95th and 99th percentile latency between HTTP requests and responses in my cluster? Which services are performing the worst? What is the latency between two services? Security observability

Overview Terminology Networking Network Security eBPF Datapath Kubernetes Integration Multi-Cluster (Cluster Mesh) Getting Help FAQ Slack GitHub Security Bugs Operations System Requirements Summary Scalability report Performance Evaluation Setup Evaluation Results Tuning Troubleshooting Component & Cluster Health Observing Flows with Hubble Observing flows with Hubble Relay Connectivity Problems Policy clusters? What is the 95th and 99th percentile latency between HTTP requests and responses in my cluster? Which services are performing the worst? What is the latency between two services? Security observability

Agent Monitoring & Metrics Installation cilium-agent cilium-operator Troubleshooting Component & Cluster Health Connectivity Problems Policy Troubleshooting Symptom Library Useful Scripts Reporting a problem requires the firewalls on all servers to be manipulated whenever a container is started anywhere in the cluster. In order to avoid this situation which limits scale, Cilium assigns a security identity to groups store. Secure access to and from external services Label based security is the tool of choice for cluster internal access control. In order to secure access to and from external services, traditional CIDR