VMware SIG Deep Dive into Kubernetes Scheduling Performance and high availability options for vSphere Steve Wong, Hui Luo VMware Cloud Native Applications Business Unit November 12, 2018 2 Open across availability zones, while still respecting resource access and availability concerns. When Kubernetes runs on vSphere, the hypervisor platform also supports an underlying tier of high availability ​This session will explain the options to gain better performance, resource optimization and availability through tuning of vSphere, and Kubernetes configuration and labeling. This is applicable to any

VMware SIG Deep Dive into Kubernetes Scheduling Performance and high availability options for vSphere Steve Wong, Michael Gasch KubeCon North America December 13, 2018 2 Open Source Community Relations across availability zones, while still respecting resource access and availability concerns. When Kubernetes runs on vSphere, the hypervisor platform also supports an underlying tier of high availability This session will explain the options to gain better performance, resource optimization and availability through tuning of vSphere, and Kubernetes configuration and labeling. This is applicable to any

Pipeline / Stage/ Task / Task Template / Version Control • Logging, monitoring, autoscaling, high availability • Extensibility / Integration • CI/CD examples • Future plan Our DevOps Expectations ask Template/Version Control/UI generation/Volume... • Logging, monitoring, autoscaling, high availability • Extensibility/Integration • CI/CD examples • Future plan Overall Architecture Kubernetes Tasks in same stage can run sequentially or in parallel Logging, Monitoring, Autoscaling, High Availability Kubernetes Cluster Node Node Node Node Job Job Job Job Pod Pod Pod ElasticSearch ElasticSearch

Methods to achieve high SLOs on a large scale Kubernetes cluster Kang FAN, Jinghua YAO Why SLO? SLO (Service-Level Objective). Within service-level agreements (SLAs), SLOs are the objectives that must process — to provide the best opportunity for service recipient success. — Gartner Latency SLI Availability QPS Correctness SLO …… Punishment SLA SLI defines an indicator, which can represent user number value indicates the count of pods that can not be deleted in a certain period. 5. Nodes Availability A number value indicates the number of unhealthy node in the cluster. Pods scheduled to unhealthy

在该VM上运⾏⽤户的容器。请阅读 Building High-Availability Clusters 以实现多主机VM配置。 kube-apiserver kube-apiserver 暴露Kubernetes的API。它是Kubernetes控制能⼒的前端。它被设计为可⽔平扩展——也就是通过部署 更多实例来实现扩容。详⻅ Building High-Availability Clusters 。 etcd Controller将驱逐速率限制为 --node-eviction-rate （默认为0.1）每秒，这意味着它不会1个 Node驱逐Pod花费的时间不会超过10秒。 当给定可⽤区（availability zone）中的Node变得不健康时，Node驱逐的⾏为就会发⽣变化。Node Controller同时也会 检查区域中有多少百分⽐的Node是不正常的（NodeReady condit 区（整个群集）。 译者按：可⽤区举例：AWS可⽤区：http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions- availability-zones.html 在可⽤区之间传播Node的⼀个关键原因是：当整个区域停⽌时，⼯作负载可以转移到健康区域。因此，当⼀个区域中 的所有Node都不健康时，那么Node Controller就以正常速率

changes from current to desired • Policy-based workload scheduler • Topology aware • Assists with availability, performance and capacity • Affinity/Anti-Affinity Capable The Kubernetes Worker Node Basic Operations PKS simplifies Day 2 operations with built-in network security—powered by NSX, high availability, logging, monitoring, analytics, and automated health checks. On-Demand Provisioning PKS

MTTR Project X was used as deployment tool for Go-Jek International Expansion Projects • High availability DBs lead to fewer outage Higher Uptime • System resources like CPU, memory, etc. are more

/mutating-secret • Annotation: /storage-transform-disable= • Emergency management • High Availability guarantee • KMS • API server & kms-plugin • Cron job backup for KEKs (from KMS) • Static

and Offline installation ● Private container registry support ● Latest 3 versions of k8s ● High-availability control plane ● Auto-repair Installation and Configuration $ gke-on-prem create cluster --dry-run

日志、系统指标等采集、分析；监控配置与报警；性能 指标收集与分析等等。 Backup & Restore 备份策略、备份方式、恢复方式、备份管理等等。 Disaster Recovery & High Availability Failover/Switchover、多可用区、数据恢复等等。 Security & Compliance 访问控制、审计、安全链接、加密存储等等。 Patching &