Bins/Libs Container Engine Docker Host Kubernetes Slave Kubernetes Master P1R3 P2R2 P2R2 P1R2 P1R2 P2R1 P1R1 P1R1 P2R1 P1R1 P2R1 App_X.yaml ContainerImage1 Replicas: 3 ContainerImage2 Replicas: 2 Kubernetes vSphere NSX Manager NSX Controllers T1 NSX Edge Cluster Architecture NSX-T • NSX Container Plugin: NCP is a software component provided by VMware in form of a container image, runs in K8s as a standardized interface to the NSX API Network Container Plugin (NCP) NSX Manager Kubernetes Master etcd API-Server Scheduler NSX Container Plugin (NCP) NSX Infra NSX Manager API Client Kubernetes

EKS private endpoints - New Amazon EKS Regions: Sao Paulo, Canada Central - Next-generation CNI plugin © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential © All rights reserved. Amazon Confidential 开源与 Amazon EKS Amazon EKS 的主要模块已经开源 • Amazon VPC CNI plugin • AWS IAM authenticator • Amazon EKS AMI AWS团队贡献或管理着超过20个与Kubernetes相关的开源项目 • /kubernetes • 简单安全 GitHub开源 … { } Amazon VPC CNI Plugin 支持 © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Amazon VPC CNI plugin Elastic network interface Secondary

要求docker<=20.10 k8s 1.24及之后版本： kubelet→cri-containerd→containerd→runC 后来cri-containerd重构进containerd中（CRI Plugin），合为一个containerd进程 默认调用的cri-socket： unix:///var/run/containerd/containerd.sock 本小节讲解k8s v1 sandbox_image = "cof-lee.com:5443/k8s/pause:3.9" #和k8s需要的pause镜 像版本保持一致 #如果要启用CRI-Plugin，注释掉其中的 disabled_plugins = ["cri"] #再重启containerd即可有 unix:///run/containerd/containerd.sock 接口 #信 面了 # kubectl get secret | grep sa-k8s-dashboard # kubectl describe secret sa-k8s-dashboard-token-7r699 | grep token # 或者使用kubeconfig文件登录，先创建sa-k8s-dashboard服务账号的config认证 文件 #获取集群初始化信息 kubectl config

22190 7261 16462 4861 17442 5548 21461 6828 0 5000 10000 15000 20000 25000 TCP_RR(r/s) TCP_CRR(r/s) Overlay方案性能 host vxlan ipip gateway 23540 8368 22127 7675 21007 7231 0 5000 10000 cephFS ceph RBD ceph RBD 权限管理 quota 在线扩容 containe r containe r containe r 本地磁盘 containe r 空间上报 自动调度 共享云盘 containe r 内置云盘 containe r • 基于本机磁盘 • 上报至调度器 • 作为资源进行调度 本地磁盘 2类存储，3种场景 本地磁盘：延时低，不可迁移 ConvGPU 仅支持内存资源的共享且仅处理单个GPU 容器使用GPU的问题： • 需要特定的硬件设备 • 不支持容器共享 • 仅支持内存资源虚拟化 • 仅支持单个GPU卡 采用Device Plugin： • GPU资源的发现 • 为任务分配相应的硬件 资源及配置容器运行时环境 transparent. GaiaGPU不应修改Kubernetes代码或容器镜像以共享GPU。使用共享GPU执行应用程序应该就像

Nexus Sonarqub e Jenkin s Slelenium RedWoodH Q Clair Registr y Elastic kubernet es Docke r 总体流程设计 u 在概念阶段完成产品⽴项评审 u 在迭代0阶段完成系统总体架构审 计，总体架构设计完成之后，启 动迭代启动评审。 u 在迭代阶段包含1.需求分析、2.应 ⽤设计、3.开发、4 K u bernetes In tegration API S e r ver C o r e /C ustom R e s ourc es API S e r ver C o r e /C ustom R e s ourc es API S e r ver C o r e /C ustom R e s ourc es Oth er tool s/systems In 平台管理员创建／分配资源 租户管理员订阅⼯具 租户管理员创建／分配⼯具资源 容器化持续集成、持续交付 • Jenkins + Kubernetes • Alauda-Jenkins-Plugin/DSL • 流水线模版 • 图形化模块 • 用户打通 • 权限同步 • Jenkins/Pipeline CRDs/Custom Controllers/API Aggregation

Container Storage Interface (CSI) is a standard API allowing a storage provider to write just one plugin that will work for all major container orchestration systems: Kubernetes, Mesos, Docker and Cloud Status within the Kubernetes project 9 Moving out of tree: the CSI Provider Why it exists Handles C/R/U/D of storage volumes Coordinate storage with availability zones Controls advanced storage functionalities

Mesos. l Spark 2.3 added native support for Kubernetes. l Spark 2.4 added support for client mode, R, python etc. l Spark 3.0 will add support for dynamic resource allocation, external shuffle service task-topology to improve the spark workload efficiency. Summary p Queue priority p Queue reclaim p Queue plugin p Hierarchical queue p Dynamic resource allocation p External shuffle service p Resource reservation

EncryptionConfig uses aescbc with a KMS provider ● Sidecar pod for the KMS plugin Master kube-apiserver etcd kms-plugin SECRETDEK DEKKEK KEK Terminology and Notation DEK Data encryption key KEK kube-apiserver etcd kms-plugin SECRET KMS 1.10 Kube-ApiServer Generates a DEK Master kube-apiserver etcd kms-plugin SECRET KMS 1.10 Kube-ApiServer Sends DEK to Plugin Master kube-apiserver kube-apiserver etcd kms-plugin Encrypt(DEK) SECRET KMS 1.10 Plugin Forwards to KMS Master kube-apiserver etcd kms-plugin Encrypt(DEK) SECRET Encrypt(DEK) KMS 1.10 KMS Encrypts a DEK Master kube-apiserver

KMS Plugin [1] • Address performance & latency concerns • Reduce / minimize remote KMS interactions w/o compromising security • Address security threats • etcd compromise • Host (KMS plugin) compromise compromise Ø leak DEKs Ø leak KEKs [1] KubeCon NA 2019: "TEE-based KMS Plugin for encryption of Kubernetes Secrets”, by Raghu Yeluri & Haidong Xia, Intel Corp. TEE-based KMS Provider • Address security Experience @ Ant Group KMS Plugin • Workflow • Encryption • Decryption • Engineering decisions • apiserver is responsible for • DEK generation • Secret en/decryption • kms-plugin • keeps KEK cache • only

com/GoogleContainerTools/jib ... <plugin> com.spotify dockerfile-maven-plugin 1.4.8 io/petclinic-app ${project.version} plugin> ... github.com/GoogleContainerTools/jib What did we do? 1. Write first better base image 5. Write .dockerignore 6. Improve incremental speed 7. Switch to use a Maven plugin github.com/GoogleContainerTools/jib What did we do? 1. Write first Dockerfile 2. Reduce image