要求docker<=20.10 k8s 1.24及之后版本： kubelet→cri-containerd→containerd→runC 后来cri-containerd重构进containerd中（CRI Plugin），合为一个containerd进程 默认调用的cri-socket： unix:///var/run/containerd/containerd.sock 本小节讲解k8s v1 #k8s v1.23支持的docker 最新版本为20.10.x # systemctl enable docker # systemctl start docker # docker info ★配置docker服务使用systemd去管理（以及信任本地镜像仓库） # vi /etc/docker/daemon.json { "data-root": "/docker_data" "exec-opts": [ "na�ve.cgroupdriver=systemd" ] } # mkdir -p /etc/systemd/system/docker.service.d # docker info ★docker会修改防火墙规则，导致pod网络不通 # vi /usr/lib/systemd/system/docker.service #在[Service]下的ExecStart=/usr/bin/dockerd

Pod Pod Pod Build task configuration - map to k8s Job, can also be a raw k8s job Job / Pod / Node info BuildJob / Job status Pipeline / Stage / Task Task Template Pipeline / Stage / Task build logs convention • Get the commits between two builds • Invoke Jira API to mark the status and add the build info • Call harbor API to add/remove label of image, can also lock the passed image Agenda • Our management tools • Optimize UI generation methodology • Improve development experience, such as CLI, plugin for IDE, dev on Cloud • Move forward to better DevOps under micro-service architecture • Consolidate

EncryptionConfig uses aescbc with a KMS provider ● Sidecar pod for the KMS plugin Master kube-apiserver etcd kms-plugin SECRETDEK DEKKEK KEK Terminology and Notation DEK Data encryption key KEK kube-apiserver etcd kms-plugin SECRET KMS 1.10 Kube-ApiServer Generates a DEK Master kube-apiserver etcd kms-plugin SECRET KMS 1.10 Kube-ApiServer Sends DEK to Plugin Master kube-apiserver kube-apiserver etcd kms-plugin Encrypt(DEK) SECRET KMS 1.10 Plugin Forwards to KMS Master kube-apiserver etcd kms-plugin Encrypt(DEK) SECRET Encrypt(DEK) KMS 1.10 KMS Encrypts a DEK Master kube-apiserver

KMS Plugin [1] • Address performance & latency concerns • Reduce / minimize remote KMS interactions w/o compromising security • Address security threats • etcd compromise • Host (KMS plugin) compromise compromise Ø leak DEKs Ø leak KEKs [1] KubeCon NA 2019: "TEE-based KMS Plugin for encryption of Kubernetes Secrets”, by Raghu Yeluri & Haidong Xia, Intel Corp. TEE-based KMS Provider • Address security Experience @ Ant Group KMS Plugin • Workflow • Encryption • Decryption • Engineering decisions • apiserver is responsible for • DEK generation • Secret en/decryption • kms-plugin • keeps KEK cache • only

com/GoogleContainerTools/jib ... <plugin> com.spotify dockerfile-maven-plugin 1.4.8 io/petclinic-app ${project.version} plugin> ... github.com/GoogleContainerTools/jib What did we do? 1. Write first better base image 5. Write .dockerignore 6. Improve incremental speed 7. Switch to use a Maven plugin github.com/GoogleContainerTools/jib What did we do? 1. Write first Dockerfile 2. Reduce image

vSphere NSX Manager NSX Controllers T1 NSX Edge Cluster Architecture NSX-T • NSX Container Plugin: NCP is a software component provided by VMware in form of a container image, runs in K8s as a standardized interface to the NSX API Network Container Plugin (NCP) NSX Manager Kubernetes Master etcd API-Server Scheduler NSX Container Plugin (NCP) NSX Infra NSX Manager API Client Kubernetes Creation Workflow NSX Manager NS: foo NS: bar NSX / Kubernetes Topology C C C C NSX Container Plugin (NCP) NSX Infra NSX Manager API Client Kubernetes Adapter 1. NCP creates a ‘watch’ on K8s

EKS private endpoints - New Amazon EKS Regions: Sao Paulo, Canada Central - Next-generation CNI plugin © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential © All rights reserved. Amazon Confidential 开源与 Amazon EKS Amazon EKS 的主要模块已经开源 • Amazon VPC CNI plugin • AWS IAM authenticator • Amazon EKS AMI AWS团队贡献或管理着超过20个与Kubernetes相关的开源项目 • /kubernetes • 简单安全 GitHub开源 … { } Amazon VPC CNI Plugin 支持 © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential Amazon VPC CNI plugin Elastic network interface Secondary

Deamonset Node should be tainted when critical Daemonset is unhealthy. Case 4: Plugin registry Registration of plugin such as CSI plugin should be checked. Case 5: Capacity The QPS Limit and Capacity Limit should

Platform bar Serverless baz Common Workload Types Manual Scaler K8s Operators Kubernetes + OAM K8s Plugin HPA Deployment scale-to-0 Function Unified Model Layer Platform Capability Pool 统一的模型层 平台统一“能力池” Push Docker Image Image Registry AutoScaling Controller Rollout Controller GitOps OAM K8s Plugin + CUE Abstraction Processor Kubernetes traffic code Raw k8s API resources 面向应用开发者的 appfile

Container Storage Interface (CSI) is a standard API allowing a storage provider to write just one plugin that will work for all major container orchestration systems: Kubernetes, Mesos, Docker and Cloud