#安装k8s二进制组件 （<=1.23版本） # systemctl enable kubelet # systemctl start kubelet ③k8s集群初始化 # kubeadm version #先查看k8s版本 # GitVersion:"v1.19.4" # kubeadm config images list #查看k8s其他组件的docker镜像名，默认用 7个镜像） ★直接使用命令行方式初始化集群 （以下是非HA模式的master初始化，如果要部署高可用集群，则参考第4章） kubeadm init --kubernetes- version=v1.19.4 \ --apiserver-adver�se- address=10.99.1.51 \ # api server地址 --pod-network-cidr=10.244 kubeadm init --config /etc/kubeadm-init.yaml #初始化集群 当出现Your Kubernetes control-plane has ini�alized successfully!这行时说明初始化 k8s成功了 记住最后2行命令，是用来让node结点加入集群的命令（含token） ★第2章、部署k8s版本>=1

Architecture and Features • CRD and operator design • Pipeline / Stage/ Task / Task Template / Version Control • Logging, monitoring, autoscaling, high availability • Extensibility / Integration • CI/CD • Architecture and Features • CRD and operator design • Pipeline/Stage/Task/Task Template/Version Control/UI generation/Volume... • Logging, monitoring, autoscaling, high availability • Extensibility/Integration / Job status Pipeline / Stage / Task Task Template Pipeline / Stage / Task build logs Version Control sync / watch clean history jobs Basic Concepts（partial） Repository Managed Project Pipeline

configure, and manage clusters in GKE and GKE On-Prem ● Cluster environments are consistent (k8s version, OS image, plug-ins, components configuration) Orchestrate and manage on-prem containers just installation ● Private container registry support ● Latest 3 versions of k8s ● High-availability control plane ● Auto-repair Installation and Configuration $ gke-on-prem create cluster --dry-run Welcome want to install your cluster? [1] vSphere v6.5 Please enter your numeric choice [1]: 1 What version of GKE On-Prem do you want to install? [1] 1.10.3 (Uses k8s v1.10.3) [2] 1.9.2-rc2 (Uses K8s 1.9

逐渐成为开发 operator 的首选 Operator Pattern 是官方定义的标准扩 展机制，是 K8s Native Application; Operator = CRD + control loop, i.e, Declaretive API + Automation; kubebuilder + controller-runtime + helm Operator Operator understands how to upgrade older versions of the Operand, managed previously by an older version of the Operator Upgrade of the Operator • Operator can be upgraded seamlessly and can either still versions of the Operand or update them • Operator conveys inability to manage an unsupported version of the Operand in the status section of the CR Lifecycle features • Operator provides the ability

cluster Need an initial control plane to bootstrap a self-hosted cluster Bootkube: ● Acts as a temporary control plane long enough to be replaced by a self-hosted control plane. ● Run only on very loss of control plane components (Kubernetes) Power cycling the entire control plane (Kubernetes) Permanent loss of control plane (External tool) Disaster Recovery Permanent loss of control plane ● running v1.4.3 and configured to run v1.4.5 ● API Server is v1.4.3 ● Scheduler is v1.4.3 Kubernetes Version Operator Differences from desired config ● API Server should be v1.4.5 ● Scheduler should be v1

RollingUpdate ? Ø What are the advantages of batch gray release ? • more reliable and better control • More flexible • More efficient StatefulSetPlus StatefulSetPlus Service (Kube-proxy, CLB, etc (Vertical Workload Autoscaler) Ø Keep share memory during Pod upgrade Ø Scaled Up with LGV (Last Good Version) Ø Per Pod Per PV Ø Per Workload Per PV Ø Pod Auto Migrate when Node Abnormal Ø Gray Release Pod biz-container:v2 EmptyDir Volume version1=1 version2=1 filelock.lc EmptyDir Volume version1=1 version2=2 filelock.lc EmptyDir Volume version1=2 version2=2 filelock.lc ������������� ���������������

its Affiliates. All rights reserved. Amazon Confidential Amazon EKS 服务路线图摘要 已发布 - Amazon EKS control plane logs - Support for public IP space in VPC - Amazon EKS: Deep Learning Benchmarking Utility 0 - New Regions: Hong Kong 即将发布 - Service linked role for Amazon EKS - EKS Support for K8s version 1.13 + ECR AWS PrivateLink - EKS-optimized AMI metadata SSM parameter - IAM for Pods - New Amazon NODE 配置 升级 加固 监控 NETWORK 配置 VPC 网络策略 路由表 NACLs 数据 网络流量保护 客户端加密 服务端加密 EKS CONTROL PLANE CONTROL PLANE 配置 PRIVATE CONTROL RBAC 策略 © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved

also supports an underlying tier of high availability and automated placement options, for both control plane and worker nodes. 2 levels of scheduling and resource management are active. ​Currently performance effects. (e.g interleaving get predictable albeit reduced performance) • A cgroup aware version (e.g. Java jre v10) can be deployed • This is often not available – many were developed in a enforcement takes place Kubernetes -> container runtime -> Linux -> hypervisor (optional) ​Kubernetes control plane manages desired policy. ​Enforcement passes Pod -> container runtime -> Linux OS ​Cgroups

also supports an underlying tier of high availability and automated placement options, for both control plane and worker nodes. 2 levels of scheduling and resource management are active. Currently no performance effects. (e.g interleaving get predictable albeit reduced performance) • A cgroup aware version (e.g. Java jre v10) can be deployed • This is often not available – many were developed in a pre-container enforcement takes place Kubernetes -> container runtime -> Linux -> hypervisor (optional) Kubernetes control plane manages desired policy. Enforcement passes Pod -> container runtime -> Linux OS Cgroups are

©2019 VMware, Inc. 7  關閉公開存取 (Disable public access)  實施角色型存取權控管 (Implement role-based access control)  將 Kubernetes密鑰加密 (Encrypt secrets at rest)  設置 Kubernetes 的許可控制器 (Configure admission controllers) container images)  開啟稽核日誌 (Enable audit logging)  跟上最新的 Kubernetes版本 (Keep your Kubernetes version up to date) Kubernetes Security Best Practices Kubernetes安全性的最佳實務指導 資料來源: https://blog.sqreen org/benchmark/kubernetes/ 控制措施 如何查核 如何查核 參考資訊 預設配置 原因理由 如何查核 1. 控制平面元件 (Control Plane Components) 2. etcd 狀態資料庫 3. 控制平面設置 (Control Plane Configuration) 4. 工作節點 (Worker Node) 5. 政策 (Policies) ©2019