Cluster Services API Cluster3 NSX-T vSphere PKS • Includes • PKS Controller, NSX-T • CFCR, Harbor, Broker • Deploys & Configures - CFCR - vSphere - NSX-T Integration - Harbor • Manages Cluster create-cluster K8s-3 n=3 #pks resize K8s-3 n=5 Architecture NSX-T Bosh PKS Admin Network NCP POD 1 POD 4 POD 2 POD 3 POD 5 POD 6 T0 kube-system PODs – Logical Switch Namespace ‘foo’ PODs – ‘VM’ Pod 3 Pod 4 Worker ‘VM’ Pod 5 Pod 6 T1 T1 T1 T1 VMware vSphere NSX Manager NSX Controllers T1 NSX Edge Cluster Architecture NSX-T • NSX Container Plugin: NCP is a software component

��Iptables������� IPVS��Service���� Iptables vs. IPVS Kubernetes�Service ����onl��a�o� - ��������������t� - ���������� - �����IP�n������� - �������� - ��������� Kubernetes Service�Endpoints Label Selector Ø DNAT��IPP��oI�X iptables -t nat -A PREROUTING -d 1.2.3.4/32 --dport 80 -j DNAT --to-destination 10.20.30.40:8080 • IptablesU����R�� Ø statistic�S����o���� iptables -t nat -A PREROUTING -d 1.2.3.4 .25 -j DNAT --to-destination 10.20.30.40:8080 • IptablesU����>�� Ø recent�S����>��� iptables -t nat –A FOO -m recent --rcheck --seconds 3600 --reap --name BAR -j BAR Iptables�Kubernetes������ ��)�)���

com/GoogleContainerTools/jib What did we do? 1. Write first Dockerfile 2. Reduce image size 3. Don’t run installs 4. Use better base image 5. Write .dockerignore 6. Improve incremental speed 7. Switch com/GoogleContainerTools/jib What did we do? 1. Write first Dockerfile 2. Reduce image size 3. Don’t run installs 4. Use better base image 5. Write .dockerignore 6. Improve incremental speed 7. Switch com/GoogleContainerTools/jib What did we do? 1. Write first Dockerfile 2. Reduce image size 3. Don’t run installs 4. Use better base image 5. Write .dockerignore 6. Improve incremental speed 7. Switch

status: availableReplicas: 2 conditions: - lastTransitionTime: 2016-10-04T12:25:39Z lastUpdateTime: 2016-10-04T12:25:39Z message: Replica set "nginx-deployment-4262182780" is progressing status: "True" type: Progressing - lastTransitionTime: 2016-10-04T12:25:42Z 16-Deployment 65 lastUpdateTime: 2016-10-04T12:25:42Z message: Deployment has minimum availability. reason: e status: "True" type: Available - lastTransitionTime: 2016-10-04T12:25:39Z lastUpdateTime: 2016-10-04T12:25:39Z message: 'Error creating: pods "nginx-deployment-4262182780-" is

time I help run a Kubernetes SIG.. Kris Nova . . t h a t b r i n g s a n o p e n s o u r c e p r o j e c t . . Kris Nova ..to Kris Nova Microsoft ACS ..while falling behind Kops 1.5 ..also Kops 1.5 ..also 700 open GitHub issues Kops 1.5 ..also We couldn’t test our “text/template” code Kops 1.5 ..also We would still get panics at runtime.. Kops 1.5

可觀測性 (Observability) 在 Kubernetes Day2 Operation的考量與實踐 E . W. K u o @ i T h o m e K u b e r n e t e s S u m m i t 2 0 2 2 Click to edit Master title style 2 “ 二哥 2 Wistron DX Lab 緯創數位轉型技術實驗室 7 Click to edit Master title style 8 Challenge of Kubernetes Day 2 Operation 運 營 K u b e r n e t e s 的 挑 戰 8 Click to edit Master title style 9 Kubernetes Day2 Ops 要作那些事? • 集群標準化和生命週期管理 • 安全訪問和環境隔離

云原生改造实践 ❖ k8s 规模及性能优化实践 ❖ 云原生应用管理演进路线 主要内容阿里巴巴容器的发展历程 2013 初步探索 使用容器的方式替换传统使用 VM 部署应用的，基于 lxc 自研 了 t4 容器并构建了 AI 集团管理 系统 2017 统一资源池 构建了 Sigma 调度系统，收敛了 众多运维平台之下的资源调度系 统，并构建了集团统一资源池， 在此基础上发展出弹性、混部等 技术成果，大幅降低了数据中心 Ready Cache Read & Index APIServer Client list/get @t0 ETCD rv=nil 1. Get rv@t0 Cache 2. Request Notify Index rv Reflector 3. Wait rv > rv@t0 Add Indexs 1. nodename 2. Namespace 3. Labels …… Describe

GitCommit:"89a4ea3e1e4ddd7f7572286090359983e0387b2f", GitTreeState:"clean", BuildDate:"2023-09-13T09:34:32Z", GoVersion:"go1.20.8", Compiler:"gc", Pla�orm:"linux/amd64"} # kubeadm config images �gera-operator NAME READY STATUS RESTARTS AGE �gera-operator-59b69c49dc-t6crt 1/1 Running 1 (10m ago) 51m # kubectl get pods -n calico-system NAME calico-node-qn7cp 1/1 Running 1 (10m ago) 11m calico-typha-84cb54bfd-hh57t 1/1 Running 1 (10m ago) 11m csi-node-driver-fn7zd

114.114 IPV6INIT=no 打开虚机网络： Step3. 虚拟机磁盘 2 分区&格式化 fdisk -l fdisk /dev/sdb 依法选择 n,p,1,t,l,8e,w fdisk –l pvcreate /dev/sdb1 vgdisplay vgextend centos /dev/sdb1 vgdisplay lvcreate -l GLuxURTJXuc 26OR+knAxi7vP3Gwo5Wz6mOmdklNsMWsO3E4BJA35eIWYBV7QHiYar8hl9k3XqIN C9GvsJRpv3T3ZJxHmjrUerwg3lNtLAN4nQEIXd6HRmfVuDmFyM9fCv/oA5NoAz32 lYTz58B7kOEkq83Rh5MBI4785tefcQTcABJPPXJv wxa5Wl WY2HO5fqv1VOSqEUD2yKc/QdMwYi2eh7l7cMrVM7YCerFj33NNFbccXcwgmDgo5J fiYQLElOC0xlGT9t/QA= -----END LICENSE KEY----- Step5. templates/statefulset.yaml vi openservice/templates/statefulset

sensitive resources ● Common attack vector ○ Checked into Github ○ Accessible by users who shouldn’t have access, e.g., CEO ○ Stored in public storage buckets Secret management requirements Identity DEKs are encrypted at rest ● Don’t use the same DEK to encrypt data from two different apps/users ● Generate a new DEK every time you write the data. This means you don't need to rotate the DEKs Managing