GPU Resource Management On JDOS 梁永清 liangyongqing1@jd.com 提供的服务 1. 用于实验的 GPU 容器 2.基于 Kubeflow 的机器学习训练服务 3.模型管理和模型 Serving 服务 Experiment Training Serving 均基于容器，不对业务方直接提供 GPU 物理机 GPU 实验 JDOS 常规的容器服务

Node Operator: Kubernetes Node Management Made Simple 陈俊(Joe), Ant Financial Agenda • Background and Motivation • Introduction of Operators • Node-Operator • Advanced Topic: • Upgrade Master & Node Components reliably • Canary Rollout • Master & Node Component Versions Management Motivation: Work Order Deployment Worker Order • Upgrade Nodes Versions • Upgrade Node 10.10 Complicated architecture Work order deployment system can not meet the requirements of resource management. Operator Observe Action Analyze • Observe: watch desired resource and actual resource

placement options, for both control plane and worker nodes. 2 levels of scheduling and resource management are active. ​Currently no automatic scheduling integration occurs, that is, Kubernetes is not not aware of the underlying vSphere topology (sites, affinity groups, NUMA, etc.). ​This session will explain the options to gain better performance, resource optimization and availability through tuning to solve potential issues with CPU and memory intensive workloads ​Kubernetes default resource management ​How it works ​Extending the functionality of Kubernetes ​Using vSphere DRS with Kubernetes

placement options, for both control plane and worker nodes. 2 levels of scheduling and resource management are active. Currently no automatic scheduling integration occurs, that is, Kubernetes is not not aware of the underlying vSphere topology (sites, affinity groups, NUMA, etc.). This session will explain the options to gain better performance, resource optimization and availability through tuning to solve potential issues with CPU and memory intensive workloads Kubernetes default resource management How it works Extending the functionality of Kubernetes Using vSphere DRS with Kubernetes High

关于Node的⼀般信息，如内核版本、Kubernetes版本（kubelet和kube-proxy版本）、Docker版本（如果使⽤了Docker 的话）、OS名称。信息由Kubelet从Node收集。 Management（管理） 与 pods 、 services 不同，Node不是由Kubernetes创建的：它是由Google Compute Engine等云提供商在外部创建 的，或存在于物理机或虚 ⽌，除⾮web-0已经Running and Ready。 Pod Management Policies（Pod管理策略） 在Kubernetes 1.7及更⾼版本中，StatefulSet允许您放松其排序保证，同时通过 .spec.podManagementPolicy 字段保留其 唯⼀性和身份保证。 OrderedReady Pod Management（OrderedReady的Pod管理） OrderedReady OrderedReady Pod管理是StatefulSet的默认值。 它实现了上述 ⾏为。 Parallel Pod Management（并⾏ Pod管理） Parallel Pod管理告诉StatefulSet Controller 并⾏启动或终⽌所有Pod，并且不要等待Pod在启动或终⽌另⼀个Pod之 前变为“Running”和“Ready”或完全终⽌。 Update Strategies（更新策略）

@frapposelli Steve Wong Fabio Rapposelli Presenter Bios 3 Abstract Join the SIG VMware introduction session to learn our mission, recent accomplishments and discuss future work. We will also focus on how Kubernetes project to bring declarative, Kubernetes-style APIs to cluster creation, configuration, and management. It provides optional, additive functionality on top of core Kubernetes. Minikube driver for Fusion

輪，如此循環反覆。kube-proxy 的負載平衡器在 ROUND ROBIN 演算法的基礎上 還支援 Session 持續。如果 Service 在定義中指定 Sesssion 持續，則 kube-proxy 接 收請求時會從本地端記憶體中尋找是否存在來自該請求 IP 的 affinityState 物件，如 果存在該物件，且 Session 沒有超時，則 kube-proxy 將請求轉向該 affinityState Endpoint，並建立一個 affinityState 物件，記錄請求的 IP 和指向的 Endpoint。後面的請求就會綁定到這個建立好的 affinityState 物件上，這就實現了用戶端 IP session 持續的功能。 接下來將深入分析 kube-proxy 的實作細節。kube-proxy 程序為每個 Service 都建立 了一個“服務代理物件＂，服務代理物件是 kube-proxy 程式內部的一種資料結構，

Accessible by users who shouldn’t have access, e.g., CEO ○ Stored in public storage buckets Secret management requirements Identity Require strong identities and least privilege Auditing Verify the use security against penetration. Similarly, poor key management may easily compromise strong algorithms.” NIST SP 800-57, Recommendation for Key Management Keys get old Key rotation ● Key rotation is meant stored cardholder data against disclosure and misuse. 3.6 Fully document and implement all key-management processes and procedures for cryptographic keys used for encryption of cardholder data, including

distributed system Self driving infrastructure Topics ● Cluster management systems ● Today’s problems with operating cluster management systems ● A self-driving approach Motivation: microservices components ○ dynamic dependencies ○ fast deployment iteration ● Solution: automation Cluster management system ● Automation ○ Scheduling ○ Deployment ○ Healing ○ Discovery/load balancing ○ Scaling Kubernetes? ● Operational expertise around app management in k8s extends to k8s itself ○ E.g. scaling ● Bootstrapping simplified ● Simply cluster life cycle management ○ E.g. updates ● Upstream improvements

way to release stateful service Ø Advanced scheduling to improve service stability Ø Quota management to optimize resource orchestration efficiency Ø High performance and comprehensive autoscaling systems like Route System, CMDB, CI, Security Platform, etc. • Declarative application lifecycle management. • Support big data and AI jobs. • Optimize the isolation of resources, and improve resource Service Mesh. • Large-scale and high-performance autoscaling capabilities. • Multi-tenant and quota management. • etc. TKEx Architecture EKS (Elastic Kubernetes Service) TKE (Tencent Kubernetes Engine)