GPU Resource Management On JDOS 梁永清 liangyongqing1@jd.com 提供的服务 1. 用于实验的 GPU 容器 2.基于 Kubeflow 的机器学习训练服务 3.模型管理和模型 Serving 服务 Experiment Training Serving 均基于容器，不对业务方直接提供 GPU 物理机 GPU 实验 JDOS 常规的容器服务

Node Operator: Kubernetes Node Management Made Simple 陈俊(Joe), Ant Financial Agenda • Background and Motivation • Introduction of Operators • Node-Operator • Advanced Topic: • Upgrade Master & Node Components reliably • Canary Rollout • Master & Node Component Versions Management Motivation: Work Order Deployment Worker Order • Upgrade Nodes Versions • Upgrade Node 10.10 Complicated architecture Work order deployment system can not meet the requirements of resource management. Operator Observe Action Analyze • Observe: watch desired resource and actual resource

NIST在容器安全指南中揭露了五種容器應用最應關注的風險 映像風險 Image Risk 登錄風險 Registry Risk 容器調度平台風險 Orchestrator Risk 容器風險 Container Risk 實體作業系統風險 Host OS Risk ©2019 VMware, Inc. 9 針對Kubernetes的安全強化實作參考: Day 1 & Day 2 for k8s clusters Manages access to k8s API for developers IT Operator IaaS Management Internet User Application User Trust Boundary Trust Boundary Trust Boundary Trust Boundary Authentication and Authorization i. Compliance j. File System Permissions k. User Account Management 所有強化在發佈前都經過測試驗證 您不再需要每回合升級都從頭來過 若發現CVE漏洞官方立刻提供修補 •The following servers are not used

the cluster SLIs on Large k8s Cluster 1. Cluster health state A combination value indicates the risk in the cluster. Currently Healthy, Warning and Fatal are the possible value. 2. Success rate A

Accessible by users who shouldn’t have access, e.g., CEO ○ Stored in public storage buckets Secret management requirements Identity Require strong identities and least privilege Auditing Verify the use security against penetration. Similarly, poor key management may easily compromise strong algorithms.” NIST SP 800-57, Recommendation for Key Management Keys get old Key rotation ● Key rotation is meant stored cardholder data against disclosure and misuse. 3.6 Fully document and implement all key-management processes and procedures for cryptographic keys used for encryption of cardholder data, including

placement options, for both control plane and worker nodes. 2 levels of scheduling and resource management are active. ​Currently no automatic scheduling integration occurs, that is, Kubernetes is not to solve potential issues with CPU and memory intensive workloads ​Kubernetes default resource management ​How it works ​Extending the functionality of Kubernetes ​Using vSphere DRS with Kubernetes pre-container era Active discussions regarding Kubernetes enhancements going on now in Resource Management Working Group – please join in • See Issue #49964 14 Using a NUMA aware hypervisor to solve

placement options, for both control plane and worker nodes. 2 levels of scheduling and resource management are active. Currently no automatic scheduling integration occurs, that is, Kubernetes is not to solve potential issues with CPU and memory intensive workloads Kubernetes default resource management How it works Extending the functionality of Kubernetes Using vSphere DRS with Kubernetes High pre-container era Active discussions regarding Kubernetes enhancements going on now in Resource Management Working Group – please join in • See Issue #49964 14 Using a NUMA aware hypervisor to solve

distributed system Self driving infrastructure Topics ● Cluster management systems ● Today’s problems with operating cluster management systems ● A self-driving approach Motivation: microservices components ○ dynamic dependencies ○ fast deployment iteration ● Solution: automation Cluster management system ● Automation ○ Scheduling ○ Deployment ○ Healing ○ Discovery/load balancing ○ Scaling Kubernetes? ● Operational expertise around app management in k8s extends to k8s itself ○ E.g. scaling ● Bootstrapping simplified ● Simply cluster life cycle management ○ E.g. updates ● Upstream improvements

way to release stateful service Ø Advanced scheduling to improve service stability Ø Quota management to optimize resource orchestration efficiency Ø High performance and comprehensive autoscaling systems like Route System, CMDB, CI, Security Platform, etc. • Declarative application lifecycle management. • Support big data and AI jobs. • Optimize the isolation of resources, and improve resource Service Mesh. • Large-scale and high-performance autoscaling capabilities. • Multi-tenant and quota management. • etc. TKEx Architecture EKS (Elastic Kubernetes Service) TKE (Tencent Kubernetes Engine)

Very manual, no fault tolerance, hard to scale, etc • Scheduling, provisioning, and resource management of multiple containers – Docker, Mesos à Kubernetes Support – AWS, Azure, Google à Kubernetes ContainerImage2 Replicas: 2 Kubernetes 101 at the Highest Level • Container Cluster = “Desired State Management” – Kubernetes Cluster Services (w/API) • Node = Container Host w/agent called “Kubelet” • Application Switch Namespace ‘foo’ PODs – Logical Switch Namespace ‘demo’ PODs – Logical Switch Cluster Management Nodes – Logical Switch Master ‘VM’ etcd NCP API Srv Worker ‘VM’ Pod 1 Pod 2 Worker ‘VM’