practice dedicated to the study of building, evolving and operating rapidly-changing resilient systems at scale” (Jez Humble) • Applying Agile practices to operations • Infrastructure as code • Ops Solution is Kubernetes: • Highly Scalable • Easy to explore hyper-parameters space • Easy to do distributed training But really, Data Scientists shouldn’t have to care about containers, kubernetes and stuff • Pachyderm can version datasets and trigger new trainings when changes occur • Distributed File Systems • NFS • HDFS • … Classic DevOps solutions: • Containers • CI/CD • Autoscaling • A/B

xiang.li@coreos.com | Head of distributed system Self driving infrastructure Topics ● Cluster management systems ● Today’s problems with operating cluster management systems ● A self-driving approach

deployment strategies and operate our edge devices in a clustered fashion. It really does support distributed processing across devices.” Ben Reif Lead Developer Booz Allen Hamilton 4 www.susergs.com Advancing responder, and the kit they wear and carry. Our focus is on creating highly sensitive and effective systems that enable teams to do their jobs quicker and more efficiently.” The industry is accustomed to responder, and the kit they wear and carry. Our focus is on creating highly sensitive and effective systems that enable teams to do their jobs quicker and more efficiently.” An array of embedded sensors

Service). Ø Serve Tencent’s business by using kubernetes native approach. • Adapt to various internal systems like Route System, CMDB, CI, Security Platform, etc. • Declarative application lifecycle management ThreadPressure • KubeletProblem • DockerdProblem • ContainerdProblem • KernelDeadlock • … Ø Distributed Ping Test • Self-agent • Self-holder de-scheduler npd self-holder Pod Kernel Kubelet Dockerd

that kind of AD） • What I will not talk? • Kubernetes usage and under the hood • Internal systems or commercial software Kubernetes • The container orchestration and management project created re-use images • well-designed architecture for your container workloads • “How can I build distributed micro-services with container?” Programming Pattern • Sidecar apiVersion: v1 kind: Pod metadata:

Kubernetes满⾜了在⽣产中运⾏的应⽤程序的⼀些常⻅需求，例如： Co-locating helper processes ，促进组合应⽤程序和保留”⼀个应⽤程序的每个容器“模型 Mounting storage systems Distributing secrets Checking application health Replicating application instances Using Horizontal 数据更改观察者，⽇志分配器，⽇志记录和监视适配器，事件发布者等 代理，桥接器和适配器 控制器，管理器，配置器和更新器 ⼀般来说，单个Pod不会运⾏同⼀应⽤的多个实例。 详情请看The Distributed System ToolKit: Patterns for Composite Containers 考虑替代⽅案 为什么不在单个（Docker）容器中运⾏多个程序？ 1. 透明。 使

the zones in your cluster have uneven available resources due to node variations or unevenly distributed pre-existing workloads, this might prevent perfectly even spreading of your pods across zones systemd) 20 Supplement Kubernetes Resource Management with vSphere DRS What is DRS? The vSphere Distributed Resource Scheduler (DRS) is a load balancer for VMs deployed on a hypervisor cluster. It has advanced

the zones in your cluster have uneven available resources due to node variations or unevenly distributed pre-existing workloads, this might prevent perfectly even spreading of your pods across zones systemd) 20 Supplement Kubernetes Resource Management with vSphere DRS What is DRS? The vSphere Distributed Resource Scheduler (DRS) is a load balancer for VMs deployed on a hypervisor cluster. It has advanced

Kubernetes-based Scheduler Framework》 The IEEE ISPA 2018 (16th IEEE International Symposium on Parallel and Distributed Processing with Applications) 能力扩展：GPU支持 资源-访问代价树 四类通信方式分类中，通信开销最大的是SOC，其次是PXB，再次是PHB，PIX通信方式 GPUs in Container Clouds》 The IEEE ISPA 2018 (16th IEEE International Symposium on Parallel and Distributed Processing with Applications) 生态 Next Tencent Kubernetes Engine

are? • Sensitive information • Passwords • OAuth tokens • ssh keys etc. • Stored in etcd • distributed Key-Value data store • How about their security? • Default K8s setup • etcd contents not encrypted