Solution is Kubernetes: • Highly Scalable • Easy to explore hyper-parameters space • Easy to do distributed training But really, Data Scientists shouldn’t have to care about containers, kubernetes and that stuff • Pachyderm can version datasets and trigger new trainings when changes occur • Distributed File Systems • NFS • HDFS • … Classic DevOps solutions: • Containers • CI/CD • Autoscaling

the zones in your cluster have uneven available resources due to node variations or unevenly distributed pre-existing workloads, this might prevent perfectly even spreading of your pods across zones systemd) 20 Supplement Kubernetes Resource Management with vSphere DRS What is DRS? The vSphere Distributed Resource Scheduler (DRS) is a load balancer for VMs deployed on a hypervisor cluster. It has advanced

the zones in your cluster have uneven available resources due to node variations or unevenly distributed pre-existing workloads, this might prevent perfectly even spreading of your pods across zones systemd) 20 Supplement Kubernetes Resource Management with vSphere DRS What is DRS? The vSphere Distributed Resource Scheduler (DRS) is a load balancer for VMs deployed on a hypervisor cluster. It has advanced

Kubernetes-based Scheduler Framework》 The IEEE ISPA 2018 (16th IEEE International Symposium on Parallel and Distributed Processing with Applications) 能力扩展：GPU支持 资源-访问代价树 四类通信方式分类中，通信开销最大的是SOC，其次是PXB，再次是PHB，PIX通信方式 GPUs in Container Clouds》 The IEEE ISPA 2018 (16th IEEE International Symposium on Parallel and Distributed Processing with Applications) 生态 Next Tencent Kubernetes Engine

are? • Sensitive information • Passwords • OAuth tokens • ssh keys etc. • Stored in etcd • distributed Key-Value data store • How about their security? • Default K8s setup • etcd contents not encrypted

Xiang Li xiang.li@coreos.com | Head of distributed system Self driving infrastructure Topics ● Cluster management systems ● Today’s problems with operating cluster management systems ● A self-driving

ThreadPressure • KubeletProblem • DockerdProblem • ContainerdProblem • KernelDeadlock • … Ø Distributed Ping Test • Self-agent • Self-holder de-scheduler npd self-holder Pod Kernel Kubelet Dockerd

re-use images • well-designed architecture for your container workloads • “How can I build distributed micro-services with container?” Programming Pattern • Sidecar apiVersion: v1 kind: Pod metadata:

deployment strategies and operate our edge devices in a clustered fashion. It really does support distributed processing across devices.” Ben Reif Lead Developer Booz Allen Hamilton 4 www.susergs.com Advancing

JAKARTA EE 2019 开发者报告 Config Server Service Discovery Circuit Breaker Gateway Load Balancer Distributed Tracing Messaging Serverless Task Spring Boot Build anything 快速构建应用 Spring Cloud Coordinate