Huawei(Now) - Cloud Native batch system (Volcano) development • IBM spectrum computing - Cluster resource and workload scheduling platform development l Gaps for Spark • Agenda l Why Spark on Kubernetes fair-share, job fair-share to share resource l Use task-topology to improve the spark workload efficiency. Summary p Queue priority p Queue reclaim p Queue plugin p Hierarchical queue p Dynamic resource

rs • secrets • services • loadbalancers 17 Kubernetes Default Resource Management Goals Efficiency Fairness Quotas Prioritization Isolation 18 Kubernetes built-in resource management Enforcement resource reservations (not just shares) • with governed sharing of unutilized capacity for improved efficiency • Allows maintenance with less service level disruption VM DRS Cluster VM VM VM VM VM VM VM

rs • secrets • services • loadbalancers 17 Kubernetes Default Resource Management Goals Efficiency Fairness Quotas Prioritization Isolation 18 Kubernetes built-in resource management Enforcement resource reservations (not just shares) • with governed sharing of unutilized capacity for improved efficiency • Allows maintenance with less service level disruption VM DRS Cluster VM VM VM VM VM VM VM

resources like CPU, memory, etc. are more effectively utilized in container world than in VMs. Efficiency • Automatic service discovery allows engineers to not maintain any configuration for multi-data

ECI ECI ECI ECI • Bulti-in Security • Light-Weight Virtualization • High Density • Scale and efficiency • Optimization Alinux2 for Sandbox Storage Network Computing Alibaba Cloud Sandbox Engine

scheduling to improve service stability Ø Quota management to optimize resource orchestration efficiency Ø High performance and comprehensive autoscaling What is TKEx Ø Based on TKE (Tencent Kubernetes

/年 237万/年 节约成本 COST 290人日/月 测试成本降低 运维效率提升6倍 建立CI/CD流程，将开 发、测试、运维有机串 联，SDK版本升级由30分 钟缩至5分钟 Efficiency 6 倍 提升 30 时间缩短 弹性伸缩应高峰 容器配置与数量根据业 务量秒级扩缩容，轻松 应对热点业务高峰浪涌 应业务量弹性扩缩容 占用资源减半 该项目实施后，节约一次性

running the container Java Development on Kubernetes Skaffold + Jib Continuous development for Kubernetes Skaffold is a command line tool that facilitates continuous development for Kubernetes applications com/GoogleContainerTools/skaffold official website code Development Process application k8s config build push deploy connect update code Development Process application k8s config skaffold github

About us Thomas Hector Head of IT Operations Key Qualifications ● 12+ years’ experience with development and operations ● Automating everything ● Been handling Kubernetes clusters in production for for 2 years Jan-Erik Revsbech CTO Key Qualifications ● 20+ year of experience in web-development & operation/hosting ● OpenSource advocate ● Extensive hand-on experience with Kubernetes and Containerization and orchestration Containerized development and production environment Cloud orchestration with Kubernetes Hosted on AWS Deployment, CI and CD Local development Gitlab runner

Hypervisor VMs Docker Containers User Cases 9 •Ready-to-go development •Self-service portal Developer Sandbox • New application development • 12-factor apps, PCF Cloud Native •Simplify app maintenance