Xiang Li xiang.li@coreos.com | Head of distributed system Self driving infrastructure Topics ● Cluster management systems ● Today’s problems with operating cluster management systems ● A self-driving it wrong, lose the cluster! gcc // gcc source code #include int main() { compile_c(argv[1]); } gcc Self hosting go // golang source code package main import "os" func main() { compile_go(os safely one-by-one ● Upgrade all Scheduler Deployments to v1.4.5 ● Update status to v1.4.5 The infrastructure Workload driven Automation driven Easy to manage: self driving approach (Today’s topic) Security

flow of an app in Go-Jek… Infrastructure as Code Started in the beginning of 2016, we now have Terraform modules, Chef cookbook, and Ansible playbook for most of infrastructure use cases within Go-Jek better abstractions Work with small number of teams at a time • Setting up the whole Go-Viet infrastructure only took four days. Faster Setup Time • Repeatable/immutable nature of containerizing helps maintain any configuration for multi-data center deployments. Easy Configuration • Save 60% infrastructure cost per year per country. Cost-effective Project X was used as deployment tool for Go-Jek

@peytzco @Peytz & Co @Peytz & Co About Peytz & Co ● 5 offices ● +100 employees ● OpenSource ● Infrastructure projects ● Custom built IT solutions ● 15 years’ experience Danish TV Show The task Danish runner Dockerhub Pod Pod Pod -Build -Test -Push to dockerhub -Deploy Application infrastructure The services around Kubernetes Take control Easy to manage and scale Experience KOPS: Installation dashboards Graylogs search and dashboards Performance test Performancetest: LOCUST ● Test as code ● Highly scalable ● Less hardware consuming Connections LOCUST: Fast Forward 3 Cluster 3 Data

Stages: 1. Filter out impossible worker nodes a. Filters are called predicates - extensible in code with a default list 7 Kubenetes scheduling What does the scheduler do: As pod are created, they Stages: 1. Filter out impossible worker nodes a. Filters are called predicates - extensible in code with a default list 2. Rank remaining nodes a. ranking is driven by priorities - this is extensible Stages: 1. Filter out impossible worker nodes a. Filters are called predicates - extensible in code with a default list 2. Rank remaining nodes a. ranking is driven by priorities - this is extensible

rapidly-changing resilient systems at scale” (Jez Humble) • Applying Agile practices to operations • Infrastructure as code • Ops teams embracing source control (git) • Automated testing • Repeatable/consistent • Production accuracy vs expected accuracy when possible • Rolling-updates • … Resources • Source code for this talk: https://github.com/ritazh/kubecon-ml • Kubeflow labs for AKS: https://github.

cluster HA, k8s Job controller and cluster autoscaler, can also use container probe if needed Infrastructure Layer Cloud provider insufficient resource remove / add nodes vCenter openstack Extensibility ConfigMap Job - pod template - volumes user build task • build the docker images init task • prepare code repository - volumes DevOps Operator Manage the Job environment variables image information completes - volumes Storage APIs user build task • build the application package init task • prepare code repository sidecar build task lifecycle - preStop - volumes storage config using secret Query

Functions as a Service • Zero server ops – No provisioning, updating, and managing server infrastructure. – Flexible Scalability • No compute cost when idle Benefits Serverless landscape defined • True, community-driven open source (Apache 2 License) • Proven on IBM Cloud • Exact, same code in open source Architecture of Apache OpenWhisk Deploy Apache OpenWhisk on Kubernetes • The

to a new “out of tree” model, this effort spans all the touching points with the underlying infrastructure: compute, storage, network. This allows cloud providers to have independent feature and patch kube-controller-manager, but they were tightly coupling the kube-controller-manager to cloud- provider specific code. In order to free the Kubernetes project of this dependency, the cloud-controller-manager was introduced

系统的案 例包括 Kratix 和 Humanitec Platform Orchestrator。我们建议平台团队考虑这些工具，作为自己的脚本、本 地工具和基础设施即代码（infrastructure as code，IaC）的独特集合替代方案。我们还注意到，与开放应用模 型（OAM）及其参考编排器 KubeVela 有相似之处，尽管 OAM 声称更加面向应用程序而不是工作负载为中心。 21 评估 Llama 2 是一个来自 Meta 的强大的语言模型，可免费用于研究和商业用途。它既提供原始的预训练模型，也 提供了经过微调的用于对话的 Llama-2-chat 和用于代码补全的 Code Llama。Llama 2 提供了多种尺寸的模型 —— 7B、13B 和 70B，因此如果您想控制自己的数据，Llama 2 是自托管式大型语言模型的一个好选择。 Meta 声称 Llama 分有用。根据我们的实验，我们发现这两个模型都可以使用提示词中的 小样本示例 进行上下文学习。尽管如此， 对于特定的下游任务（例如为 Postgres 等特定数据库生成 SQL），模型仍需要微调。最近，Meta 推出了 Code Llama，一款专用于编程的 Llama 2。使用这些开源模型时务必要小心谨慎。在选择任何这些编码 LLMs 供您的 组织使用之前，请考虑它们的许可，包括代码的许可和用于训练模型的数据集的许可，仔细评估这些方面后再

Applications • Scalable Kubernetes Applications • Scalable Infrastructure for Applications Application Operating System Physical Infrastructure Platform Containers as Enabler Fast Boot Environments Containers Freely Lightweight Minimal Resources Needed Application Operating System Physical Infrastructure Containers and VMs - A Practical Comparison Containers Containers virtualize the operating Allows you to run multiple OS on the same hardware Application Operating System Physical Infrastructure Containers VMware Hypervisor VMs Docker Containers User Cases 9 •Ready-to-go development