A Day in the Life of a Data Scientist Conquer Machine Learning Lifecycle on Kubernetes Brian Redmond • Cloud Architect @ Microsoft (18 years) • Azure Global Black Belt Team • Live in Pittsburgh, PA Repeatable/consistent • CI/CD • This has worked well for App Dev. Now time for AI/ML • But, must ensure data scientist are not hindered by structure Why Containers, Kubernetes & Helm? • Container • Contains training instead of sequential: huge time saver for large trainings Kubeflow • Machine Learning Toolkit for Kubernetes • To make ML workflows on Kubernetes simple, portable, and scalable • Training

FailedPostStartHook, Unhealthy… Trace system Increase of SLO Data Collect Audit log Event The unhealthy node Monitoring Isolation Recover Degrade Data Analysis Failures/Machine Failures/Reason Report Lifecycle Daily Report Validation Housekeepi ng High Available Fast Recovery Display Board Alert Analysis Platform Weekly Report SLO： Indicate the cluster is healthy or there is something unexpected happened Storage Analysis Platform Trace Report Weakness The trace system Data Collect: Collect Audit log for the whole cluster. Data analysis: Analyze failure reason if pod is failed. Reason analysis: Analyze

locations with the use of groundbreaking technologies, to enable decision-making at the point of data collection. Fast, insight-driven decision-making in highly dynamic and dangerous conditions is tive edge computing solution, SmartEdge, addresses the increasing need to gather data in real time and perform analysis at the point of collection, supplying imme- diate insight which results in faster battlefields. The an- alytics enabled and performed by Smart- Edge allow battalions to make real-time, data-driven decisions which dramatically improve operational outcomes and in- crease the probability of

数据更改观察者，⽇志分配器，⽇志记录和监视适配器，事件发布者等 代理，桥接器和适配器 控制器，管理器，配置器和更新器 ⼀般来说，单个Pod不会运⾏同⼀应⽤的多个实例。 详情请看The Distributed System ToolKit: Patterns for Composite Containers 考虑替代⽅案 为什么不在单个（Docker）容器中运⾏多个程序？ 1. 透明。 使Pod中的容器对基础设施可⻅，以便 capacity: storage: 10Gi accessModes: - ReadWriteOnce hostPath: path: "/mnt/data" --- kind: PersistentVolumeClaim apiVersion: v1 metadata: name: pv-claim spec: storageClassName: 发布了5个“example.com/foo”资源。 curl --header "Content-Type: application/json-patch+json" \ --request PATCH \ --data '[{"op": "add", "path": "/status/capacity/example.com~1foo", "value": "5"}]' \ http://k8s-master

privileged users, no scans, trust • code analysis • source available? • gotchas: big surface, many languages { } } • sanitizing user input • static code analysis • gotchas: log-leaking} • sensitive password) { • business core data • Personal Identifiable Information (PII) • gotchas: leaks, GDPR (in Europe) { host container dependencies code config user data © 2019, Amazon Web Services

orchestration, it has been expanded to support for data-intensive and stateful apps. Benefit: l Autoscaling in Cloud l Consolidate online service and offline analysis l Ecosystem( Monitor, logging etc) l Fine

(IAST) 動態應用安全測試 (黑箱測試): Dynamic Application Security Test (DAST) 軟體元件分析: Software Component Analysis (SCA) 測試 提交 部署 生產 持續整合 持續交付與部署 預提交 提交 監控 滲透測試 紅隊測試 互動應用安全測試 動態應用安全測試

intensive cryptanalytic attacks ● A cryptoperiod is the time during which a key is used to encrypt data Key rotation: cryptoperiod There are lots of factors that influence the choice of cryptoperiod Strength of cryptographic algorithms used ○ Implementation ○ Operating environment ○ Volume of data ○ Re-keying method ○ Number of key copies ○ Personnel turnover ○ Threat model ○ New and disruptive cardholder data against disclosure and misuse. 3.6 Fully document and implement all key-management processes and procedures for cryptographic keys used for encryption of cardholder data, including

scale • Reduce the learning curve for customer and ourselves • Get consistent user experience and data, leverage with PaaS capability • Facilitate our PaaS and micro-service product Kubernetes Capabilities/Advantages agent to collecting log data ElasticSearch ElasticSearch Monitor/Alert Service CronJob Node Pod Node Pod Unified logging、monitoring、alert with PaaS Consistent data Node group of build nodes configuration and history in MySQL • Logging in central logging service - ElasticSearch • Metric data in monitoring system - prometheus • Alertmanager to invoke various alert and related actions docker

com/developer/article/1627330 Step1: 安装 docker-ce # 安装需要的支撑软件 yum install -y yum-utils device-mapper-persistent-data lvm2 # 添加 yum 源 yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce configmap vi 3-configmap.yaml kind: ConfigMap apiVersion: v1 metadata: name: traefik-config data: traefik.yaml: |- serversTransport: insecureSkipVerify: true api: 的登录密码(登录用户为 admin) kubectl -n rook-ceph get secret rook-ceph-dashboard-password -o jsonpath="{['data']['password']}" | base64 --decode && echo Step4: 配置 storageclass download https://github.c