below: • github.com/istio/istio – 7353c84b560fd469123611476314e4aee553611d • github.com/istio/proxy – c51fe751a17441b5ab3f5487c37e129e44eec823 • github.com/istio/istio.io – 26dacdde40968a37ba9eaa864d40e45051ec5448 exposes unnecessary ser- vices and is accessible to anyone within a default cluster. • The Envoy Proxy admin port is exposed via the Istio sidecar and would allow a malicious workload to override or compromise master branch up to July 15th, 2020. Commit: 7353c84b560fd469123611476314e4aee553611d istio/proxy Istio Envoy Proxy code in the master branch up to July 15th, 2020. Commit: c51fe751a17441b5ab3f5487c37e129e44eec823

connection • HAProxy transparent mode, two connections  L4 • Add IP in TCP Protocol options • Proxy Protocol  L7 • HTTP header “x-forwarded-for” • User Protocol #IstioCon LVS ① user send traffic module in kernel #IstioCon Proxy Protocol  Proxy Protocol v1 PROXY Protocol prepends every connection with a header reporting the client IP address and port. A PROXY Protocol plain-text header has the format: PROXY TCP4 192.0.2.0 192.0.2.255 42300 443\r\n  Proxy Protocol v2 #IstioCon Proxy Protocol client Server Establish TCP connection Proxy Protocol binary header Application data

● Circuit breaking Istio at Mercari Stabilizing Istio 10 Stabilizing Istio ● Istio sidecar proxy specifications ● Kubernetes shortcomings with sidecar containers ○ Controlling containers lifecycle Istio? ● A full mesh is utopian, know what you need only ● Guardrails for Istio 11 Istio sidecar proxy specifications Stabilizing Istio Pod App container Sidecar container All incoming traffic must Envoy is started before any other container in a pod ● Use a `postStart` lifecycle hook in the istio-proxy container manifest lifecycle: postStart: exec: command: - pilot-agent

Pod2 业务容器 业务容器 Istio-proxy容器 Istio-proxy容器 Istio-init 容器 Istio-init 容器 Pod内共享网络 Pod内共享网络 Virtual inbound -15006 kubelet 拦截指定命名空间 Pod创建请求 xDS Iptables 规则 ./etc/istio/proxy/SDS 证书 获取 配置 文件 可以修改全局注入参数 可以修改全局注入参数 作用于所有目标空间的 pod 证书更新 Envoy启动流程 Envoy控制面流量 Envoy数据面流量 ./etc/istio/proxy/XDS SDS xDS CSR Prometheus configmaps Copyright © Huawei Technologies Co., Ltd. All rights reserved. Page 7 Envoy原理及总体架构-说明 带有Envoy两个容器的创建POD请求，修改过的创建请求被 kubelet接收，并在节点创建POD。 • istio-init容器添加用于配置容器网络内iptables规则 • istio-proxy容器启动pilot-agent进程，使用UID=1337 GID=1337创建Envoy启动命令行与配置文件 • 可以通过自定义deployment内istio注解sidecar.istio.io/inject:

Point of Failure Service Mesh API Gateway Book Info Payments Product Info Proxy Proxy Proxy Proxy + k8s Istio mTLS mTLS mTLS ✓ Security ✓ Visibility ✓ Traffic Shaping for service owners Demo Admiral API Gateway Payments Product Info Proxy Proxy Proxy Book Order Proxy + k8s Istio mTLS mTLS mTLS + k8s + k8s Multi-region Endpoint ○ Istio config integrated with gitops deployment ○ Init modifications to prevent proxy startup race conditions Thank You Admiral Istio Ecosystem Project https://github.com/istio-ecosystem/admiral

Istio Community, 2020-Present Anthos Service Mesh, Google Inc, 2020 Envoy is an edge and service proxy that allows traffic in an infrastructure to flow in a mesh, allowing you to visualize problem areas Condition issues in istio CNI during Node bootstrap Community Solutions to istio CNI CNI Basics Kube Proxy: exists in each node and manage iptable IPTables: Responsible for translating service IP addresses init container in workload Istiod watch updates & start networking sidecar proxy init container update iptable rule for proxy terminate init container Start workload with updated ip routing rules Networking

Workload security Operation security Mesh security Edge Security Cluster security Service Proxy Ingress 1. Define ingress security policies to control accesses to services. Deploy web application external access. Edge security best practices Cluster security Access control Service Proxy Ingress Token exchange 1. Istio authentication and authorization policies for every automatically validate policy exceptions are as expected. Gatekeeper Service 1 Proxy Service 2 Proxy Namespace foo Istio authn & authz policies Namespace bar 2. Enforce k8s RBAC policies:

tp.wasm/envo y.wasm.metad ata_exchange Istio_authn kubectl exec -it [productpage-xxx] -c istio-proxy curl localhost:15000/config_dump envoy.filters .http.cors envoy.filters .http.fault envoy.filters 简化了扩展Envoy的过程, 这种解决方案通常使用WebAssembly（WASM）的新技术， 它是一种有效的可移植二进制指令格式，提供了可嵌入和隔离的执行环境。 6 使用Wasm for Proxy ● Pros ○ 敏捷性：过滤器可以动态加载到正在运行的Envoy进程中，而无需停止或重新编译。 ○ 可维护性：不必更改Envoy自身基础代码库即可扩展其功能。 ○ 多样性：可以将流行的编程 ○ 由于需要启动一个或多个WASM虚拟机，因此会消耗一定的内存使用量; ○ The WebAssembly ecosystem is still young; 7 Wasm in Envoy Proxy ● Wasm动态加载 ● 一致性校验: ○ https://github.com/pr oxy-wasm/spec ● 内置的Wasm runtime ○ ~20MB for WAVM

Gateway Proxy Low to high Traffic flowing from Ingress Sidecar or Ingress Gateway to a Proxy might be required to pass further security policies. Proxy Service Low to high Incoming traffic to proxy can be specified policies before it reaches the service. The traffic crosses a trust boundary as it passes the proxy. Controlplane Dataplane High to low Policies are created by users with privileges. The policies are Policy Enforcement Points (PEPs) ● A set of Envoy proxy extensions to manage telemetry and auditing Certificate management Alongside each Envoy proxy, an instance of the Istio agent is located and communicates

services | CONFIDENTIAL 10 Capture API interactions with lua filters Service A Proxy Proxy Service B Service C Proxy Mesh Dynamics Data Store Deploy: kubectl apply -f Capture using Lua filter API data + TraceIDs | CONFIDENTIAL 11 Assemble API request traces Service A Proxy Proxy Service B Service C Proxy req req[A B], trace:r, span:s1 res[A B], trace:r, span:s1 req[B C], trace: