Tetrate’s product build on top of the upstream Istio ● Why not Istio OSS? ● Problems unsolved ○ Multi-cluster and VM (lower onboarding cost) ○ Enterprise team structure gap (Workspace, Tenants, etc) ○ UI&UX controls, across clusters ● High availability & resiliency enabling active-active deployments ● Cross cluster security policies & access control ● Unified telemetry and availability reporting ● Service discovery agility More About Multi Cluster ● Multi tenancy ● Resource hierarchy ● NGAC Two-tier Gateway ● Tier-1 Gateways sit at the application edge and are used in multi-cluster environments to route traffic

Technologies Co., Ltd. All rights reserved. Page 9 Envoy启动配置及xDS listener router upstream pool Envoy cluster istiod pilot-agent LDS RDS CDS EDS tls证书 管理 SDS CSR创建证书 stat tracing 支持采集或 主动上报 监控系统 过滤器 service.route.v3.RouteDiscoveryService/StreamRoutes CDS 上游cluster配置 POST /envoy.service.cluster.v3.ClusterDiscoveryService/StreamClusters EDS 上游cluster endpoint配置 POST /envoy.service.endpoint.v3.EndpointD • 启动admin RESTful监听，处理运行状态输出，prometheus收集等请求 • 定期将工作线程内监控数据stat进行合并 • 定期刷新DNS信息，加速域名解析。 • 目标cluster内主机列表健康状态判断。 • worker线程： • 通过启动配置参数concurrency指定，不支持动态调整。 • 启动virtualoutbound/virtualinbound网络

focus testing efforts. Istio does not currently have a reference design for what an ideal Kubernetes cluster with Istio running within it. Instead, NCC Group used various hosting options (i.e. Minikube, GKE controlPlaneSecuri ty configuration directive or other means. This left all default services exposed within the cluster. • The default istio profile that is labeled for produc- tion lacks many hardening controls and Pilot admin interface exposes unnecessary ser- vices and is accessible to anyone within a default cluster. • The Envoy Proxy admin port is exposed via the Istio sidecar and would allow a malicious workload

domain socket)，可以被 下游客户端连接。在 Envoy 中,Listener 可以绑定到端口上直接对外服务，也可以不绑 定到端口上，而是接收其他 listener 转发的请求。 • Cluster：集群是指 Envoy 连接的一组上游主机，集群中的主机是对等的，对外提供相 同的服务，组成了一个可以提供负载均衡和高可用的服务集群。Envoy 通过负载均衡 策略决定将请求路由到哪个集群成员。 协议的主要概念： • Listener Discovery Service (LDS) : 监听器发现服务。 • Route Discovery Service(RDS) : 路由发现服务。 • Cluster Discovery Service (CDS)： 集群发现服务。 • Endpoint Discovery Service (EDS) ：集群中的服务实例发现服务。 • Secret Discovery – 数据面 – Istio 中的 Envoy Sidecar 配置 Istio中的 Envoy Sidecar 配置： • Istio 通过 Listener、Route Config 和 Cluster 为 Mesh 中的 Envoy 生成了入向和出向两个不同方向的处理流程的配 置。 • 在 Envoy 的基础上增加了 VirtualInboundListener，VirtualOutboun

io Didier Grelin Sr. Technical Program Manager dgrelin@google.com Ethan Jackson Staff Engineer jethan@google.com Francis Zhou Senior Technical Program Manager francisz@google.com Greg Hanson So�ware jdpettit@google.com Lei Tang Technical Lead leitang@google.com Neelima Balakrishnan So�ware Engineering Manager neelimabk@google.com Shankar Ganesan So�ware Engineer shankgan@google.com OSTIF 4 Istio Security boundaries We identify the following trust boundaries: From Into Trust flow Description Outside of cluster Ingress Sidecar or Ingress Gateway Low to high Ingress traffic can have the lowest level of privilege

envoy/haproxy/nginx which have already supported proxy protocol #IstioCon Istio Traffic Flow – inner cluster svcA svcB envoy envoy Pod1：10.244.0.20 Pod2：10.244.0.25 Dest: 127.0.0.1 Src：127.0.0.1 #IstioCon configuration use_remote_address: Envoy will only append to XFF if the use_remote_address HTTP connection manager option is set to true and the skip_xff_append is set false. xff_num_trusted_hops : If use_remote_address set “envoy.filters.listener.proxy_protocol". ② enable Proxy Protocol Transport Socket in upstream cluster. #IstioCon Preserve TCP Original Src Addr - ingress - svcB ① Set “envoy.filters.listener.proxy_protocol“

America 1.5% from Oceania Participant demographics 20.4% of attendees were CxO / Engineering manager / Tech Lead 43.8% of attendees were either evaluating Istio for production use, or have tried hours On the following topics: ● Istio debugging, ● Istio Security, ● WebAssembly, ● Multi Cluster, ● Istio Roadmap and ● Istio in production. Participant feedback The majority of participants (Tetrate) Member Zhonghu Xu (Huawei) The team (3/3) Event Production (Software Guru) Event Manager Mara Ruvalcaba Content Coordination Pedro Galván Streaming and website Alberto Rodríguez Streaming

Deployment: Cloud Layout ● Multiple K8s Clusters in an AZ ○ Each K8s cluster ~ 200 - 5,000 nodes ○ Upto 100,000 Pods in a cluster ○ 10,000+ K8s services - including prod, pre-prod, staging, etc. scenario Region R1 AZ 1 AZ 2 AZ n Data Center DC1 K8s Cluster K8s Cluster K8s Cluster K8s Cluster K8s Cluster K8s Cluster Region Rn #IstioCon Application Specs Region R1 Application failures to AZ boundary AZ 1 AZ 2 AZ n Data Center DC1 K8s Cluster K8s Cluster K8s Cluster K8s Cluster K8s Cluster K8s Cluster AZ Control Plane AZ Control Plane AZ Control Plane

and Surfaces Istio is both a collection of security controls and an attack target. Workload Cluster Edge Operations Workload Data Exfiltration Man-In-The-Middle Denial of Service Privilege Escalation Escalation Application Compromise Control Plane Service mesh security architecture Cluster Workload Edge Operations Ingress Policies Egress Policies WAF / IDS Firewall User AuthN/Z Data Completeness Service mesh security best practices 2 Cluster security Edge security Workload security Operation security Mesh security Edge Security Cluster security Service Proxy Ingress 1

Internal Load Balancers (ILBs) for Kube DNS, Pilot, Mixer and CA ○ Generate configs for VMs, incl. `cluster.env`, DNS config, Istio authN secrets etc. ○ Setup dnsmasq, Istio components in the VM and verify 1. Dnsmasq accepts DNS queries 2. Access the built-in Kube DNS (exposed by ILB) 3. Obtain the Cluster IP resolved 4. Traffic intercepted by the sidecar proxy 5. xDS ■ Traffic forwarded to ingress DNS query httpbin.ns1.svc.cluster.local 2. DNS response – 10.4.4.4 http req to 10.4.4.4 GET /status/200 http req to 172.16.1.3 GET /status/200 httpbin.ns1.svc.cluster.local SVC IP: 10.4.4.4 #IstioCon