Istio End of 2021 100% services migrated to Istio 8 Features currently used: ● HTTP/2 Load-balancing ● Traffic Shifting ● mTLS Features under investigation: ● Retries ● Circuit breaking Istio ● Moving HTTP/2 load-balancing from client-side to Envoy ● Label selector updates for app and version labels ● Istio default retry policy ● Istio proxy performance and load testing ● Abstracting Istio features 44 Moving HTTP/2 load-balancing from client-side to Envoy Adopting Istio ● We use gRPC heavily in our microservices ● But Kubernetes is pretty bad at load-balancing it ● So we solved it

AZ ○ Shared-Nothing Architecture ■ Hosts services catering to the AZ, e.g., AZ IPAM, Network Load-balancers, etc. ■ Full isolation by confining service failures to AZ boundary AZ 1 AZ 2 AZ n Control Plane Global Control Plane Region Rn Delegate #IstioCon Load balancing & Traffic Flow ● Two tiers of hardware Load-Balancers (LB) ● Application-Tier LB ○ K8s service realized on Application-Tier DNS lookup Application-Tier Load-Balancer Web-Tier Load-Balancer Application-Tier Load-Balancer Web-Tier Load-Balancer Application-Tier Load-Balancer Web-Tier Load-Balancer Pods Pods Pods AZ

POD POD S E R V I C E (Load balancer) www.my-application.com External Traffic 75% 25% Deployment Canary Releases Using Kubernetes Deployment POD POD POD S E R V I C E (Load balancer) www.my-application Traffic POD 50% 50% Deployment Canary Releases Using Kubernetes Deployment POD S E R V I C E (Load balancer) www.my-application.com External Traffic POD POD 0% 100% Deployment Deployment Deployment Canary Releases Using Kubernetes – Across application Layers Deployment POD POD S E R V I C E (Load balancer) www.my-application.com External Traffic 75% 25% POD POD POD POD S E R V I C E (ClusterIP)

Need consistent policy enforcement ○ Need consistent metrics aggregation ● Traffic management ○ Load balancing for VMs, failover, A/B testing, modern rollouts for VM services ● Security ○ Enforce the VMs ○ (optional) Kubernetes DNS server accessible from the VMs ● Onboard steps ○ Setup Internal Load Balancers (ILBs) for Kube DNS, Pilot, Mixer and CA ○ Generate configs for VMs, incl. `cluster.env` Smart DNS Proxy: A Step Further ● Taking control of DNS! ○ VMs to Kubernetes integration ○ Reduced load on your DNS servers w/ faster resolution ○ Automatic VIP allocation where possible ○ Multicluster

Expect From a Service Mesh? 为了将基础设施的运维管理从应用代码中剥离，我们需要七层的流量管 理能力： ● Routing based on layer-7 header ○ Load balancing at requet level ○ HTTP host/header/url/method, ○ Thrift service name/method name ○ Dubbo VirtualService API ● Generate LDS/RDS for Envoy Filter AwesomeRPC Filter ● Decoding/Encoding ● Routing ● Load balancing ● Circuit breaker ● Fault injection ● Stats ● ... Pros: ● It’s relatively easy to add deployment 中通过环境变量设置其所属地域 3. 在 consumer 的 deployment 中通过 label 声明其所处的 region 和 zone 4. 通过 dr 规则启用 locality load balancing https://docs.qq.com/doc/DVnlqUVB1ek1laFBQ #IstioCon What’s next？ 现阶段协议扩展方案面临的挑战： ●

Proxy • 集群外部流量入口：Ingress Gateway • 集群外部流量出口：Egress Gateway（可选,在一个集中点对外部访问进行控制） • Service discovery • Load balancing • Time out • Retries • Circuit breaker • Routing • Auth • Telemetry collecting 外部流量出口 外部流量入口 } ] } ], } Envoy Filter AwesomeRPC Filter • Decoding/encoding • Parsing header • Routing • Load balancing • Circuit breaker • Fault injection • Telemetry collecting Reviews v1 Reviews v2 AwesomRPC } ] } ], } Envoy Filter AwesomeRPC Filter • Decoding/decoding • Parsing header • Routing • Load balancer • Circuit breaker • Fault injection • Telemetry collecting Pilot 将通用协议路由规则解析为统一格式 的 xDS 配置下发。

#IstioCon Motivation ● Reliability of central proxy layer (HAProxy/Envoy) ● More control over load balancing ● Offload application services from networking and configuration ● Avoid other sources Immutable deployments ● Minimal blast radius ● Discover Pods for controlled and predictable routing/load balancing ● Improve performance and resilience ● Stricter zonal routing ● Capability for service

a Good Fit? What is an API Gateway? What is a Service Mesh? Common Features Common Features ● Load Balancing ● Request Routing ● Service Discovery ● JWT Authentication ● Traffic Splitting ● Canary

• (istío) n (plural ιστία) 1. sail What about the rest of the boat? Upcoming Talks: Aperture - Load Management Meshery - WASM plugin management Argo - Multi-cluster orchestration JP Morgan SLO Generation

2016 ● GoPay had services running on VM and decided to using Envoy XDS and Consul for migration & load balancing the traffic across container and VM. ● Over time, managing Envoy and Consul became a burden