Istio Security Assessment Google August 6, 2020 – Version 1.1 Prepared for Arun Kumar R Prepared by Mark Manning Jeff Dileo Divya Natesan Andy Olsen Feedback on this project? https://my.nccgroup Synopsis In the summer of 2020, Google enlisted NCC Group to perform an assessment on the open-source version of Istio and all of its components. Istio is a modern service mesh technology stack often used within certificates, provides workload identity, and includes a builtin authorization system facilitated by its control plane. The goal of the assessment was to identify security issues related to the Istio code base

Shortcoming 1: Controlling the running order for containers Stabilizing Istio Kubernetes lacks good control APIs to customize the containers lifecycle in a pod. There is no official way to instruct a pod said 32 A full mesh is utopian, know what you need only Stabilizing Istio The reality: ● The control plane is burning down when pushing your thousand services updates to the hundreds of proxies running CRD to save the mesh Stabilizing Istio The Sidecar CRD (Custom Resource Definition) allows to control the exposure of mesh configuration to a specific proxy, based on namespace or labels. apiVersion:

offers more advanced features to support A/B testing, canary deployments, rate limiting, access control, encryption and end-to-end authentication. Istio itself is implemented in Go which shields the project consist of Envoy proxies and an Istio-agent and manage network traffic between microservices. The control plane is responsible for applying user configuration to the proxies. The following diagram demonstrates allows users to create authorization policies to specify mesh-, namespace-, and workload-wide access control for workloads in the mesh. Authorization policies are created by users and are enforced at runtime

Different namespace • Project runs as tenant, need control rights Solution cluster connect core cluster with Istio multi-cluster - Replicated control planes Some standalone cluster without Istio can #IstioCon Secure Platform – Authorization Policy Using Authorization Policy enables access control on workloads in the mesh. For request from ingressgateway, need verify token For request from same Platform – Extra Authorization Version 1 : Istio Mixer authz adapt Implement role-based authorization – whether this user can access this api based on its role => Version 2: Envoyfilter ext_authz #IstioCon

application traffic end to end in production • Allow platform to use Istio authorization policy to control the access to each Knative service based on Istio service roles. How Istio is leveraged in a Knative Info K8s Cluster Capacity 12 nodes in 3 zones, 16 vCPU * 64 Gi MEM Knative Version Knative 0.16, 0.17, 0.18 Istio Version 1.5, 1.6, 1.7 Istio scalability optimization during Knative Service provisioning overload issue still exits 800 Knative Services #IstioCon o 1400 total with dev release with flow control fix looks great, ingress_ready p100 < 30s o [Istio 1.9.x] Support for backpressure on XDS pushes

with NIST ● Author SP 800-204 series on microservice security ● R&D on Next Generation Access Control (NGAC) ● Exclusively co-host annual zero trust multi-cloud conference Best in Class Team ● Creators operating an application Why is Istio? TSB: The Application-Aware Networking Platform Istio: Control Plane Tetrate Service Bridge: Management Plane Envoy: Data Plane Workload (Service) POD Workload availability & resiliency enabling active-active deployments ● Cross cluster security policies & access control ● Unified telemetry and availability reporting ● Service discovery across multiple clusters ●

#IstioCon V0.2 Mesh Expansion ● Prerequisites ○ IP connectivity to the endpoints in the mesh ○ Istio control plane services (Pilot, Mixer, CA) accessible from the VMs ○ (optional) Kubernetes DNS server accessible http req to 172.16.1.3 GET /status/200 #IstioCon V1.8 Smart DNS Proxy: A Step Further ● Taking control of DNS! ○ VMs to Kubernetes integration ○ Reduced load on your DNS servers w/ faster resolution Networks #IstioCon Current State of VM Support ● Traffic flow ○ VM connects up to the Istio control plane through a Gateway ○ WorkloadEntry created ■ VM sidecar is made aware of all services in the

#IstioCon helm ● values-overrides.yaml Install using helm install istiod manifests/charts/istio-control/istio-discovery \ -n istio-system --values values-overrides.yaml meshConfig: defaultConfig: inlineBytes' | \ sed 's/"//g' | base64 --decode | openssl x509 -noout -text Certificate: Data: Version: 3 (0x2) Serial Number: … Signature Algorithm: sha256WithRSAEncryption

Hierarchy of control planes ● Global Control Plane ○ Users provide application specs to Global Control-Plane ○ Syncs specs to AZ control-planes ○ Hosts global services - Global IPAM, Access-control Policy Policy store, etc. ● AZ Control Plane ○ Syncs specs to workload K8s clusters in the AZ ○ Shared-Nothing Architecture ■ Hosts services catering to the AZ, e.g., AZ IPAM, Network Load-balancers, etc K8s Cluster K8s Cluster K8s Cluster K8s Cluster AZ Control Plane AZ Control Plane AZ Control Plane Global Control Plane Region Rn Delegate #IstioCon Load balancing & Traffic

Exfiltration Man-In-The-Middle Denial of Service Privilege Escalation Application Compromise Control Plane Service mesh security architecture Cluster Workload Edge Operations Ingress Policies RBAC Audit Logging Image Verification Admission Control Workload Identity K8s RBAC K8s CNI AuthZ Policy Peer AuthN Policy KMS Control Plane Hardening Istio Security Releases Complete Security Cluster security Service Proxy Ingress 1. Define ingress security policies to control accesses to services. Deploy web application firewall to defend against DDoS, injection, remote