monitoring 1. Create telemetry in application & environments (to include production, pre-production, and CD pipeline) iii. Ian Malpass, Etsy – “If Engineering at Etsy has a religion, it’s the Church of Graphs identify opportunity costs. d. INTEGRATE A/B TESTING INTO OUR RELEASE i. A/B testing requires fast CD to support ii. Use feature toggles to control experiments, cohort creation, etc. iii. Use telemetry

DEPLOYMENT PIPELINE i. CD pipelines represent a new attack surface. If not properly secured, credentials could be compromised or malicious code & changes injected. 1. Harden CI/CD servers and ensure they

Requirements Definition Packages (RDPs) to capture a subset of the IS ICD scope and/or Capability Drop (CD) documents for smaller items such as applications (see Figure 13). Services and requirements oversight As illustrated in Figure 14, each RDP captures a set of requirements prioritized for a release. A CD captures the release backlog requirements allocated to a sprint and forms the basis for a sprint backlog ACAT Acquisition Category BCL Business Capability Lifecycle BPA Blanket Purchase Agreement CD Capability Drop CDD Capability Development Document CONOPS Concept of Operations COTS Commercial

Requirements Definition Packages (RDPs) to capture a subset of the IS ICD scope and/or Capability Drop (CD) documents for smaller items such as applications o Replace comprehensive Preliminary Design Reviews