control. Focused on version control and automated environment creation – time reduced from 8 weeks to 1 day b. ENABLE ON-DEMAND CREATION OF DEV, TEST, AND PRODUCTION ENVRIONMENTS i. Major contributing everything is checked into version control 1. Application code & dependencies 2. Environment scripts & creation tools 3. DB scripts and reference data 4. Containers 5. Automated tests 6. Project artifacts rather than long-lived feature branches f. BUILD A FAST AND RELIABLE AUTOMATED VALIDATION TEST SUITE i. Slow and periodic feedback kills – larger batch sizes compound effort needed to identify causes

BEL 33 21 ! 59 3B ; 85 55 U 111 6F o 8 08 BS 34 22 " 60 3C < 86 56 V 112 70 p 9 09 HT 35 23 # 61 3D = 87 57 W 113 71 q 10 0A LF 36 24 $ 62 3E > 88 58 X 114 72 r 11 0B VT 37 25 % 63 3F ? 89 59 Y 115

PRACTICE i. Ensure automated tests demonstrate use and behavior of libraries and components ii. Test suite becomes the living documentation of the system specification and represent working examples of API

without requiring changes to the whole ball of EA.  It has a robust, automated regression test suite, so that new development does not cause expensive break-fix activity.  It has good monitoring tools

deflection 1. Prevents creation of institutional knowledge about incidents and learning needed to prevent them in the future ii. Use for fact-based problem-solving e. ENABLE CREATION OF PRODUCTION METRICS A/B testing requires fast CD to support ii. Use feature toggles to control experiments, cohort creation, etc. iii. Use telemetry to measure outcomes iv. Etsy open-sourced their experimentation framework

lead times measured in minutes, the constraint usually follows this progression: a. Environment creation: b. Code deployment: c. Test setup and run: d. Overly tight architecture: iv. ELIMINATE HARDSHIPS