threshold-based alerting & health checks iii. Transform logs/events into metrics to enable statistical analysis iv. Adrian Cockcraft – “Monitoring is so important that our monitoring systems need to be more TESTING i. Random subset of users shown differing versions. User groups define a cohort. Statistical analysis of cohort’s outcomes from differing versions used to identify statistically significant difference behaviors and identify a causal link ii. Ronny Kohavi, Distinguished Engineer & General Manager of the Analysis and Experimentation group at Microsoft: “evaluating well-designed and executed experiments that

Testing standards and tools, including security 4. Deployment pipeline tools 5. Monitoring and analysis tools 6. Tutorials and standards ii. Google – Single repository with over 1B files and over 2B automated tests 1. Static Analysis – inspect program code for all possible run-time behaviors and seek out coding flaws, back doors, and potentially malicious code2. Dynamic Analysis – tests executed while

signature-based continuous scanning using Common Vulnerabilities and Exposures (CVEs), runtime behavior analysis, and container policy enforcement.The security stack in the security sidecar container will include: Retrieval Service 2. Service Mesh 3. Program-specific artifact repository 4. Runtime Behavior Analysis Artificial Intelligence (AI) service 5. DCAR for the hardened containers 6. Common Vulnerabilities

transformation but requires substantial changes in behavior at every level, and Mark’s thorough analysis will prove invaluable to leaders who must execute it.” – Jez Humble, CTO, DevOps Research & Assessment

1. Commit stage – build & package software, runs automated unit tests, and code quality (static analysis, coverage, style, etc.) 2. Package the code only once – use the same packages throughout the pipeline

architectures, demonstrated alignment of systems with business needs, and did the “rigorous” up-front analysis and centralized planning that could then be used to set boundaries for developers when they began

accurately assess probabilities. Traditional teams attempt to drive out uncertainty by planning and analysis. Agile teams tend to drive out uncertainty by developing working software in small increments and