operations, technology, and budgets  Actively involving users throughout development to ensure high operational value Agile practices integrate planning, design, development, and testing into an iterative operations, technology, and budgets  Actively involving users throughout development to ensure high operational value Agile is built around a culture of small, dynamic, empowered teams actively collaborating requirements or features up front, as the program backlog will evolve over time. Subsequent strategic, high-level planning sessions focus on the release and sprint levels. They outline the intent of a release

organization.” They are self-healing ii. Netflix – April 21, 2011; entire Amazon AWS US-EAST availability zone went down; Netflix was unaffected. Since 2008 they were building and testing with the Chaos each week with expectation that everyone is to teach or learn something in that timeframe. Place high value on mentoring ii. Encourage life-long learning. Our field is only progressing at faster rates to simultaneously achieve Information Security goals and create high degree of assurance for confidentiality, integrity, and availability b. Don’t inspect security in at the end, it’s integrated as part

accidental, not the essential, difficulties.”Solutions to Accidental Difficulties • High-level languages • The most a high-level language can do is to furnish all the constructs the programmer imagines essence, or are they remainders of our accidental difficulties?”Silver bullets? • Ada and other high-level language advances • Object-oriented programming • Artificial intelligence • Expert systems • servers, but I need more servers so I need… • an Infrastructure as a Service platform, but for high availability I need… • another rack of servers, but to pay for it I need…Always ask yourself • Am I solving

time-sucking configuration changes or complicated processes.” iv. 2015 State of DevOps Report – high performers had MTTR 168x faster than low performers b. CREATE OUR CENTRALIZED TELEMETRY INFRASTRUCTURE threats 6. Resources (RAM, disk, CPU, bandwidth, and others with hard or soft limits) 7. Health & availability 8. Startups & shutdowns 9. Faults & errors 10. Circuit breaker trips 11. Delays 12. Backup “people closest to a problem typically know the most about it.” iii. 2014 State of DevOps Report – high performing organizations relied more on peer review and less on external approval of change. The

OUR TEST SUITE i. Incorporate automate tests to validate the “-ilities” that are important (availability, capacity, security, etc.) ii. Incorporate security hardening testing and evaluation m. PULL due to potential rework and merge problems 1. Cross-cutting dependency improvements often provide high-payoffs d. ADOPT TRUNK-BASED DEVELOPMENT PRACTICES i. CI and Trunk-based development (TBD) are countermeasures smoke test to validate system behavior vi. Provides fast feedback vii. 2014 State of DevOps Report – high performers had deployment lead times measured in minutes or hours; low performers had deployment

performing work in small batch sizes. 2. Large batch sizes result in skyrocketing levels of WIP and high levels of variability in flow that cascade through the entire manufacturing plant. The result is released to production deployment. Like in manufacturing, this large batch release creates sudden, high levels of WIP and massive disruptions to all downstream work centers, resulting in poor flow and suppressing, even punishing, learning and improvement, perpetuating quality and safety problems ii. high-trust culture 1. we are all lifelong learners who must take risks in our daily work 2. we learn

 our governance decisions would not be based on a set of required requirements, but instead on high-level objectives. Instead of telling the project team how to do its job, we might begin the project To show me what the initiative has delivered so far, the team prepares a Value Delivery Register—a high-level summary of the value that has been delivered.  The second thing I want to know is how much near future and more coarse-grained and tentative for the long term.  Then I need to know what its high-level budget is for that future work.  By comparing these two things—roadmap and budget—I can decide

to enable rapid and frequent delivery of capabilities Planning DOs and DON’Ts  DO establish some high-level planning, requirements, processes, and structure; however, activities focus on what teams can requirements within the entire development period of performance.  During the program execution phase, a high-level program estimate undergoes refinement to create detailed release and sprint-level estimates

words.” What is the goal? Cost-effective purchasing? Employing good people? High technology? Producing products? Producing high quality products? Selling quality products? Capturing market share? After

delivering IT systems, where complexity is overwhelming and the number of potential failure modes is high.  In weighing the things known for certain against the uncertainties of the future, we are sure passes all of its tests. But is quality a simple yes-or-no attribute? If it passes its tests, is it high quality? And how can we assess the quality of something that is unfinished—that will later be added