need to be more available and scalable than the systems being monitored.” c. CREATE APPLICATION LOGGING TELEMETRY THAT HELPS PRODUCTION i. Dev & Ops create production telemetry as part of their daily designed and outcomes are being achieved. iii. Logging Levels 1. Debug – anything that happens in the program 2. Info – user driven actions or system specific 3. Warn – conditions that could become GTP Security & Risk Management group) 1. Authentication/authorization decisions 2. System and data access 3. System and application changes, especially privileged changes 4. Data changes (CRUD) 5. Invalid

consistent, & secure c. CREATE OUR SINGLE REPOSITORY OF TRUTH FOR THE ENTIRE SYSTEM i. ALL parts (code & environments) of the system are shared in a version control repository ii. Version control is for intervals iii. Demonstrated in a production-like environment iv. Ideally, use the same monitoring, logging, and other tools in pre-production environments as in production v. Dev & Ops gain shared mastery When a change causes the build or automated tests to fail: 1. No new work is allowed to enter the system until the problem is fixed. 2. Bring in whatever help is needed to resolve the problem iii. Prioritize

Accidents are due to the inevitable design problems in complex systems that we build; they are system problems – not individual problems iii. Effective practices 1. Blameless post-mortems 2. Controlled “could have” a. These are counterfactual statements b. Frames the problem as the system as imagined rather than the system that actually exists 4. Focus on – “Why did it make sense to me when I took that and behavior of libraries and components ii. Test suite becomes the living documentation of the system specification and represent working examples of API use e. DESIGN FOR OPERATIONS THROUGH CODIFIED

security stack in the security sidecar container will include: 1. A logging agent to push logs to a platform centralized logging service. 2. Container policy enforcement. This includes ensuring container

the government context, Agile represents a good development approach when customizing an existing system or commercial off-the-shelf (COTS) product, or building a small-scale or self-constrained application into an existing operational baseline, system, or platform. Although it may not be the easiest approach, the government can also use Agile to build a large IT system from the ground up; however, in this the operational baseline. Several large acquisition programs, such as the Global Combat Support System-Joint (GCSS-J), have adopted Agile methods to build a future increment or block of capability.

and last, that the business should be thought of as a community, or perhaps as a Complex Adaptive System, which needs to be led and managed through an inspect-and- adapt, feedback-and-vision-oriented approach dysfunctional transformation cycle. It lies, I believe, in our distinction between the development of a system and its operation and maintenance.  Dividing our IT spending into development and maintenance buckets way to incrementally modernize a legacy system as defined by Martin Fowler.  Instead of building an entirely new system, we take a small piece of the legacy system and rebuild it in a way that lets it

said it?Frederick Brooks Jr. • Joined IBM in 1956 • Manager for the development of the IBM System/360 family of computers and the OS/360 software package • In 1975, published The Mythical Man-Month: designersRequirements refinement and rapid prototyping • The hardest single part of building a software system is deciding precisely what to build • The most important function that software builders do for present-day software acquisition procedures rests upon the assumption that one can specify a satisfactory system in advance, get bids for its construction, have it built, and install it • This assumption is fundamentally

the system generates money through sales ● Inventory - all the money that the system has invested in purchasing things which it intends to sell ● Operational expense - all the money the system spends hours, or even half of it, you have lost it forever. You cannot recover it someplace else in the system. Your throughput for the entire plant will be lower by whatever amount the bottleneck produces in above decision (Making sure everything marches to the tune of the constraint) STEP 4 ELEVATE the system’s constraint (Bringing back old machines) STEP 5 If, in a previous step, a constraint has been

organization iii. ENABLING ORGANIZATIONAL LEARNING AND A SAFETY CULTURE 1. When we work within a complex system, by definition it is impossible for us to perfectly predict all the outcomes for any action we processes, often to help individual departments maintain their “turf.” Failure is processed through a system of judgment, resulting in either punishment or justice and mercy. c. Generative organizations OF RECORD AND SYSTEMS OF ENGAGEMENT i. Gartner Bi-modal IT 1. Type 1 – System of Record – “Doing it right” 2. Type 2 – System of Engagement – “Doing it fast” ii. DevOps helps reject the bi-modal IT

Relationships will be unpredictable and emergent • Limited constraints on agents in the system; agents also modify the system • Probe - Sense - Respond • Apply “Emergent Practices” Unordered Systems Obvious itself, rather than looking to outside best practices. • Set barriers - these limit behavior and the system can self-regulate within the boundaries & barriers • Stimulate attractors - phenomena that arise