EVALUATING THE EFFECTIVENESS OF PULL REQUEST PROCESSES i. Review production outages and examine the peer review process for the relevant changes ii. Sampling and examining pull requests from larger population population or those relevant to production incidents 1. Bad pull requests – typically doesn’t have enough context for the reader, little documentation of the changes intended outcome, or explanation of of the thought process behind the change 2. Good pull requests – sufficient detail on why the change is made, how the change was made, and identified risks and resulting countermeasures j. FEARLESSLY

problem to come along Changing the Laws of Engineering with GitHub Pull Requests by Ralph Bodenner (New Relic) https://bit.ly/pull-request-your-culture  How you change your process matters  Terms: culture

(availability, capacity, security, etc.) ii. Incorporate security hardening testing and evaluation m. PULL OUR ANDON CORD WHEN THE DEPLOYMENT PIPELINE BREAKS i. Keep the build in a green state ii. When a human behaviors and incentives is more challenging than the technical aspects n. WHY WE NEED TO PULL THE ANDON CORD i. If not, it becomes increasing difficult to get back to a deployable state ii.

affected by the problem 6. Anyone else who is interested in attending the meeting iv. Guidance 1. Pull all factual evidence (chat logs, etc.) to help build the timeline; any specific metrics observed

a sixty-four-character text box? What idiot built that form?”  “When my guys put in a change request, they have to wait a lifetime to get approvals, let alone get on the schedule. We have the business

group is private so the first time you visit you likely have to wait for an admin to approve your request to join but don’t let that deter you! We value your feedback and we need it to learn and do better

III Post-System Functional Review Report MAIS & ACAT III (Required for space programs only) Request for Proposal MAIS & ACAT III System Threat Assessment Report MAIS & ACAT III Systems Engineering