Guidance 1. Pull all factual evidence (chat logs, etc.) to help build the timeline; any specific metrics observed, investigative paths taken, results, and other resolutions considered 2. Don’t allow fear infrastructure, etc. 2. Security group changes 3. Changes to configurations 4. XSS, SQLi attempts 5. Server errorsii. Consider blocking and storing source of events when attacks are detected to facilitate deployment pipeline demonstrates that changes can be low risk and pre-approved. ii. Use previous metrics to demonstrate high success rates and low MTTR iii. Link and provide traceability from planning

ability to learn from mistakes and diminish integrating that learning into future work d. Google Web Server (GWS) team was struggling with changes – Hard line: no changes would be accepted into GWS without Expands upon Canary. Links production monitoring with release process to automatically roll back when metrics deviations, outside of expected ranges, are detected i. APPLICATION-BASED PATTERNS TO ENABLE SAFER

...................................................................................... 41 12 Metrics ................................................................................................ manager and/or systems engineer) who coordinates and integrates programmatics (e.g., schedules, metrics) and deliverables (e.g., code)?  Is there a clear owner of the program (or broader enterprise) review/approve?  Does the government have access or insight into the development environment (code, metrics)?  At the end of a sprint, does the team reflect on what worked and how to improve in future

environments layer a. Events, logs, & metrics b. Common service to centralize, rotate, and delete 2. Event router responsible for storing our events and metrics a. Enables visualization, trending, alerting good things b. Enable threshold-based alerting & health checks iii. Transform logs/events into metrics to enable statistical analysis iv. Adrian Cockcraft – “Monitoring is so important that our monitoring prevent them in the future ii. Use for fact-based problem-solving e. ENABLE CREATION OF PRODUCTION METRICS AS PART OF DAILY WORK i. Create infrastructure to make it easy for Dev or Ops to create telemetry

browser Metrics, metrics everywhere (but where the heck do you start?) by Tammy Everts & Cliff Crocker (SOASTA) @tameverts @cliffcrocker http://www.slideshare.net/CloudTest/metrics-metrics-everywher

how Agile can underpin EVM progress in support of tracking program cost and schedule status and metrics • The scenario is a generic SW program implementing an Agile development process • The scenario

processes, or review processes 2. Places where significant rework is generated or received. iii. Key metrics in value stream 1. Lead Time 2. Process time 3. Percent Complete & Accurate (%C/A) c. CREATE

统⼀告警 ⽀持指标告警、⽇志告警、智能告警，⽀持⼏⼗种数据源对接，收集各类监控系统的告警事件，进⾏统 ⼀的告警收敛、降噪、排班、认领、升级、协同，⼤幅提升告警处理效率。 统⼀观测 将 Metrics、Logs、Traces、Events、Profiling 等多种可观测性数据融会贯通，并预置⾏业最佳实践， 既提供全局业务视⻆、技术视⻆的驾驶舱，也提供层层下钻的故障定位能⼒，有效缩短故障发现和定位

”  “I have to manually type in hundreds of server names in one of the text boxes. Most of the time, there’s not enough room in the field! A hundred server names are supposed to fit in a sixty-four-character

technology under the sun  Solaris, Windows, Linux  Apache, IIS, TCServer, etc.  Oracle, DB2, SQL Server  How we got better  We read and we studied.  Created a self-improvement project  2 week iterations