For Developers‎ > ‎

Postmortems

Why Write a Postmortem

To understand how failures happen, in order to prevent future occurrences by education and process changes.

When To Write a Postmortem

A postmortem is expected for any tree closures lasting longer than 4 hours, within 72 hours of the outage. 

Who Should Write the Postmortem

The postmortem should be written by someone involved with detecting and correcting the issue, preferably someone who can take responsibility for the followup.

What to Include

Please use the postmortem template found here (file -> make a copy).

Your postmortem should include the following sections:
  1. Title
  2. Summary of the event
  3. Full timeline
  4. Root cause(s)
  5. What worked and what didn't (a.k.a., lessons learned)
  6. Action items (followup bugs assigned to specific people)

Where to Put It

Whenever possible, postmortems should be accessible to the entire Chromium community.  If you are a Google employee, and your postmortem contains internal details, see the internal infrastructure team's postmortem site instead.
  1. With your chromium.org, write it in a Google Doc, set sharing permissions to “Anyone who has the link can comment”
  2. Add it to the list below.
  3. Send the link to chromium-dev@chromium.org or infra-dev@chromium.org, as relevant.
See also:
Showing 8 items
TitleDate
Sort 
 
Sort 
 
TitleDate
Grit Compile Errors Require Clobber July 25, 2014 
Postmortem: 15 hour tree closure by a file named "about" May 9, 2014 
SPDY/QUIC Connection Pooling Bug Postmortem  
.dbconfig files loss on master3 September 19, 2014 
Swarming Postmortem: 2014-12-04 December 4, 2014 
chromium.perf tests on android userdebug builds failing for 7 days January 30, 2015 
No data from some android bots on chromium.perf for 2 weeks December 22, 2014 
Swarming Postmortem: Undeleteable directorie November 22, 2015 
Showing 8 items
Comments