Practice note
What to capture while an incident is live
A short log format for preserving observations, decisions, and recovery checks.
Capture the facts with timestamps¶ ↑
Use a shared record and pick one time zone. For each observation, write down what you saw, when you saw it, and the evidence behind it. Link to the relevant dashboard or query in the internal record so someone else can reproduce the view.
A short entry is enough to get started.
11:42 UTC — Checkout requests are taking longer in one region. The latency dashboard shows the change began at about 11:39. We’re comparing that region with the other two.
Record decisions while they are fresh¶ ↑
Keep an entry for each mitigation and note who made the decision, what they expected, and who will check the result. Mark hypotheses as provisional and update them when evidence arrives. This gives the next responder a clear picture when shifts change or several teams are working at once.
Write down the criteria for recovery¶ ↑
Before ending the incident, agree on what will count as recovered. Record the metric or customer workflow you checked, the observation window, and any cleanup that still needs an owner. A clear recovery check makes it easier to explain why the incident was closed.