version: "1.0.0" name: principle-postmortem description: Postmortem principles — blameless culture, root cause analysis (5 Whys, Fishbone), incident document structure, MTTD/MTTR metrics. Auto-load when running a blameless review, facilitating RCA, tracking action items after an outage, or improving incident response processes.

<!-- preload-canary: SWB-PRELOAD-PRINCIPLE-POSTMORTEM -->

Postmortem

Incidents are inevitable. A postmortem converts failure into systemic learning — the goal is not to assign blame but to close the gap between what the system can do and what the environment demands. The prevent → detect → learn triad is incomplete without this phase.

Cross-references: swe-workbench:principle-resiliency (prevent), principle-observability#SLI / SLO / Error Budget (detect / MTTD-MTTR framing), principle-release-engineering#Rollback (remediation action items).

Blameless Culture

Blame suppresses signal. When individuals fear punishment, they conceal information, underreport near-misses, and avoid risky-but-necessary work. Blameless postmortems operate from the assumption that everyone acted rationally given the information and tools available to them at the time.

Guiding principles:

• "The system allowed it" — if a person could cause an outage, the system lacked the guardrails to prevent it. The fix belongs in the system, not the person.
• Psychological safety — participants must feel safe sharing the full sequence of events, including their own mistakes. Facilitation owns this; interrupt any language that assigns personal fault.
• Hindsight bias — it is easy to see the right move after the fact. Explicitly reconstruct the information available at each decision point; do not evaluate decisions using information that only became visible later.
• Counterfactual fairness — "if only they had done X" is only valid if X was a realistic, well-understood option at the time. If X required information no one had, the real gap is in the information architecture.
• Separate the review from personnel processes — postmortems feed the improvement backlog, not performance reviews.

Root Cause Analysis

A single proximate cause is almost never the full story. Three distinct layers exist:

• Trigger — the immediate event that precipitated the incident (a deployment, a traffic spike, a configuration change).
• Condition — the pre-existing state that made the system vulnerable to that trigger (an untested code path, a missing circuit breaker, a stale cache TTL).
• Root cause — the systemic gap that allowed the condition to persist undetected (absent pre-deploy checklist, no SLO alert on the affected path, no load test in CI).

Fixing only the trigger prevents the exact same incident; fixing the root cause prevents a class of incidents.

5 Whys

Iteratively ask "why did this happen?" until you reach a controllable, systemic factor. Each answer becomes the premise of the next question.

• Start from the trigger, not the symptom.
• Stop when the answer is "we had no process / no visibility / no incentive to catch this" — that is a root cause.
• Document each why-answer pair; the chain is the evidence.
• Beware shallow stopping: "human error" is almost never a root cause — ask why the error was possible.

Fishbone (Ishikawa)

Use when the incident has multiple, parallel contributing factors that do not form a clean causal chain. Draw a central "spine" (the effect) with branches for contributing factor categories:

• People — training gaps, handoff failures, role ambiguity.
• Process — missing runbooks, inadequate on-call rotations, unclear escalation paths.
• Tooling — alert fatigue, missing dashboards, slow deploy pipelines, inadequate rollback mechanisms.
• Environment — dependency failures, infrastructure limits, third-party SLA violations.
• Measurement — absent or lagging metrics, poorly calibrated SLOs, no baseline data.

Each branch can have sub-branches. Fishbone complements 5 Whys: use 5 Whys to drill into the most critical branch found by the Fishbone diagram.

Postmortem Document Structure

Time-box the draft to 24–72 hours after mitigation while memory is fresh.

Required sections, in order:

• Title + Severity + Status — descriptive title, severity tier (P0–P3 or equivalent), draft/reviewed/closed.
• Timeline — chronological sequence of events with UTC timestamps: when did the condition begin, when was the incident detected, when were responders paged, when was mitigation applied, when was the incident closed.
• Impact — who was affected, how many users/requests/dollars, duration. Quantify; avoid vague language.
• Root cause — one paragraph. Uses the trigger/condition/root-cause framing from above.
• Contributing factors — list of conditions that compounded the impact or delayed detection.
• Lessons learned — what went well (do not skip this; it anchors future behavior), what went poorly, where there was luck.
• Action items — see Action-Item Discipline below.

Action-Item Discipline

"Improve monitoring" is not an action item. An action item is a specific, measurable change assigned to a named owner with a due date.

Every action item must answer three questions:

1. What exactly will change? — name the alert, the runbook section, the circuit-breaker threshold, the test case.
2. Who is the owner? — one person; committees do not ship.
3. When is it done? — a specific date or sprint, not "soon".

Categorize by impact tier:

• Prevent — eliminates the root cause (adds the missing guard, removes the dangerous code path).
• Detect — closes the detection gap (adds the missing SLO alert, reduces MTTD).
• Mitigate — reduces blast radius for the next occurrence (adds a kill-switch, improves rollback speed).

Track in the same system as engineering work (sprint backlog, issue tracker). Postmortem action items that live only in a doc rot and get forgotten.

Incident Metrics

Track over rolling quarters to surface trends, not just individual incidents:

• MTTD (Mean Time To Detect) — gap between incident start and first alert. Feeds swe-workbench:principle-observability SLO work; high MTTD signals missing or miscalibrated alerts.
• MTTR (Mean Time To Recover) — gap between detection and full mitigation. Feeds swe-workbench:principle-resiliency rollback and degradation work; high MTTR signals slow deploy pipelines or inadequate runbooks.
• Repeat-incident rate — same root cause recurring within a quarter. The single strongest signal that action items are not being completed or are not addressing the right layer.

Review trends in retros, not just in individual postmortems. A rising MTTD means the observability investment is not keeping pace with system complexity.

When a Postmortem is Overkill

Not every incident warrants a full postmortem. Reserve the full process for:

• Customer-visible impact lasting more than a few minutes (adjust threshold to your SLO tier).
• Incidents that required escalation beyond the on-call engineer.
• Any incident in the same root-cause class as a prior postmortem (repeat signal).
• Incidents where the responder felt they were close to making a much larger mistake.

For minor incidents below this threshold, a lightweight 5-line "incident note" (what happened, what was done, follow-up if any) is sufficient. The overhead of a full postmortem can discourage reporting if applied indiscriminately.

Red Flags