Case studies / Platform engineering
How an on-call rotation stopped writing incident reviews from memory
Before Murmurator, the write-up after an incident depended on which engineer was on call, how much of the timeline they could still reconstruct, and how much of the week was left.
The draft review is now produced by a workflow that reads the same sources in the same order every time, and a person edits a document rather than reassembling an evening.
Steps in the workflow
8
Two call a model. The other six gather evidence or write it down.
Sources read on every incident
4
Alerts, deploys, the incident channel and the dashboards — in that order, whoever was on call.
What the workflow publishes on its own
A draft
Nothing is closed, assigned or communicated until a person edits and says so.
This is an example scenario, not a customer. We wrote it to show how this kind of work is put together in Murmurator. The workflow is real and you can build it; the team, the people and the situation are invented. Nothing on this page reports a measured result, because we do not have one to report.
The problem
The write-up that gets worse the longer it waits
Every team has this one. The incident is resolved at 02:40, the review is due Friday, and by Wednesday the timeline is a reconstruction. Some engineers had started pasting Slack scrollback into a model to speed it up, which helped — and produced a different document each time.
The same incident, once per engineer
- Which evidence gets gathered depends on who is writing and how tired they are
- The timeline is reconstructed from memory days later, and quietly gets smoother each time
- Two reviews of similar incidents are not comparable, so trends are invisible
- Contributing factors that nobody thought to look for are simply absent
- The engineer who is best at this becomes the person who writes all of them
Nobody is cutting corners. Reconstructing four hours across four systems is genuinely expensive, and the cost lands on whoever is least able to pay it that week.
The workflow
Gather first, reason second, publish nothing
The evidence gathering is mechanical and belongs to the workflow. The parts that need reading between the lines — what the timeline says happened, what is worth asking about — are the two model calls. What the workflow produces is a draft with an author still to come.
- The same four sources, in the same order, for every incident
- The model sees evidence the workflow fetched, not evidence someone remembered to paste
- Nothing is published, assigned or sent to anyone until a person edits it
2 steps of 8 call a model. The rest are the definition doing exactly what it says.
Trigger Deterministic
The incident is marked resolved in PagerDuty, or an engineer runs the workflow against an incident id.
Gather the record Deterministic
Alert history and responder timestamps from PagerDuty, the incident channel from Slack, deploys and merged pull requests in the window from GitHub, and the relevant dashboards from Datadog. Same four, same order, every time.
Build the timeline Deterministic
A sandboxed JavaScript step merges the four sources into one ordered list of events with typed output. Sorting timestamps is not a judgment call, and a model doing it would be slower, more expensive and occasionally wrong.
Read the timeline AI
A model call summarizes what the evidence shows: when it started, what changed immediately before, what the responders tried, and what finally moved the metric. It answers in a fixed shape, and is told to say when the evidence does not support a conclusion.
Draft the open questions AI
A second call proposes the questions a review should ask, based on what the timeline does not explain. Proposals, not findings — this is the step most likely to be wrong and it is deliberately the one with the least authority.
Validate the output Deterministic
Both calls declare a schema. A summary missing its timeline, or questions that came back as a paragraph, fail the step instead of becoming a document.
Write the draft Deterministic
The review is an artifact named for the incident, created from the account's template with the timeline, the summary and the questions filled in, and every version kept.
Hand it to a person Deterministic
A link goes to the incident channel with a note saying it is a draft assembled from the record. The engineer edits it; the workflow's job is over.
A model reasons here The workflow decides, the same way every run
Why it is built this way
Why two steps reason and six do not
The expensive part of an incident review is not writing prose — it is gathering four systems' worth of evidence at the worst possible time. That part is mechanical, so the workflow owns it. The judgment is left where judgment actually is.
AI
Summarizing what the timeline shows
Turning ninety Slack messages and forty deploy events into "the cache warmer was deployed at 01:58 and the error rate moved at 02:03" is reading, not retrieval. There is no query that produces it.
AI
Proposing the questions worth asking
Noticing what is missing from an account is a genuinely open-ended task. It is also the step most likely to produce something confidently wrong, which is why its output is labelled as questions and never as findings.
Deterministic
Ordering events into a timeline
Sorting by timestamp has one correct answer. A model doing it is slower, costs tokens, and will occasionally reorder two events in a way that changes what the review concludes.
Deterministic
Publishing the review
A review is a statement the team makes about itself. A workflow can assemble the evidence and draft the words; putting a name on the conclusion is a person's job, so the workflow stops at a draft by construction rather than by policy.
The test for every step is the same: does this require reading a situation, or does it require doing the same thing correctly? Only the first kind gets a model.
Before and after
What actually changed
Before
Everyone's own AI process
- The evidence gathered depends on who is on call and how much time they have
- Timelines reconstructed from memory, days after the fact
- Reviews that are not comparable to each other, so patterns stay invisible
- Some engineers use AI for this, some do not, and the results look nothing alike
- The best writer on the team ends up writing all of them
After
One workflow the team owns
- The same four sources read in the same order for every incident
- A timeline assembled from records rather than recollection
- Reviews that share a structure, so trends across them are actually readable
- The model's two contributions are labelled, schema-checked and bounded
- A draft waiting on Friday instead of an evening spent reassembling Tuesday
What this would change
What the team gets out of it
The workflow does not make anyone a better incident analyst. It removes the four hours of archaeology that used to sit between the incident and the analysis, and it makes every review start from the same evidence.
- The review starts from a record rather than from what people remember
- Every incident gets the same evidence gathered, including the quiet ones nobody would have written up
- Reviews become comparable, which is the only way recurring causes ever get noticed
- The engineer's time goes into the analysis instead of the reconstruction
- What the AI contributed is visible in the run, step by step, if anyone asks
No numbers here, on purpose. Time saved, error rates and volumes depend entirely on the team running the process, and we have not measured this one. When a customer has measured theirs and is willing to publish it, their figures will appear here with their name on them.
More case studies
Read next
Turn an AI process into a workflow.
14-day free trial with $5 of built-in AI included. No card, no per-run fees, cancel anytime.