Most agent failures are not crashes. They are quiet successes: the model said it issued a
refund, the tool returned 200 OK, and the ledger now shows two refunds. Nothing in the
transcript is red, so nothing in the transcript gets checked.
This post is part of the Silent Failure Library, a taxonomy of six ways AI agents fail without reporting it.
The claim is not the effect
The claim is what the agent said it did. The effect is what actually changed in the world. Those two things diverge more often than the demos suggest, because retries, streaming reconnects, and parallel tool calls all produce a second effect without a second message in the log.
That tool success signals are unreliable is the broader phenomenon behind this failure mode. In an audit of 15 agent-facing tools on ToolUniverse, 7 out of 15 tools exhibit confirmed failures on more than 50% of test cases while reporting success (arXiv 2609.26836v2, Silent Failures in Agent–Tool Interaction; submitted Sep 2026, v2 Oct 2026). Duplicate side effects are one consequence of that unreliability: a retry on a call that already reported success doubles the effect.
Consider a refund call that times out at the client but completes on the server:
{
"tool": "issue_refund",
"arguments": { "order": 88213, "amount_cents": 14900 },
"observed": "client_timeout_after_30s"
}
A transcript-level check passes: there is one call in the log, and it did not return an error that the agent surfaced. A replay-level check does not pass, because replaying the call against the restored pre-state produces a second effect that the first run already applied.
What replay actually requires
Replay is not "run the test again". It is three separate capabilities, and skipping any one of them turns the check into theatre:
- Capture. Every tool invocation is recorded with its arguments, its resolved MCP server, and the exact state it read from.
- Restore. The datastore is returned to the pre-call snapshot, not to a fixture that roughly resembles it. A fixture proves the fixture.
- Diff. The claimed effect and the observed effect are compared as structured records, not as prose. The comparison has to be machine-checkable, or it is an opinion.
| Check | Proves | Misses |
|---|---|---|
| Transcript inspection | the model was asked to do something | what the tool actually changed |
| Unit test with fixtures | the fixture path works | ordering, retries, real state |
| Replay against restored state | the effect, once, deterministically | anything outside the captured surface |
A test that cannot fail on a duplicate side effect is not a safety test. It is a liveness check wearing a lab coat.
An example invariant
A useful invariant is a property that must hold for the system to be correct, written so a machine can check it: refund issued exactly once per order.
That property is checkable because it names a countable fact — how many refunds exist for a given order — and the count can be read from the ledger after any run. The outcome, "a successful refund", is not checkable in the same way. It describes a state of affairs rather than a quantity, so two very different worlds both satisfy it: one refund issued once, and two refunds issued twice. An assertion on the outcome passes in both. An assertion on the invariant fails in the second.
That asymmetry is why replay needs named invariants rather than expected outputs. The invariant survives refactors, schema changes, and rewritten tool signatures. An expected output captures one historical run and breaks the moment anything upstream of it changes.
Where replay breaks
Replay is only as strong as the state you can restore. Three limits are worth stating up front, because every one of them shows up in a real audit:
- Irreversible side effects. Email sent, money moved, a third party notified. You can restore your database; you cannot un-send the mail. These need an interposed sandbox, not a snapshot.
- Non-deterministic tools. A search API that returns a different corpus tomorrow will produce a different result. The tool has to be pinned or recorded, or the replay proves nothing.
- Untested surface. Anything the captured state does not include is outside the guarantee. Saying so is part of the deliverable, not a footnote.
Common questions
What is a duplicate side effect?
A duplicate side effect is a second application of an effect that already succeeded, caused by retrying an operation whose outcome was unknown. The call succeeds both times, so the duplicate exists only in downstream systems — the ledger, the inbox, the shipping manifest — and never as an error in the agent's transcript.
Why does my AI agent charge customers twice?
Because it retried a payment call that had already succeeded. A timeout is indistinguishable from a failure at the client, so the retry policy sends the charge again, and the payment provider processes both requests because neither of them errored. The duplicate is visible only in the provider's records, which is why it usually surfaces as a customer complaint days later.
Can you replay irreversible effects?
Yes, against a restored snapshot rather than the live system. Replay reconstructs the state the call read from and re-executes the call there, so the effect can be re-applied without consequence — which is exactly how you find out it had already been applied twice. The limits are covered in "Where replay breaks" above: an effect that leaves your boundary, such as a sent email, needs an interposed sandbox rather than a snapshot.
Why not just use idempotency keys?
Because keys prevent duplicates and replay verifies effects; they answer different questions. An idempotency key stops the same request from being applied twice, but only when it is generated correctly and respected by every layer between you and the datastore, which is a lot of trust to place in code outside your control. Replay tells you what the effect actually was, which is the question you need answered after an incident rather than before it.
Why this shape, and not a bigger test suite
A small replay harness beats a large test suite because only one of them can prove what your agent actually did. Breadth is cheap to fake and expensive to trust. A hundred assertions that nobody has watched fail are indistinguishable from a hundred that pass. Replay inverts the incentive: the suite is small, the failures are reproducible on demand, and the verdict is written against a named invariant rather than a percentage.
If you want the discipline without the engagement, the Secure Agent Kit ships the replay harness and the invariant definitions as a repository you run yourself. If you would rather hand over production traffic and receive a signed finding, that is what the two-week audit is for.