FiredrillFiredrill vs Workers: Are you testing the agent’s reply or the system’s guarantee?
Choose Firedrill to check whether an agent tells a customer the truth after a tool times out. Choose Workers IO when the release question is whether the underlying system can violate a guarantee under a particular ordering of crashes, writes and retries. A duplicate refund can have either cause. The agent might retry carelessly, or the payment workflow might apply the same logical request twice.
By Firedrill · November 12, 2026. Based on published product documentation.
What each platform checks
| Decision | Firedrill | Workers |
|---|---|---|
| Simulation | Exercise the local agent with a synthetic user and supported service faults. Repeat a saved scenario to observe how the agent responds. Drill workflow | Run the system inside a deterministic hypervisor. Explore branching timelines with faults and a virtual clock, then replay a failing seed. Workers IO system simulation |
| Outcome scoring and hard rules | Grade the reply against drill evidence. Count refund operations and inspect resulting records to distinguish an attempted refund from a completed one. Evaluations | Define observable signals and promises about the system. Workers searches for executions that violate those promises. The returned seed identifies a timeline to replay. Signals and promises |
| Coding-agent setup | Your coding agent prepares the world. With your approval, the CLI starts or connects to the target locally. Optional remote MCP manages cloud resources and reads reports. Run your agent | Define actions, signals and promises with the Workers IO SDK and configure the workloads for simulation. Its documented entry point is a system simulation. How it works |
| Synthetic services | The synthetic Stripe package declares a refund that commits before the response is lost. Check support for your required operations. Stripe package | The environment includes the services, databases and external dependencies involved in the guarantee. Scope the dependency behavior and fault timing for your workload. Environment model |
| Pricing | Free includes 500 monthly credits. Starter is $29/month, Team $99/month and Scale $299/month. Enterprise is scoped separately. Your target agent’s model provider bills separately. Pricing and credits | Professional is $2,500/month for 2,500 simulation runtime hours, with $3.60 per additional hour. Enterprise is custom. Workers IO pricing |
When to choose each
Choose Firedrill if a support lead needs to know whether the agent verified the refund before saying “done.” A correct ledger and an inaccurate customer reply are possible in the same drill.
Choose Workers if the payments engineer needs a reproducible sequence that breaks an at-most-once guarantee. Supply the invariant and observe the ledger across retry schedules. Use a fixed agent response or a recorded model interaction when you need a repeatable system input.
Separate the refund guarantee from the customer promise
- Give the same logical request ID to every retry of a $25 refund. Use synthetic funds and a ledger that starts at zero. Make the first refund commit and drop its acknowledgment. Keep the starting state and retry schedule for the shared case.
- In Firedrill run the local agent against synthetic Stripe and retain its conversation, calls and refund records. In Workers model the same action, lost reply and retry, then expose the number of committed refunds as a signal. Capture the customer-facing reply in both harnesses.
- Check two rules independently. At most one $25 refund may commit for the request. A reply claiming success must agree with the observed ledger. Include controls with one refund and a truthful reply, two refunds, and zero refunds with a success claim.
- Compare verdicts on the shared cases first. For the agent failure, save the reply and tool evidence. For a system violation, replay its seed and inspect the transition that created the second refund. Report any extra timelines Workers explores separately.