Firedrill

Run your first AI agent simulation at a hackathon

The problem

AI agents need a safe place to practice the actions they’ll take in production. They need to call tools, interact with MCP servers, and make API requests to external apps. Testing against live systems can send real messages, change real data, or trigger workflows. Reading the agent’s replies alone won’t tell you which calls it made, how it handled failures, or what it changed.

What Firedrill is

Firedrill is a simulation platform for AI agents. Run your agent locally against synthetic tools and services it can call over MCP or HTTP, without touching the live versions of those services. Define a scenario and the expected outcome, then inspect the conversation, tool calls, and recorded state changes in the drill report.

Get set up

  1. Create a Firedrill account and select your workspace.
  2. If your event provided a code, open Billing in the web app and apply it under Event code.
  3. Connect your coding agent. Check firedrill --version first; use 3.0.5 or newer. Update an older install with npm install -g firedrill-run@latest. Ask the agent to prepare one scenario for the application you are building.
  4. Review the plan and credit estimate. After approval, run the agent locally against the synthetic world and open the drill report.

Apply your event credits

Tokens& Personal Agents Hackathon: get 1,500 extra credits on top of your Free credits. One claim per person and organization while awards are available. The extra credits do not expire. Free workspaces also get up to 5 builder seats for 7 days, including the seat already on Free. Redeemed before the increase? We added another 1,000 credits automatically. Check Billing. You don’t need to apply the code again.

Event code: TOKENSAND · Open the prefilled redemption page

  1. Sign in and select the workspace you want to use.
  2. Open Billing and paste the code under Event code.
  3. Select Apply code. Billing shows the credit award and the new balance. No payment card or subscription is required.

Get help

If sign-in expires or the CLI hasn't selected a workspace, follow the agent setup guide. If applying returns billing_builder_assignment_required, an organization admin can check the person's seat under Billing → Builders. Assigning an available included seat does not require payment.

Plan exceeds the conversation budget

If firedrill plan returns native_participant_capacity_exceeded, the plan allows more simulated participant messages than its conversation budget. It is not a low balance or a full queue. Nothing started or reserved credits.

Ask your coding agent: “Check the scenario revisions pinned in my world. Set max_turns to the fewest simulated participant messages each scenario actually needs. For five short drills that need one or two messages each, try 2 turns per drill. Save the scenarios, update the existing world to point to those exact revisions, and show me a fresh firedrill plan estimate. Don’t apply or run yet.”

max_turns (shown as Maximum turns under Scenarios → Edit) caps messages within each drill, not how many drills run. Check the total across scenarios and repetitions. If a larger group still won’t fit, plan smaller batches without changing the scenario intent. Saving a scenario alone does not update the version in your world.

Join Firedrill on Discord to ask a setup question or share what you are building.

What to show in your demo

Give the agent a task with a checkable outcome. For an inbox assistant, you could ask it to draft a reply and leave the message unsent. Inspect the recorded actions and the synthetic mailbox. Change the agent and repeat the same scenario to see whether the result holds.

The trap to avoid

Don’t count a convincing reply as a pass. Check what actually changed in the tool. For a scheduling agent, verify there’s exactly one event at the right time and that no invitations went out before approval.

Run your agent · Explore synthetic tools · Pricing and credits