--- title: Spirit Room · Train AI agents on your real APIs and MCP servers description: Spirit Room runs AI agents against your API, CLI or MCP server, scores every run against your requirements, rewrites tool descriptions, docs and errors, and re-scores. Zero code. Early access pilots. canonical: https://spiritroom.app/ source: https://spiritroom.app/ status: early-access (not generally available) generated: 2026-10-14 --- > Markdown version of https://spiritroom.app/ for agents. Product screens are illustrative (Example data). Agent instructions: /for-agents.md · Index: /llms.txt · Contact API: POST /api/contact Early access · limited pilots · Q4 2026 [New use case · cut token cost: make cheaper models succeed on your tools ↓](/#downshift) # One day outside. *One year inside.* Every AI agent that touches your API, CLI or MCP server is a user you can win or lose. Spirit Room puts those agents through a year of training in a day, scored against your requirements, and shows you, model by model and harness by harness, where they succeed and where they break, so even cheaper models get it right. No code changes. [Book a pilot: find what breaks your agents ](/#contact)[Join early access ](/#contact) 01 / The problem ## Your next million users are agents. *Most are failing silently.* Every failed agent run is a developer who decides your product “doesn't work with AI.” ### Agents fail quietly They skim `--help`, guess parameters, fall back to curl and declare victory. You see churn, never the reason. ### Tool surfaces aren't agent-ready MCP tool names, schemas, descriptions and error messages decide whether a model picks the right tool. Most were written in an afternoon. ### Every model × harness differs What works for one model in Cursor breaks for another in Codex. Nobody tests the matrix. 02 / Who it's for ## Built for teams whose users *are now agents.* i. ### MCP servers Names, descriptions and schemas tuned so models pick the right tool first time. ii. ### Public APIs Docs, OpenAPI descriptions and errors agents can recover from. iii. ### Dev-tool companies CLIs and SDKs that work in Claude Code, Codex and Cursor, out of the box. iv. ### Internal harnesses System prompts and internal tools verified against your requirements. 03 / Dream outcome ## Agents that get it right *on the first try.* Every agent, in every popular harness, reads the right line, picks the right tool, recovers from errors and finishes the task your way, measured against your requirements. Your agent, before Skims the help. Guesses a flag. Reaches for curl. Declares victory. 9.4 steps · 2/5 reqs → After one year inside Reads the right line. Picks the right tool. Recovers from the error. Done. 3 steps · 5/5 reqs Illustrative · Example data 03b / Cost optimization ## Same pass rate. *Cheaper model.* Frontier models can muddle through vague tool names, thin schemas and cryptic errors. Smaller, cheaper models can't: they need every description, parameter and error message to be unmistakable. Spirit Room tunes your API, MCP server, CLI and harness until the cheaper tier succeeds too, so you can downshift models and cut token cost without losing quality. Model downshift · task pass rate · Example data | Model tier | Before tuning | After tuning | Cost / successful task | | --- | --- | --- | --- | | Frontier | 86% | 94% | $0.42 → $0.36 | | Mid-tier | 58% | 91% | $0.19 → $0.09 | | Small / cheap | 31% | 88% | $0.11 → $0.03 | Illustrative mock · Example data, not a measured result. Cost per successful task = token cost ÷ passing runs. The value equation - **Dream outcome:** frontier-level pass rate on your tasks - **Likelihood:** measured per model × harness against your requirements, not guessed - **Time:** one pilot, about two weeks - **Effort:** zero code changes, winners come back as a patch or via Wrap Tell us your target model in the pilot form and we will test whether it can carry your tasks. [Book a pilot: test a cheaper model ](/#contact) 04 / Mechanism ## The mechanism: a training loop for your *tool surface.* Zero code. Spirit Room wraps your existing CLI, API or MCP server as a proxy and overrides surface values to run experiments. Your source and MCP definitions are never touched. [Start with a pilot ](/#contact) 1. ### Wrap Proxy your CLI, API or MCP server and override names, descriptions, schemas, help, errors, docs and prompts. 2. ### Run A task suite, QA for agents, runs real tasks with evaluation criteria across models and harnesses. 3. ### Score Every run is checked against written requirements (scoring engine: Proof). 4. ### Listen Agents self-report friction and confusion during each run. 5. ### Rewrite Variants of descriptions, docs and errors are generated, re-run and kept only if they win. 6. ### Re-score. Repeat. Hundreds of rounds in a day. Winners ship as a patch you review, or are served live by Wrap. 05 / Agent journey ## See exactly where your agent *cut the corner.* Every run becomes a journey: what it read, what it guessed, where it gave up and improvised. Each step is annotated with what to change in your surface. Two phases, as in regulated engineering: **design-time verification** through the training loop, and **runtime monitoring** in production to confirm agents still succeed as models, harnesses and your API change. **[Illustrative mock · Example data, not a real benchmark]** journey · rotate\_api\_keyclaude-sonnet × claude-code · round 0 00:00.401 $ acme --help | head -10 Read 10 of 42 lines. CUT CORNERThe `keys rotate` subcommand is documented on line 11. The agent never saw it. 00:03.102 $ acme keys create --replace error: unknown flag (no hint) INSIGHTThe error offers no recovery. Variant B: “did you mean `keys rotate`?” 00:07.903 $ curl -X DELETE …/keys/old && curl -X POST …/keys Deleted the old key before creating the new one. 00:09.204 proof verify --task rotate\_api\_key 2 / 5 requirements · REQ-3 “zero-downtime rotation” violated agent self-report › “I wasn't sure whether rotate existed, so I used the API.” ROUND 6 Help reordered, error hint added **3 steps · 5/5 requirements · 94% pass across 12 cells** Illustrative mock · Example data, not a real benchmark 06 / Inside the room ## What you get to see. *Every model, every harness.* Illustrative mocks · Example data **[Illustrative mock · Example data, not a real benchmark]** matrix · pass rate by model × harness | | Claude Code | Codex | Cursor | ChatGPT | | --- | --- | --- | --- | --- | | Claude | | | | | | GPT | | | | | | Gemini | | | | | | Open-weight | | | | | Illustrative · Example data, not a real benchmark **[Illustrative mock · Example data, not a real benchmark]** requirement score · by round 0% from 31% at round 0 · 25 tasks Illustrative · Example data **[Illustrative mock · Example data, not a real benchmark]** variant B · tools/search\_issueswinner · +32pp ``` - "description": "Search issues." + "description": "Full-text search over issues in ONE project. Requires project_key + (e.g. 'WEB'); call list_projects first if unknown. Max 50 results; use cursor." … - error: "400 Bad Request" + error: "project_key missing. Call list_projects to find it, then retry." ``` wrong-tool calls **−62%**avg steps **9.4 → 4.1**pass **58% → 90%** Illustrative · Example data, not a real benchmark **[Illustrative mock · Example data, not a real benchmark]** runs · live - create\_invoicegpt · codex**5/5** - refund\_partialgemini · cursor**3/5** - rotate\_api\_keyclaude · cc**5/5** - paginate\_allopen · custom**2/5** - search\_issuesclaude · cursor**4/4** Illustrative · Example data Want this matrix for your own API, CLI or MCP server?[Book a pilot ](/#contact) 07 / Evidence ## Evidence, *not vibes.* Each run is checked against explicit requirements and leaves a traceable record. Want to see what requirement-level evidence looks like? [Example: live public evidence dashboard (Omarchy) ](https://portal.reqproof.com/projects/omarchy) Early design partners **Proof**requirements & verification **Probe**by ProbeLabs Proof and Probe (ProbeLabs) are shaping Spirit Room during early access. 08 / The offer ## The offer: train once, or *wrap and stay ahead.* A · Train pilot · “Agent Readiness Sprint” ### Fixes, exported into your own code. A pilot finds the best descriptions, help text and errors per model and harness, then exports the winners into your code and MCP definitions as a reviewed patch. - **Agent Readiness Audit**: where agents fail today, 3 models × 2 harnesses, up to 25 tasks - **Training rounds**: up to 10 rewrite / re-score rounds - **Agent Journey Report**: annotated journeys for every failing task - **The Winning Patch**: best variants as a reviewed PR - **Evidence pack**: before/after per requirement, per model × harness - Pilot bonuses Error-message rewrite pack · Agent-ready docs checklist · One re-score after your next release (within 60 days) [Book a Train pilot ](/#contact) B · Wrap · “Always-on Agent Surface” ### An always-on, optimized surface. A subscription proxy in front of your CLI, MCP server or API that serves the optimized surface to each model and harness. Zero code changes. - Runtime monitoring: are agents still succeeding? - Agent-journey analytics on real traffic - Engineered for reliability, with pass-through fallback to your original surface - Optional, opt-in docs interception on domains you allowlist (off by default) [Early access to Wrap ](/#contact) 08b / Wrap · Adaptive surface ## One surface *per model.* Wrap detects which model is behind each session and serves the surface that model does best with: tool names, descriptions, schemas, help, errors and docs, tuned per model when it matters. For specific tasks it can serve a super-optimized surface, merge several tool calls into one, and add scripted *virtual* tools composed from your existing ones. Everything is opt-in and customer-controlled, with pass-through fallback to your original surface. Adaptive surface · live sessions · Example data | Model detected | Variant served | Changes | | --- | --- | --- | | frontier-large | base | original surface (pass-through) | | mid-tier | v-mid-3 | explicit enums, shorter descriptions | | small-fast | v-small-7 | renamed tools, examples in schema, error hints | Merged call · opt-in per task ``` before: list_projects → get_project → list_issues → get_issue ×3 (6 calls) after: project_issue_digest(project="api", state="open") (1 call) ``` Virtual tool · scripted from existing endpoints ``` tool release_readiness(repo) = GET /checks?ref=main + GET /issues?label=blocker + GET /deploys/latest → { ready: bool, blockers: [...], last_deploy: "…" } ``` Illustrative mock · Example data. Model names generic; merges and virtual tools run only where you allow them. Customer-controlled - **Per-model variants:** served only when they score better for that model - **Merged calls:** opt-in per task, only for calls you mark safe to combine - **Virtual tools:** scripted on top of your endpoints, reviewed by you - **Fallback:** any error or unknown model gets your original surface - **Audit:** every variant, merge and script is versioned and logged [Talk to us about Wrap ](/#contact) 09 / Wrap · real traffic · early access ## Wrap keeps learning from *real agent traffic.* Local-first and opt-in. Nothing leaves a machine without explicit consent. ### Gateway mode Run inside LLM apps and gateways (e.g. Tyk AI Studio or OpenRouter-style gateways) to observe how real agents use your tools, and turn failures into training tasks. ### Local agent An opt-in agent on developer machines reads Codex, Claude Code and Cursor session history to surface friction with your tools, MCP servers and APIs. ### Docs interception optional · off by default When enabled, Spirit Room intercepts the agent's HTTP requests to your docs sites and serves tuned, experimental versions. A/B per model and harness shows which wording leads to success. Strictly opt-in, scoped to domains you own and allowlist. 10 / Pricing ## Two weeks inside. *Priced as a pilot.* Platform fee plus compute. Choose the models and harnesses to test, or bring your own API keys and pay only the platform fee. What this takes by hand · rough estimates | | | | --- | --- | | Build a model × harness eval harness | 3–6 eng-weeks | | Run and grade hundreds of agent sessions | 1–2 eng-weeks / round | | Write and A/B test descriptions, errors, docs | ongoing | | Requirement-level evidence for every run | rarely done | What the pilot delivers Evidence and an experience: measured before/after results for every model × harness you pick, annotated journeys, and the winning variants. If your surface is already agent-ready, you get proof of that, which is useful too. Limited pilot slots · we run pilots hands-on Early access Free - Join the waitlist - First invites & updates - Shape the roadmap [Join early access ](/#contact) Train pilot · recommended $2,500one-time · 2 weeks - 1 surface: MCP, API or CLI - Up to 25 tasks - 3 models × 2 harnesses - Up to 10 training rounds - Compute included (capped) - Journey report + winning patch - Cheaper-model downshift check - Evidence pack + 3 bonuses - Credited 100% toward Wrap [Book a pilot ](/#contact) Wrap · Team $990/mo from · + compute - 3 surfaces · 100 tasks - Wrap proxy, zero code changes - Adaptive surface per model - Merged calls & virtual tools (opt-in) - Any models & harnesses - Re-score every release - Runtime monitoring & journeys [Talk to us ](/#contact) Enterprise Custom - Private deployment - Internal harnesses - Gateway & local agents - SSO, BYO keys [Contact ](/#contact) Early-access pilot pricing; may change. Compute above the included cap billed at provider cost plus a small margin. 11 / Questions ## Before you *step inside.* [All questions →](/faq/) ### Can Wrap serve a different surface to each model? Yes, if you turn it on. Wrap detects the model behind a session and serves the variant that scored best for it: tool names, descriptions, schemas, help, errors and docs. For specific tasks it can also merge several tool calls into one (only where you allow it) and expose scripted virtual tools composed from your existing endpoints. It is all opt-in, customer-controlled and versioned, with pass-through fallback to your original surface. ### Can this let us use cheaper models? Often, yes, and that is a key use case. Cheaper and smaller models need tool surfaces to be very clear. Spirit Room tunes your descriptions, schemas, errors, docs and harness prompts, then measures whether a cheaper tier reaches your target pass rate. Results vary per surface; we report what we measure, with no guarantees. ### Is Spirit Room available today? We're in early access, running a limited number of hands-on pilots. Join early access or book a pilot and we'll reach out. ### Do I need to change my code or MCP server? No. Spirit Room wraps your surface as a proxy and overrides descriptions, help, errors and docs during experiments. Winners return as a patch you choose to merge, or are served live through Wrap. ### Which models and harnesses? GPT, Claude, Gemini and open-weight models, across Claude Code, Codex, Cursor, ChatGPT and your own internal harness. ### What does “scored by Proof” mean? Each task has written requirements. Proof checks every run against them, so improvements are measurable and traceable. ### What about my data, keys and docs? Use sandbox credentials or bring your own keys. The local agent is opt-in and local-first. Docs interception is off by default and limited to domains you allowlist. ### What if my surface is already good? Then you get proof of that: measured results per model × harness and the journeys to back it up. ### Are the numbers on this page real? No. Every product screen is an illustrative mock labeled “Example data”. [Book a pilot ](/#contact)[Join early access ](/#contact) 12 / Close ## Your agents get one day outside. *Give them a year inside.* Tell us what your agents should master. We reply within two business days. Limited pilot slots. ## Contact (form) The page has a form that posts JSON to `POST /api/contact` with fields `name`*, `email`*, `train`*, `company`, `intent` (pilot|early_access), `path` (train|wrap|undecided), `tier` (waitlist|pilot|team|enterprise). Agents: submit only on your user's explicit request. Details: /for-agents.md