# Spirit Room > Spirit Room optimises your MCP, CLI or API for AI agents. You give us the jobs agents need to get right; we run every major agent through hundreds of simulations, trying new tool descriptions, help text, errors and combined tools, and keep what makes agents finish in fewer steps with fewer mistakes, even on cheaper models. Site numbers are Example data. No guarantees. Offers: Pilot (USD 1,999, 2 weeks, start here) and Enterprise ongoing optimisation (custom, monthly, after the pilot). Safety: Changes are safe to ship: before anything goes live, every agent runs against the current and new version and real traffic is replayed against the new one; old tool names keep working, rollout is gradual and rollback is instant. Nothing has to route through Spirit Room. Options: (1) a Cloudflare Worker in the customer's own Cloudflare account; (2) the SDK inside their own MCP server; (3) fixes applied directly, handed over as a ready-to-paste list for their coding agent. Nothing calls Spirit Room at request time. ## Docs - [Instructions for agents](https://spiritroom.app/for-agents.md): offers, endpoints, JSON examples - [Home page as Markdown](https://spiritroom.app/index.md) - [FAQ as Markdown](https://spiritroom.app/faq.md) - [Agent policy](https://spiritroom.app/agents.txt) - [Terms](https://spiritroom.app/terms.md) - [Privacy](https://spiritroom.app/privacy.md) ## Guides and research - [How to make your MCP server or API agent-ready, with proof (guide hub)](https://spiritroom.app/guides/agent-ready-tools/): The four steps an agent goes through with your tools (discover, understand, act, recover), what fails at each one in real runs, and the guides that go deeper. - [Agent-ready tools glossary](https://spiritroom.app/glossary/): Plain definitions of the terms we use when testing MCP servers with agents, each grounded in what we saw in real captures and runs: deferred tools, tool search, code mode, peak vs cumulative context, pass/N, cost per success, AX and more. - [How to test an MCP server with real AI agents (guide)](https://spiritroom.app/guides/test-mcp-servers-with-agents/): Agent run testing: real jobs, ground truth, the agent apps your users run, read-only on production, pass/N with confidence ranges, and transcripts read by hand. - [How to measure MCP server quality: the agent scorecard (2026)](https://spiritroom.app/blog/measure-mcp-server-quality/): Fifteen metrics in three layers (server checks, agent behaviour per app, outcome and cost), each with a target and the measured value on a 99-tool production MCP server. - [MCP Inspector says your server works. Agents still fail. Here's what it can't catch](https://spiritroom.app/blog/mcp-inspector-vs-agent-testing/): A 99-tool production server answered every direct call correctly. Agents still missed the tools in 6 of 40 runs and hit a 25k-token refusal 40 times. What Inspector checks, and what only agent runs show. - [How to evaluate AI agents on your API: real jobs, ground truth and pass/N](https://spiritroom.app/blog/evaluate-ai-agents-on-your-api/): Eval platforms score the agent you built. If you ship an API or MCP server, your users bring the agent. A worked 8-job suite on a 99-tool server: how we wrote jobs, got ground truth from direct calls, caught our own scorer bugs, and reported pass/N per app. - [MCP benchmark and testing tools compared: load testing vs agent-quality testing (2026)](https://spiritroom.app/blog/mcp-testing-tools-compared/): MCP Inspector, MCPJam, k6, mcpbench, Promptfoo, MCPMark and more, sorted by the question each one answers, and what none of them can see. - [“10 of 10 passed” isn't 100%: reporting agent test results with confidence ranges](https://spiritroom.app/blog/agent-pass-rates-confidence-intervals/): Agent runs vary, so a pass count needs a range. 24 of 24 is 86–100%; 2 of 2 is 34–100%. Real examples from our MCP tests: when two apps are really different, when they aren't, and why repeat runs make ranges optimistic. - [How to test agents against a production MCP server without spending, writing or sending anything](https://spiritroom.app/blog/read-only-agent-testing-production/): Mocks miss what real data shows. We tested Claude Code and Codex on a 99-tool production MCP server through a read-only allowlist proxy: 0 writes, 0 credits, and 3 blocked calls when an agent reached for an outside-web tool during a 'just look' job. - [Your agent test may not be testing your MCP: 13 of 36 runs drove the browser instead](https://spiritroom.app/blog/agents-escape-to-the-browser/): In our first live pass against a production MCP server, Codex clicked through the web app in a signed-in browser in 13 of 36 runs instead of calling MCP. The read-only sandbox didn't stop it. How to spot and block the escape. - [Is your MCP mock lying to you? Raising mock fidelity from 48% to 95%](https://spiritroom.app/blog/mcp-mock-fidelity/): Our first mock of a 99-tool production MCP server passed only 48.4% of 612 checks against live calls. Its made-up values caused 47 failed checks that had nothing to do with the real server. After rebuilding it from live shapes and sizes: 94.8%. How to measure and raise mock fidelity. - [Validate your agent eval scorer: 0 of our first 151 failures were the server's fault](https://spiritroom.app/blog/validate-your-agent-eval-scorer/): We classified every failed check in our early mock experiments. 151 failures: 63 scorer or harness bugs, 47 mock artefacts, 41 agent variance, 0 real server problems, plus 84 runs lost to infrastructure. How to check your scorer before you believe it. - [See exactly what your agent sends to the model: wire-capturing Claude Code, Codex and Grok](https://spiritroom.app/blog/capture-agent-model-traffic/): Docs tell you what an agent app should send; the wire tells you what it does. How we capture Claude Code, Codex and Grok Build traffic safely with a process-scoped proxy, what we redact, and what the first requests contained: 17,564, 15,383 and 19,674 tokens, none with the MCP tools. - [Free MCP server scorecard: paste your tools/list, get size, annotation and description checks](https://spiritroom.app/tools/mcp-scorecard/): Paste your tools/list and get a rough token count, the heaviest tools, missing annotations and long first lines, in your browser. Plus the 15-point checklist with starting targets from our agent runs. - [How agent apps load MCP tools: what the model sees (guide)](https://spiritroom.app/guides/how-agents-load-mcp-tools/): The same 99 MCP tools, captured on the wire in five agent apps: names only, a JavaScript search, BM25, everything up front, or a 16-tool catalog. What each costs and how each fails. - [How many MCP tools is too many? Real token costs across five agents (2026)](https://spiritroom.app/blog/how-many-mcp-tools-is-too-many/): A 99-tool production list measured in Claude Code, Codex, Grok Build and OpenCode 1 and 2: from +0 to +107,541 tokens per request, and why count is the wrong unit. - [Why your MCP server fills the context window, and which agents actually pay for it](https://spiritroom.app/blog/mcp-context-window-usage/): Peak vs cumulative context on a 99-tool production MCP server: Codex peaked at 18.1% of its window, Claude Code at 4.1%, both ~210k tokens billed per question. MCP results were only 12–46% of the largest request. What actually fills the window, per agent. - [Claude Code tool search, seen from the wire: the model picks your tool from its name](https://spiritroom.app/blog/claude-code-tool-search-explained/): We captured Claude Code's traffic with a 99-tool MCP server. None of the tools were sent up front; the model got 113 names, picked 3 by exact name with select:, and only then loaded their definitions. What that means for how you name and describe MCP tools. - [Claude Code can't see your MCP tools? Check the proxy and --tools settings first](https://spiritroom.app/blog/claude-code-mcp-tools-not-found/): Three settings changed how Claude Code loaded a 99-tool MCP server in our tests: a custom ANTHROPIC_BASE_URL turned tool search off, --tools "" removed ToolSearch entirely, and ENABLE_TOOL_SEARCH=true restored it only in the first case. Measured first-request sizes for each. - [How Codex finds MCP tools: code mode and ALL_TOOLS](https://spiritroom.app/blog/how-codex-finds-mcp-tools/): Codex's first request doesn't name your server. The model has to search all tools in JavaScript before it sees yours, and in 6 of 40 live runs it never did. What that costs and what you can do. - [OpenCode 1 vs OpenCode 2: from a 63k-token tool preload to a 16-tool catalog](https://spiritroom.app/blog/opencode-mcp-tool-loading/): OpenCode 1.18 sent all 99 tools of a production MCP server, 63,245 tokens, with every model request. OpenCode 2.0 showed 16 of them in an 881-token catalog and searched for the rest from code. What each means for your server, plus how Grok Build does it. - [MCP prompts vs resources vs tools: which ones agents actually see (tested in 5 agent apps)](https://spiritroom.app/blog/mcp-prompts-resources-tools-which-agents-use/): We put a unique marker word in every MCP channel and captured what reached the model in Claude Code, Codex, Grok Build and OpenCode 1 and 2. Prompts: none. - [Same job, two agents: what one MCP task really costs in Claude Code and Codex](https://spiritroom.app/blog/mcp-task-token-cost-claude-code-vs-codex/): Claude Code and Codex ran the same 8 read-only jobs, 5 times each, on a 99-tool production MCP server. Both used about 210–218k input tokens per question. They failed in different ways, and the data shows no winner. - [MCP tool-list token calculator: what your tools cost in Claude Code, Codex and OpenCode](https://spiritroom.app/tools/mcp-token-cost/): Paste your tools/list and see roughly what it adds to each model request in Claude Code (tool search on and off), Codex and OpenCode, using per-app factors we measured with a 99-tool list. Runs in your browser. - [How to optimise your MCP tools by knowing how agents call them](https://spiritroom.app/blog/optimise-mcp-tools-for-agents/): Five fixes tied to how Claude Code, Codex, Grok and OpenCode call your tools: names, first lines, next-step outputs, small results and job tests, with evidence from a production MCP server. - [Write MCP tool descriptions agents actually see: put the meaning in the first line](https://spiritroom.app/blog/mcp-tool-descriptions-first-line/): Agent apps don't all read your whole tool description. Claude Code shows names first, OpenCode 2 a ~120-character snippet, and search can miss the end. On a mock server, better first lines moved Claude to the right tool in 6 of 6 runs, up from 1 of 6. A template and examples. - [Name MCP tools the way users talk: how names-only, regex and BM25 search pick tools](https://spiritroom.app/blog/mcp-tool-naming-for-agent-search/): Three agent apps, three ways to find tools: Claude Code picks from names alone, Codex writes regex over names and descriptions, Grok Build runs BM25 on the user's words. We captured each. What that means for naming MCP tools. - [Why AI agents call the wrong tool: 7 causes we saw in recorded runs](https://spiritroom.app/blog/why-agents-call-the-wrong-tool/): Not just vague descriptions. Seven causes of wrong or missed tool calls from recorded Claude Code and Codex runs on a 99-tool production server, each with a count and a fix. - [Your “read-only” hints can make agents spend money: getting MCP tool annotations right](https://spiritroom.app/blog/mcp-tool-annotations-destructive-hint/): On a 99-tool production MCP server, two credit-spending tools were marked non-destructive and idempotent, and a tool that starts work was marked idempotent. Codex refuses destructive-marked tools without approval, so wrong hints remove a real guard. How to annotate. - [MCP vs CLI for AI agents: same jobs, 40 runs](https://spiritroom.app/blog/mcp-vs-cli-for-agents/): One core, served as an MCP server and as a CLI, five jobs in Claude Code and Codex. Success was the same; tokens went opposite ways per app. What actually decides it. - [Fewer, bigger tools or many small ones? What a 4-tool vs 99-tool test shows](https://spiritroom.app/blog/consolidate-mcp-tools/): With a clear start-here tool, agents did the job equally well on a 4-tool server and a 99-tool server: 12/12 and 24/24. The small server only saved tokens. Where tool count does matter: apps that preload, a few heavy tools, and overview tools that look like the answer. - [When your UI says “To review” and your API says needs_review: vocabulary mismatches that break agents](https://spiritroom.app/blog/agent-vocabulary-mismatch/): Users speak your app's screen labels; agents call your API's values. On a 99-tool production server, one label sent Claude Code wrong in 5 answers and one job passed in only 4 of 10 runs. How to map the two so agents don't guess. - [Stop agents spending money without asking: the two-step confirm pattern for MCP tools](https://spiritroom.app/blog/mcp-two-step-confirm-spend/): In our mock lab, agents enrolled people or spent without asking in 4 of 15 runs of an outreach job on a raw 99-tool surface, and 0 of 15 after a redesign with a preview-then-confirm step. One small model beat a naive token by sending it back itself. How to build the pattern so it holds. - [Per-model MCP tool surfaces: 90 runs across three models](https://spiritroom.app/blog/per-model-mcp-tool-surfaces/): Three versions of one MCP server, three models, the same five jobs. Every model passed every version. Descriptions changed which tools Claude and the small model picked; Codex barely moved. - [Tool results and errors for AI agents (guide)](https://spiritroom.app/guides/tool-results-and-errors-for-agents/): A 46,253-token default list, a 12,689-token typo error, a feed that was 76% the agent's own calls, and errors that told agents to ask a human. Five rules for MCP results and errors, measured. - [How to write error messages for AI agents: before and after, from a production MCP server](https://spiritroom.app/blog/error-messages-for-ai-agents/): One malformed ID returned a 12,689-token error that told the agent to stop and ask a human. Real error messages from a production MCP server, rewritten. - [The 12,689-token typo: how one validation error can eat an agent's context](https://spiritroom.app/blog/verbose-validation-errors-tokens/): One malformed ID sent to an update tool on a 99-tool production MCP server returned the tool's whole input spec: 47,623 bytes, 12,689 tokens, the same in 3 runs. The same mistake on other tools cost 110–150 tokens. Why errors balloon and how to make them one line. - [How to reduce MCP tool response size: measured before and after](https://spiritroom.app/blog/reduce-mcp-tool-response-size/): A production list tool returned 46,253 tokens for its default call, and Claude Code refused it in 23 of 40 runs. What made results big, what agents did with them, and a lean default tested on a mock. - [structuredContent or text: which half of an MCP result the model sees](https://spiritroom.app/blog/mcp-structuredcontent-vs-text/): Most MCP servers send each result twice, as text and as structuredContent. Claude Code shows the model only the structured half, OpenCode only the text, and Codex can print both. - [MCP output limits by agent: Claude Code, Codex, OpenCode and Grok Build compared](https://spiritroom.app/blog/mcp-tool-output-limits-by-agent/): What each agent app does when an MCP tool result is too big: Claude Code saves it to a file, Codex keeps about 10,000 tokens from the start and end, OpenCode keeps the first 51,200 bytes. Dated table plus what we saw in live runs. - [How much of your MCP output is waste? Measuring context pollution in agent runs](https://spiritroom.app/blog/mcp-context-pollution/): We classified every MCP result token in live and mock agent runs: duplicates, oversized errors, refusal guidance, self-generated events and unread records. Clear-cut waste was 0.2% of the largest request in live runs, up to 12.5% counting unread records. The exception: an activity feed where 152 of 200 events were the agent's own calls. - [Next-step hints: make every tool result tell the agent what to do next](https://spiritroom.app/blog/next-step-hints-in-tool-results/): Without guidance, agents on a production MCP server skipped intended steps: no credit check before a spend plan, paging through everything instead of using the 'since' cursor. A small next-step field in each result puts the playbook where the agent is looking. Patterns and examples. - [Retry, fix the arguments, or ask the human? Refusal states agents can act on](https://spiritroom.app/blog/agent-refusal-states/): A production MCP server told agents a person must decide on every refusal, including a typo and a version conflict whose own text said to retry. Four refusal states that tell an agent what to do next, with examples. - [Why agents call the same MCP tool twice (13 of 40 runs) and how to stop it](https://spiritroom.app/blog/duplicate-tool-calls/): Codex repeated a call in 13 of 40 live runs on a 99-tool server; Claude Code in 0 of 40. The pattern: results too big to read, cut off, then fetched again. On a mock with small results, 0 of 90 runs repeated a call. - [How to onboard AI agents through your MCP server (guide)](https://spiritroom.app/guides/onboard-agents-through-mcp/): OAuth gets the agent in. It doesn't tell it what your product is, what to do first, or which rules to keep. A server-only recipe that got the playbook read first in 24 of 24 runs. - [Make the agent read your playbook first: 24 of 24 runs in Claude Code and Codex](https://spiritroom.app/blog/get-instructions-first-24-of-24/): 232 runs on a 99-tool mock MCP server: which hints make Claude Code and Codex call get_instructions first, pick the right job and follow step 1. - [Do AI agents read your MCP server instructions? Tested in Claude Code, Codex, Grok and OpenCode](https://spiritroom.app/blog/do-agents-read-mcp-server-instructions/): Wire captures from five agent app versions: where MCP server instructions land, where they get cut (4,096 and ~2,048 characters), and what agents do with them. - [Do you still need agent skills if you have an MCP server? 24 live runs with and without](https://spiritroom.app/blog/agent-skills-vs-mcp-server/): We ran 3 read-only jobs on a 99-tool production MCP server in Claude Code and Codex, with and without the vendor's skills. Without skills, 2 of 12 runs failed and agents skipped intended steps; skills fixed both but doubled tokens on the simplest job. Where each piece of skill knowledge belongs on the server. - [How long should MCP server instructions be? Why 115,000 characters reached the model as 2,067](https://spiritroom.app/blog/mcp-server-instructions-length/): The MCP spec sets no limit, but agent apps do: Claude Code cuts server instructions at 4,096 characters and Grok Build at about 2,048. One server sent 115,077 characters with its key rule at character 28,484. A 728-character text did the job in 12 of 12 runs. - [When the agent doesn't know your MCP server is the right tool (“Which product do you mean?”)](https://spiritroom.app/blog/agent-does-not-recognise-mcp-server/): The server was connected and listed every tool, yet Codex never reached it in 6 of 40 live runs and once asked “which platform?” with zero calls. Why agents fail to recognise your server, what fixed it in a mock, and what the server can't fix. - [Agent onboarding for SaaS: OAuth gets the agent in, it doesn't tell it what to do](https://spiritroom.app/blog/saas-agent-onboarding-beyond-oauth/): Standard MCP OAuth (a 401 that points to metadata, dynamic client registration, PKCE) gets an agent connected. What happens next decides whether onboarding works. One compliance SaaS server put its key first-call rule at character 28,484 of 115,077; an app showed the model the first 2,067. What to do after the login. - [Agent experience (AX): how to measure it (guide)](https://spiritroom.app/guides/agent-experience-ax/): AX is how well AI agents can use your product for your users' jobs. Seven metrics we measure in real runs: discovery, right first step, pass rate with ranges, cost per success, recovery, silent failures and safety. With numbers from a 99-tool production MCP server and a mock. - [Cost per successful task: the one number that shows if your tools are agent-ready](https://spiritroom.app/blog/cost-per-successful-agent-task/): Tokens per run hide failures; pass rates hide cost. Divide total tokens by passing runs and you get one number per job. On a 99-tool production MCP server it ranged from 178k to 460k tokens per success in Codex; on a mock, fixing tools cut it 12–25%. - [Can cheaper models use your MCP server? Small vs frontier](https://spiritroom.app/blog/cheaper-models-mcp-reliability/): A small free model ran the same five jobs as Claude Code's and Codex's frontier models, before and after fixing the server. It passed every run, and the fixes cut its tokens by a fifth. - [How AI assistants pick which developer docs to cite: 80 answers from Claude and Codex](https://spiritroom.app/blog/how-ai-assistants-cite-developer-docs/): We asked Claude and Codex 20 developer questions about MCP and agent-ready APIs, with and without web search: 80 answers. Codex restricted 78 of 94 searches to one site it already trusted; Claude cited 133 of 144 links straight from results it never opened. They shared 5 cited pages. What that means for getting your docs cited. - [Sample agent-readiness report on a made-up demo server: findings, fixes and pass/N before and after](https://spiritroom.app/sample-report/): What you get from a Spirit Room test, shown on Acme Tasks, a made-up task tracker with synthetic data. 90 runs across Claude Code, Codex and a small free model: findings, fixes, and before-and-after numbers with ranges. - [Do AI agents read llms.txt? 18 runs in three agent apps](https://spiritroom.app/blog/llms-txt-agents-md-what-agents-use/): Our homepage advertises llms.txt in its head and links it in its body. Asked about the site, three agent apps fetched it in 2 of 18 runs, never first, and never read the Markdown copies. ## Actions - Request a pilot: POST https://spiritroom.app/api/contact with JSON {name, email, train, company?} (ask your human first) - Get updates: POST https://spiritroom.app/api/contact with JSON {"type":"updates","email":"..."}