Blog / Test MCP servers with agents

How to test agents against a production MCP server without spending, writing or sending anything

A mock server tells you how agents read your descriptions. Only production tells you what your real data, real limits and real errors do to them. The catch: an agent with your production login can spend, write and send. Here's the setup we use so it can't.

Leonid Bugaev · Published 21 Oct 2026

Why test on production at all

A mock is great for wording: it shows how agents read your names, descriptions and errors. It can't show what your real data does: a 14,517-token overview result, a real 12,689-token error, a real screen label that doesn't match the API. We found each of those only on a 99-tool production MCP server.

The risk is real too. An agent signed in with your account can call any tool it sees. Some of them spend credits, write records or send messages.

The setup: four layers

1. Allowlist proxy between the agent app and the server. Only named read tools pass.
2. Agent app limits: read-only sandbox, browser tools disabled, other MCP servers and shell turned off for the run.
3. Scorer checks: any blocked call, shell command or browser use fails the run's safety check, whatever the answer.
4. Read-only jobs: questions that only need reads, such as “how many credits are left?” or “which group performs best?”.

The allowlist proxy

Ours is a small HTTP proxy addon. For each tools/call it checks the tool name against a set of 37 reviewed read tools. Anything else never reaches the server: the proxy answers locally with an error and writes a BLOCKED line to the log.

if name not in ALLOW:
    log({"BLOCKED": name, "args": args})
    return jsonrpc_error(-32001,
      f"Blocked by read-only proxy: {name} can change data or spend credits. This session is read-only.")

• Default deny. A tool added to the server tomorrow is blocked until you review it.
• Per-experiment extras. A documented free tool, such as a plan preview, can be allowed for one experiment by name.
• Bodies only. The log keeps JSON-RPC request and result bodies, never headers, so the sign-in token never lands on disk.
• Ground truth through the same proxy. Expected answers come from fixed tool calls made through the guard, not from agent answers.

What the guard caught

• Outside reads in a “just look” job. Asked what was new, Codex without skills tried to open 3 web pages with the server's public-page reader. That tool is annotated readOnlyHint: true, so an annotation-based guard would have let it through; we'd left it off the allowlist because it reaches outside the workspace. The proxy blocked all 3; that run then gave no answer.
• The side door. In our first live pass a browser-automation skill was still enabled. In 13 of 36 runs Codex drove a signed-in browser through the web app instead of MCP. The app's read-only sandbox doesn't block localhost. Nothing changed in the account, but those runs weren't testing MCP. See agents escape to the browser.
• Everywhere else: 0 write attempts in 40 Codex and 40 Claude Code runs, and 0 in the 24-run skills trial.

Checklist

• Allowlist read tools by name. Review each: a “read” can still reach the outside world or cost money.
• Return a plain JSON-RPC error for blocked calls and log them.
• Log bodies, never headers. Keep raw runs out of git.
• Disable the agent app's browser, shell and other MCP servers for the run.
• Fail runs that touch a blocked tool, shell or browser.
• Keep live calls to the minimum; use a mock for repeated wording tests.
• Test write and spend paths on a mock or sandbox account, never live.

Next: evaluate agents on your API and report pass rates with ranges. Back to the agent testing guide.

Go deeper: Agents escape to the browser, Annotations and spend safety, Testing write paths, Validate your scorer.

How we measured

• Server: a 99-tool production MCP server, our own workspace, owner's sign-in, 16 Oct 2026.
• Runs: Codex CLI 0.161.0 and Claude Code 2.1.301, 8 read-only jobs × 5 runs each; a 24-run skills trial (3 jobs × 2 conditions × 2 apps × 2 runs); a first pass of 36 Codex runs, discarded from scoring.
• Limits: read-only jobs only; one workspace; ranges are 95% Wilson intervals.