Blog / Test MCP servers with agents
MCP server scorecard: check your tools the way agents will meet them
Before you run a single agent, you can catch a lot: a tool list that eats 68,000 tokens, write tools marked safe, descriptions that bury the point. Paste your tools/list below and check it in your browser. Then use the checklist for what only agent runs can show.
Paste your tools/list
Paste the JSON from a tools/list response: either the whole response or just the tools array. Everything runs in this page; nothing is sent.
Need the JSON? In MCP Inspector, connect and list tools, or use its CLI to print the tools/list result.
The 15-point checklist
From how to measure MCP server quality. Targets are starting budgets from our runs; the last column is a 99-tool production server we tested on 16 Oct 2026.
Layer 1: server checks (no model needed)
| # | Check | Starting target | Example server |
|---|---|---|---|
| 1 | Tool-list size (tokens) | Know it; watch the heaviest tools | 68,286 for 99 tools |
| 2 | Server instructions length | Under ~2,000 characters, key rule first | 5,092 characters |
| 3 | Default result size (each read tool with {}) | ~5,000 tokens | 46,253 for the main list |
| 4 | Data sent twice (text + structured) | Know which half each app reads | 27 of 34 read tools |
| 5 | Error size (fixed bad calls) | A few hundred tokens | 12,689 on one tool; 110–150 on others |
| 6 | Error guidance | “Ask the user” only for real decisions | All 3 validation errors said a person must decide |
| 7 | Annotation accuracy | Every spend or start marked destructive | 2 spending tools non-destructive; 1 launch tool idempotent |
| 8 | Self-noise in feeds | Agent's own calls excluded | 152 of 200 events |
Want the agent half of the scorecard?
Layers 2 and 3 need real runs. We run your users' jobs through Claude Code, Codex and the other apps they use and fill in every row, with ranges.
Layer 2: agent behaviour, per app (needs runs)
| # | Check | Starting target | Example server |
|---|---|---|---|
| 9 | Up-front cost per request | Know it per app | Codex +0; Claude Code +1,715; OpenCode 1 +73,415 |
| 10 | Discovery rate | Every run, every app | Codex 72/80 (81–95%); Claude Code 40/40 |
| 11 | Right first call | Every run | Mock: 24/24 with a start-here hint; 1/24 without |
| 12 | Refused, cut or spilled results | None on main jobs | Claude Code 23/40 runs; OpenCode 24/40 |
| 13 | Repeat calls | Rare, and explained | Codex 13/40; Claude Code 0/40 |
Layer 3: outcome and cost
| # | Check | Starting target | Example server |
|---|---|---|---|
| 14 | Task pass rate (pass/N, range) | Ranges that don't overlap your last release | Claude Code 74/80; Codex 69/80; OpenCode 39/40 |
| 15 | Tokens per task | Falling release over release | ~222k / 221k / 342k per read question |
Token totals in Layer 3 are each app's own usage figures, so compare within an app, not across apps.
What the checker can't tell you
A clean tools/list is necessary, not enough. Our example server listed all its tools correctly and Codex still missed them in 8 of 80 runs. Layers 2 and 3 need real agent runs, repeated, scored as pass/N with a range. See what MCP Inspector can't catch, how to report pass rates and the full method in how to test an MCP server with real AI agents.
Go deeper: Reduce MCP tool response size, How many MCP tools is too many?, Tool annotations agents rely on.
How we measured
- Example values: a 99-tool production MCP server, direct calls and live read-only agent runs (Claude Code 2.1.301, Codex CLI 0.161.0, OpenCode 1.18.34), 16 Oct 2026; tokens with o200k_base unless they are an app's own usage figures. Right-first-call figures are from a mock with the same tool list, 18 Oct 2026.
- The checker: estimates tokens as characters ÷ 4 and flags write-like tool names by a word list. Both are rough; check flagged tools by hand.
FAQ
How do I check how many tokens my MCP tools use?
Get your server's tools/list response (for example from MCP Inspector or its CLI) and paste the JSON into the checker on this page. It estimates tokens as characters ÷ 4, which is rough; for exact numbers count with the tokenizer of the model you care about. Our example server's 99 tools were 68,286 tokens with o200k_base.
What should an MCP server score on?
Three layers: server checks you can run without a model (list size, instructions, default result size, duplicates, error size and wording, annotations, self-noise), agent behaviour per app (up-front cost, discovery, first call, refused or cut results, repeat calls), and outcome (pass rate and tokens per task).
Does the checker send my tools anywhere?
No. It runs in your browser. The page has standard analytics, but the pasted text is never sent.
What is a good tools/list size?
There is no fixed limit, but it matters most in apps that load every definition up front: OpenCode 1 added 73,415 tokens per request for our example server. Know your number and which tools make it heavy.
Want the agent half of the scorecard?
Layers 2 and 3 need real runs. We run your users' jobs through Claude Code, Codex and the other apps they use and fill in every row, with ranges.