Blog / How to optimise your MCP tools by knowing how agents call them

MCP server or CLI for AI agents? We ran the same jobs through both

Everyone has an opinion on MCP versus CLI, and most of it is about tokens. Before you build one or the other, you want to know if agents get the job done better with either. So we built one tool core, served it both ways, and ran the same five questions in Claude Code and Codex.

Leonid Bugaev · Published 21 Oct 2026

The setup

We wanted a fair fight, so both interfaces share one core: a mock task tracker, “Acme Tasks”, with synthetic data (4 projects, 6 people, 36 tasks, 76 activity events) and 21 tools. The MCP server exposes them as tools. The CLI exposes the same 21 as commands, with the same descriptions as help text and the same JSON output. Write commands exist in both but change nothing.

Five questions, in users' words, each with one right answer:

• How many tasks in our Mobile app project are waiting on someone right now?
• Which of our projects has the most tasks ready for review, and how many?
• Who on the team has the most open tasks, and how many?
• Which Billing migration tasks were closed since October 14?
• Who is working on the payment webhook retry task, and what's its status?

Each ran twice per setup: 2 apps × 2 interfaces × 5 jobs × 2 runs = 40 runs. In MCP mode the app could use only the server; in CLI mode only a shell restricted to the acme command.

Results

With 10 of 10 everywhere, the ranges overlap completely: there is no evidence that either interface gets the job done more often. Ten runs per setup can't rule out a small difference, but it does rule out the big one people argue about. The median number of server calls was 2 in every setup (2.5 for Claude Code's CLI runs), and median time was within 2 seconds.

Why the token difference flips

Through MCP, the app gives the model tool definitions. Through a CLI, the model reads help text instead, as command output, and needs a few extra turns to do it. Which costs more depends on how the app handles each:

• Claude Code defers MCP tools behind tool search and loads only the ones it needs, so its MCP runs started small. Its CLI runs paid for --help output and more back-and-forth: a median of 3.5 shell commands per run.
• Codex was slightly cheaper through the CLI. We didn't isolate why, and the gap (−7%) sits well inside its run-to-run spread of 54k–100k tokens.

Treat both token gaps as small: the ranges overlap in both apps. Codex's numbers are bigger in both modes, so compare each app with itself, not with the other.

The bigger lever is the size of your tool list. This mock's 21 tools take 1,730 tokens. A 99-tool production server we measured takes 68,286 tokens, about 690 per tool. At that size, any app that loads every definition pays for it on every request, and a CLI's on-demand help looks much better. That's why published benchmarks against large servers report MCP costing many times more, while a right-sized tool list closes most of the gap. Public studies agree that the agent app matters more than the interface: see the controlled study across seven agent apps and Scale Labs' comparison.

Errors and discovery

All 20 CLI runs started the same way: acme --help. A command-level --help followed in 8 of 10 Claude Code runs and 4 of 10 Codex runs. Claude Code sometimes guessed first. Asked who works on a task, it tried acme get-item --id it015, got “item_id is not valid”, then read the help and used --item-id. That happened in 3 of 10 Claude Code CLI runs (4 errors), and every run recovered. Through MCP the model had the schema, so the same lookups had 0 errors.

Two practical lessons for CLI authors:

• Make --help as good as an MCP tool description: what the command is for, the exact flag names, an example.
• Make errors say the right flag, not just that something is missing. The recovery here worked because the error named the expected argument. More in error messages for AI agents.

Which to build

Whichever you pick, the work that moves results is the same: names and first lines in users' words, small default results, and errors that teach. That's the subject of how to optimise your MCP tools by knowing how agents call them.

Go deeper: How agent apps load MCP tools, How many MCP tools is too many.

How we measured

• Apps: Claude Code 2.1.304 with its default model and Codex CLI 0.161.0 with its default model, headless, 21 Oct 2026, about 20:50–21:00 (UTC+3). Each run was isolated from other MCP servers and settings.
• Interfaces: one Python core served as a stdio MCP server and as a CLI on the PATH, with the same tool descriptions and the same output. In CLI mode Claude Code could run only acme commands; Codex ran in its sandbox.
• Scoring: each answer checked in code against the known answer from the synthetic data, then failures read by hand. One run was overridden to a pass after review: it named the right two tasks and correctly explained why a third was excluded, which tripped the automatic check.
• Tokens: input tokens from each app's own usage report, cached tokens included. Tool list size in o200k_base.
• Limits: a mock with small results, read-only jobs, 2 runs per job, one model per app. Ranges are 95% Wilson intervals. This shows where differences are not, not small differences.