---
title: MCP vs CLI for AI Agents: Same Jobs, 40 Runs
description: Should you build an MCP server or a CLI for AI agents? Same jobs in Claude Code and Codex: 10/10 each way. Tokens and errors differed by app, not interface.
canonical: https://spiritroom.app/blog/mcp-vs-cli-for-agents/
published: 2026-10-21
updated: 2026-10-21
---

> Markdown version of https://spiritroom.app/blog/mcp-vs-cli-for-agents/ for agents. Index: /llms.txt

[Blog](/blog/) / [How to optimise your MCP tools by knowing how agents call them](/blog/optimise-mcp-tools-for-agents/)

# MCP server or CLI for AI agents? We ran the same jobs through both

Everyone has an opinion on MCP versus CLI, and most of it is about tokens. Before you build one or the other, you want to know if agents get the job done better with either. So we built one tool core, served it both ways, and ran the same five questions in Claude Code and Codex.

Leonid Bugaev · Published 21 Oct 2026

**TL;DR**

- **Success was the same.** All four setups passed 10 of 10 (95% range 72–100%): Claude Code and Codex, each through MCP and through a CLI, on five questions with known answers.
- **Tokens went opposite ways, and not by much.** Claude Code used a median 25,700 input tokens via the CLI against 19,837 via MCP (+30%). Codex used 72,781 via the CLI against 78,390 via MCP (−7%). The run-to-run ranges overlap in both apps.
- **The CLI costs discovery.** All 20 CLI runs started with `acme --help`. Claude Code guessed a flag name and hit argument errors in 3 of 10 CLI runs; through MCP, with a schema, it had none.
- **What decides the cost is your tool list, not the protocol.** This server's 21 tools cost 1,730 tokens. A 99-tool production server we measured costs 68,286. Big lists are where MCP gets expensive.
- **Build both from one core** if your users run both coding agents and chat apps; keep the MCP tool list small and the CLI's help text as good as your tool descriptions.
[The setup](#setup)[Results](#results)[Why the token difference flips](#why)[Errors and discovery](#errors)[Which to build](#choose)[How we measured](#how-we-measured)[FAQ](#faq)

## The setup

We wanted a fair fight, so both interfaces share one core: a mock task tracker, “Acme Tasks”, with synthetic data (4 projects, 6 people, 36 tasks, 76 activity events) and 21 tools. The MCP server exposes them as tools. The CLI exposes the same 21 as commands, with the same descriptions as help text and the same JSON output. Write commands exist in both but change nothing.

Same data + 21 tools**→**MCP server**/**CLI: acme <command>**→**Claude Code, Codex**→**Answer vs known truth

Five questions, in users' words, each with one right answer:

- How many tasks in our Mobile app project are waiting on someone right now?
- Which of our projects has the most tasks ready for review, and how many?
- Who on the team has the most open tasks, and how many?
- Which Billing migration tasks were closed since October 14?
- Who is working on the payment webhook retry task, and what's its status?

Each ran twice per setup: 2 apps × 2 interfaces × 5 jobs × 2 runs = 40 runs. In MCP mode the app could use only the server; in CLI mode only a shell restricted to the `acme` command.

## Results

Same jobs, two interfaces, two apps (mock, 21 Oct 2026)

| Setup | Passed | Median input tokens | Range | Runs with argument errors |
| --- | --- | --- | --- | --- |
| Claude Code + MCP | 10/10 (72–100%) | 19,837 | 13,751–25,665 | 0/10 |
| Claude Code + CLI | 10/10 (72–100%) | 25,700 | 16,371–42,720 | 3/10 |
| Codex + MCP | 10/10 (72–100%) | 78,390 | 57,821–100,482 | 0/10 |
| Codex + CLI | 10/10 (72–100%) | 72,781 | 54,328–76,437 | 0/10 |

Claude Code + MCP*19,837*

Claude Code + CLI*25,700*

Codex + MCP*78,390*

Codex + CLI*72,781*

With 10 of 10 everywhere, the ranges overlap completely: there is no evidence that either interface gets the job done more often. Ten runs per setup can't rule out a small difference, but it does rule out the big one people argue about. The median number of server calls was 2 in every setup (2.5 for Claude Code's CLI runs), and median time was within 2 seconds.

**Find out whether your CLI or MCP server works for agents.**

We run your users' real jobs through Claude Code, Codex and the other apps they use, against your MCP server, your CLI or both, and report pass rates, tokens and failures with confidence ranges.

[Test your CLI or MCP →](/#contact)

## Why the token difference flips

Through MCP, the app gives the model tool definitions. Through a CLI, the model reads help text instead, as command output, and needs a few extra turns to do it. Which costs more depends on how the app handles each:

- **Claude Code** defers MCP tools behind tool search and loads only the ones it needs, so its MCP runs started small. Its CLI runs paid for `--help` output and more back-and-forth: a median of 3.5 shell commands per run.
- **Codex** was slightly cheaper through the CLI. We didn't isolate why, and the gap (−7%) sits well inside its run-to-run spread of 54k–100k tokens.

Treat both token gaps as small: the ranges overlap in both apps. Codex's numbers are bigger in both modes, so compare each app with itself, not with the other.

The bigger lever is the size of your tool list. This mock's 21 tools take 1,730 tokens. A 99-tool production server we measured takes 68,286 tokens, about 690 per tool. At that size, any app that loads every definition pays for it on every request, and a CLI's on-demand help looks much better. That's why published benchmarks against large servers report MCP costing many times more, while a [right-sized tool list](/blog/how-many-mcp-tools-is-too-many/) closes most of the gap. Public studies agree that the agent app matters more than the interface: see the [controlled study across seven agent apps](https://arxiv.org/abs/2608.08654) and [Scale Labs' comparison](https://labs.scale.com/blog/mcp-vs-cli).

## Errors and discovery

All 20 CLI runs started the same way: `acme --help`. A command-level `--help` followed in 8 of 10 Claude Code runs and 4 of 10 Codex runs. Claude Code sometimes guessed first. Asked who works on a task, it tried `acme get-item --id it015`, got “item\_id is not valid”, then read the help and used `--item-id`. That happened in 3 of 10 Claude Code CLI runs (4 errors), and every run recovered. Through MCP the model had the schema, so the same lookups had 0 errors.

Two practical lessons for CLI authors:

- Make `--help` as good as an MCP tool description: what the command is for, the exact flag names, an example.
- Make errors say the right flag, not just that something is missing. The recovery here worked because the error named the expected argument. More in [error messages for AI agents](/blog/error-messages-for-ai-agents/).

## Which to build

A rule of thumb from these runs

| If your users… | Build |
| --- | --- |
| Work in coding agents with a shell (Claude Code, Codex, OpenCode) | Either works for read jobs; a CLI is enough if its help is good |
| Use chat apps or assistants without a shell | MCP: a CLI isn't reachable |
| Use both | Both, generated from one core so names, descriptions and results match |
| Need a big API surface | Keep the MCP tool list small or deferred; a big list is what makes MCP expensive |

Whichever you pick, the work that moves results is the same: names and first lines in users' words, small default results, and errors that teach. That's the subject of [how to optimise your MCP tools by knowing how agents call them](/blog/optimise-mcp-tools-for-agents/).

**Go deeper:** [How Codex finds MCP tools](/blog/how-codex-finds-mcp-tools/), [MCP across Claude Code, Codex and OpenCode](/blog/cost-per-successful-agent-task/), [How many MCP tools is too many](/blog/how-many-mcp-tools-is-too-many/).

## How we measured

- **Apps:** Claude Code 2.1.304 with its default model and Codex CLI 0.161.0 with its default model, headless, 21 Oct 2026, about 20:50–21:00 (UTC+3). Each run was isolated from other MCP servers and settings.
- **Interfaces:** one Python core served as a stdio MCP server and as a CLI on the PATH, with the same tool descriptions and the same output. In CLI mode Claude Code could run only `acme` commands; Codex ran in its sandbox.
- **Scoring:** each answer checked in code against the known answer from the synthetic data, then failures read by hand. One run was overridden to a pass after review: it named the right two tasks and correctly explained why a third was excluded, which tripped the automatic check.
- **Tokens:** input tokens from each app's own usage report, cached tokens included. Tool list size in `o200k_base`.
- **Limits:** a mock with small results, read-only jobs, 2 runs per job, one model per app. Ranges are 95% Wilson intervals. This shows where differences are not, not small differences.

## FAQ

### Is MCP or CLI better for AI agents?

In our test neither was better at getting the job done: Claude Code and Codex passed 10 of 10 runs through both an MCP server and a CLI built from the same code. Differences were in cost and errors, and they depended on the agent app.

### Does MCP use more tokens than a CLI?

It depends on the tool list size and the app. With a 21-tool server (1,730 tokens of tool definitions), Claude Code used about 30% fewer tokens via MCP, and Codex about 7% more, both within run-to-run spread. Large tool lists push MCP's cost up because apps that don't defer tools send the definitions with every request.

### Why do agents make more errors with a CLI?

The CLI gives the agent no schema up front, so it reads --help or guesses flags. Claude Code guessed a flag name and got argument errors in 3 of 10 CLI runs, then fixed them after reading help. Through MCP, with a typed schema, it made none.

### Should I build an MCP server or a CLI?

If your users work in coding agents with a shell, a CLI works as well as MCP for read jobs. Chat apps and IDE assistants without a shell need MCP. If you serve both, generate both from one core so names, descriptions and results stay the same.

### Do agents use a CLI without being told?

In our runs the prompt named the CLI. Agents didn't need any other instructions: every CLI run began by reading acme --help and found the right commands from there.

**Find out whether your CLI or MCP server works for agents.**

We run your users' real jobs through Claude Code, Codex and the other apps they use, against your MCP server, your CLI or both, and report pass rates, tokens and failures with confidence ranges.

[Test your CLI or MCP →](/#contact)

## Keep reading

- Up[How to optimise your MCP tools by knowing how agents call them](/blog/optimise-mcp-tools-for-agents/)
- Related[How Codex finds MCP tools: code mode and ALL\_TOOLS](/blog/how-codex-finds-mcp-tools/)
- Related[Cost per successful task: the one number that shows if your tools are agent-ready](/blog/cost-per-successful-agent-task/)