Blog / Onboard agents through MCP
Do you still need agent skills if you have an MCP server? 24 live runs with and without
Many MCP vendors ship a skill or an AGENTS.md alongside the server, because the server alone doesn't teach the agent the workflow. We tested whether that's still needed, and where the knowledge should live instead.
The test
A 99-tool production MCP server whose vendor ships skills (a 27.8 KB main skill and three 5–7 KB playbooks). Three read-only jobs: plan a capped spend, review what's new since yesterday, and a summary question. Claude Code and Codex, with and without the skills, 2 runs each: 24 runs, through a read-only proxy.
Find out what your agents miss without skills.
We run your users' jobs with and without your skills and show which knowledge to move into instructions, descriptions and results.
Results
| Job | App | No skills | With skills |
|---|---|---|---|
| Plan a capped spend | Claude | Previewed 2/2, skipped the balance check | Followed the playbook 2/2 |
| Codex | 1 OK without the balance; 1 asked “which platform?”, 0 calls | Full playbook 2/2 | |
| What's new since yesterday | Claude | Right 2/2, but paged through everything | Used the “since” tool 2/2 |
| Codex | 1 tried an outside-web tool 3× (blocked), no answer; 1 OK | Used the “since” tool 2/2 | |
| Summary question | Claude | Right 2/2 | Right 2/2, 2× tokens |
| Codex | Right 2/2, 5 calls | Right 2/2, 5 calls |
The “which platform?” failure is the recognition problem in when the agent doesn't recognise your server.
What skills cost
- On the summary question, the skill added 5–8 grep steps and doubled Claude's tokens for the same answer.
- On the spend job, skills cut Claude's calls from 9–11 to 7 (~214k → ~198k tokens), but raised Codex's to ~270k.
- On the “what's new” job, Claude used the intended tool with skills but made more calls (17–19 vs 11–13) at similar tokens.
Skills help on multi-step jobs and cost on simple ones.
Where skill knowledge belongs
| Knowledge | Server-side home |
|---|---|
| What the product is; “use me for X questions” | First line of the server instructions |
| Global rules: confirm every spend, words to use | Instructions, under ~2,000 characters |
| Playbooks per job | A get_instructions(job) tool |
| Field meanings | The description of the tool that returns them |
| What to do next | Next-step hints in results: next-step hints |
| Wrong paths | Errors and refusals that name the right path |
How to make agents read it first: make agents read your instructions first. Length limits: server instructions length. Back to the onboarding guide.
Go deeper: Agent skills vs MCP for docs, Put the meaning in the first line, Test against production read-only.
How we measured
- Live: Claude Code 2.1.301 and Codex 0.161, 3 jobs × 2 conditions × 2 runs = 24 runs, 16 Oct 2026, read-only proxy, 0 writes. Skills installed per run in a temp folder.
- Scoring: transcripts read by hand against the jobs' intended steps.
- Limits: 2 runs per cell shows direction, not rates. The server-side recipe was verified on a stub, not live. Ranges are 95% Wilson intervals.
FAQ
Do I need Claude skills or AGENTS.md if I have an MCP server?
Not if the server carries the knowledge itself. In our live trial, skills fixed failures and skipped steps, but every piece of skill knowledge had a server-side home: instructions, a get_instructions tool, tool descriptions, next-step hints and errors.
Why not just ship a skill?
Users have to install it, apps load it differently, and it costs tokens: on the simplest job in our trial it added 5–8 search steps and doubled Claude's tokens. Server-side knowledge reaches every app.
Are MCP prompts a replacement for skills?
Not for teaching the model. In all five apps we checked, MCP prompts weren't visible to the model.
Find out what your agents miss without skills.
We run your users' jobs with and without your skills and show which knowledge to move into instructions, descriptions and results.