If you build with coding agents, you have probably been told to give them skills: markdown files the agent loads when a task matches, full of instructions for your stack, your API, your conventions. The advice is sound. The way most people follow it is not, and the failure does not show up unless you measure.
For most of this year the norm had two forms. The first was installing someone else's pack. Public skill repos collect stars at a rate normal libraries never see; Theo's audit of Matt Pocock's skills repo cites a claimed 200,000+ stars for what is entirely markdown. The second form was generating your own: point a model at your documentation, have it write a skill per section, and cache it so only changed sections regenerate. Both forms share one assumption. More coverage means a better agent.
The number that breaks the assumption
Nick Nisi, a DX engineer at WorkOS, gave a talk on what happened when his team did the second one properly. They auto-generated more than 10,000 lines of skills from the WorkOS docs, with sensible engineering around it. Evaluation runs took 68 minutes and kept failing. Then they measured one skill in isolation: 77% correct with it loaded, 97% correct without it. The skill was making a frontier model worse at the company's own product.
They rewrote by hand. What survived was 553 lines, all of them specific gotchas, the things the model gets wrong, not a restatement of the docs. Eval time dropped to 6 minutes and pass rates went up.
His line from the talk: "I was actively making it worse, and I only knew because I was measuring."
Why it happens
A frontier model has already read a great deal about popular frameworks and well-documented APIs. Where its base rate is already near perfect, a long skill has nothing to add, but it still costs attention. It pulls the model toward paths the docs mention and the task does not need. Comprehensive documentation turned into instructions reads to the model as a list of things to do, so it does them.
The same logic explains why a skill pack that works beautifully for its author can do nothing, or harm, in your repo. The author wrote it against the mistakes their model made on their stack. Your model on your stack makes different ones.
Where I stand
My read of the WorkOS result, and the one my own notes record, is narrower than the "delete your skills" headline the talk got. The rule is not "fewer skills." The rule is measure marginal lift per skill, and keep a skill only when the lift is positive. Skills earn their place where the base rate is low: your proprietary API, a framework released after the model's training cutoff, a convention in your codebase the model cannot guess. They lose their place where the model already does the job, and generating from docs is the fastest way to fill that second category.
Applied to your situation, this becomes a test you can run in an afternoon, before adopting anything:
- Pick five tasks your team actually gives the agent.
- Run each with the skill loaded and without it. Same model, same prompt.
- Score the outputs blind.
- Keep the skill only if it wins. If it ties, delete it, because a tie still costs context on every run.
Theo's closing advice points the same way from the other side: never install a pack you have not read, and test a text-only skill by pasting its markdown into your agent before installing it at all. Reading is the cheap filter. The with-and-without run is the honest one.
A skill is a claim that the model is worse without it. Treat it like any other claim and ask for the number.