Reasoning effort
Performance or autonomy?
A little while back Anthropic put out a tutorial, Choosing the right effort level in Claude Code. I’ve been chewing on it ever since. Effort level high, you might say.
My TL;DR: “effort” is a budget, and what the budget really buys is initiative. That matters a lot more to an engineer than to a vibe coder.
Most of us assume a higher setting just means the model thinks harder, makes better decisions, and that’s why benchmark scores go up. I want to look at it as an engineer, because I think that points to a very different way of using it.
Here’s the clue. Pick an effort level in Claude Code and you get a description:
- Low: “Quick, straightforward implementation with minimal overhead”
- Medium: “Balanced approach with standard implementation and testing”
- High: “Comprehensive implementation with extensive testing and documentation”
Low means no testing or docs. Medium adds testing. High adds more testing plus documentation. That reads like each level comes with a set of instructions telling Claude what to do, and that’s a set of instructions I’d quite like to see.
So I went looking. There aren’t any. I checked the Codex source and the (leaked) Claude Code source. Things like ultracode and plan mode are genuine instruction injections in Claude Code, but the regular effort levels are just passed through to the API. Anthropic’s docs confirm the effort value gets rendered into the prompt server-side, and thanks to a great repo of extracted system prompts, we know what it looks like. For Claude Code, each level maps to a number, except max, which is just “max”:
- low: 5
- medium: 10
- high: 15
- xhigh: 40
- max: max
So it’s a budget, not a script. The model has learned what an implementation looks like at each budget, and the labels describe what that training produces (we assume). Note the jump at xhigh: steps of five, then nearly triple. Now you know why the labs love log scales on their cost charts. The top end is diminishing returns writ large.
I find this unsatisfying. I’d never tell an engineer to work on something for two hours on the assumption that’s enough time for some tests and docs. But that’s the deal here: the model gets a budget, scales everything to fit, and decides for itself what makes the cut. That’s where the initiative comes from. Anthropic’s own tutorial more or less admits it: set the effort too high and you’ll see the model overthinking and taking on more tasks than you gave it.
I’d rather be clear about the basic properties of a solution up front. Will there be tests, and what sort? Will there be docs, and for whom?
In my flavour of engineering, that gets settled in a planning process (obligatory Bower plug), so implementation is a discovery loop, not a place where architectural decisions get made. In that process I’m happy to tick off standard component shapes and trust an LLM to build and test them. So there’s a level of autonomy I want, and a level I don’t.
Rather than handing over a budget and hoping, I describe the non-negotiables up front, set the effort low-ish (Opus 5.5 at medium is comfortable overkill), and expect the model to check in with me about the form of what gets built (herdr ftw!).
The uncomfortable reality is that a well-specified feature is still more likely to come back properly finished at high or xhigh than at medium. But “more likely” isn’t good enough. At any level, I don’t trust Claude on its own to write tests good enough to prove its own implementation complete, and a bug found later costs more than the effort it takes to catch it now. The fix isn’t more slider. It’s more process.
So I’ve landed on a process that’s partly just the old way: good old-fashioned testing. For each feature, Claude sets up a fixture and I walk through it myself as a manual acceptance test. It works surprisingly well. Stuff still sneaks through, so I also run a review skill at the module level.
A quick aside on cross-model review, since there’s a lot of chatter about it. Adding Codex support to Bower means I can run a review in GPT Sol or even Astra and have the findings land where follow-up agents can pick them up. But I haven’t seen a real difference from using another lab’s model, at least nothing I couldn’t get from a bigger model or higher effort from the same lab. Reviews do benefit from larger models, but mostly they work because reviewing is a different job from writing. Neither LLMs nor humans are equally good at every kind of task, and I suspect it’s for the same reason: we get good at what we’re trained to do.
To wrap up: higher benchmark scores at higher effort aren’t a direct proxy for better engineering. Effort buys initiative, and for an experienced engineer, a good process with a Mk I meat brain in the loop keeps things on track better. A model coming back to run something past you is a feature, not something to throw away by shoving the slider to the right.