Opus 5.5
A new GOAT in town
It has been mildly upsetting to me that Anthropic had been lagging behind OpenAI in a few ways that really matter. Primarily price/performance at sensible effort levels. Performance here crucially including speed, and price being more accurately, how much you can get done with a sensible subscription.
Opus 5.5 is a hell of a come back, particularly for engineering. It’s 20% cheaper, but a lot cheaper when it comes to cache reads. This is particularly useful for us engineers, because long contexts and tool calls is where we are living day to day. I did a little investigation on how much of the price was down to token volume (they said it uses less tokens in the PR), and how much to price (per token). It’s almost all price, but with the model performance being way up, they fiddled with the token budget for effort levels in interesting ways. The top max effort has stormed to the top of the benchmark charts, such as they are, and costs almost exactly the same as Opus 5 - that feels like a calibration.
Anthropic is now firmly in the same position as OAI having essentially made their smaller model obsolete. There is zero point using Sonnet 5. I could regurgitate some figures on benchmarks, but if you want to see why, just head over to artificial analysis and see where Sonnet 5 is compared to Opus 5 low-effort. I’m surprised they’ve kept Sonnet as the default in the web chat, then again, it is probably more expensive to serve Opus 5 and the pricing is just a reflection of them defending their turf.
Anyway, using Opus 5.5 is a joy. It’s like Fable, but faster. It’s faster than Opus 5 in my heavy SDD dev environment. It’s very pro-active so the vibe coding crowd will like it. It finds more stuff ‘along the way’, which is a helpful form of being pro-active only if it’s matched with good judgement. Opus 5 frequently showed questionable judgement in a particular way; recognising when something that is technically a bug or a nit is actually worth the effort to fix, and what an appropriate scale of fix effort is. So far, Opus 5.5 has avoided this problem but honestly, I’d feel more confident about that after using it for a couple of weeks.
My first spin with Opus 5.5 was to ask the model to take a look at Craitures. We had a little brainstorm. It pushed back against my idea of extending the planning and chatting, with models retrieving body plan examples on request, and propose a frankly genius-level idea of switching the joins from placements to rules and allowing recursion. It also figured out my docs on Craiture had lagged, noted my concerns that we needed to ensure what we did was not confusing for small models, but scaled to gucci models - so it surveyed a bunch of models based on my metadata that described some favorites in weight classes, and tweaked the strategy. This is literally stuff I would have directed it to. It’s remarkable to see initiative and judgement.
I might have put that down to luck but my experience of Opus 5.5 with Bower has been absolutely first rate also, with the initiative turning into proposals for action instead of racing ahead. That’s my happy place for AI-first engineering.
Accepting I have a bias towards Claude generally, I don’t wax lyrical about particular models very often. Opus 5.5 is special sauce. Well done Anthropic, very well done indeed.
P.S. I feel a bit bad for xiaomi/mimo-v2.6-pro with this timing. After having a play in opencode it seems like cracking model with an outstanding price/performance.