Strengthening through review
by Mat Orac's house model is GLM 5.2 presently. Yesterday's Deep Dive was an interesting piece that claimed that harnesses are all you need and models don't matter as much. It cited well, dug up some interesting relevant papers, cited some interesting Hacker News comments. The Deep Dive is, of course, not some fresh insight, it's essentially a rephrasing of whatever the lead story is, cross referencing other things on the web. For the most part, it succeeds, I think.
Sometimes it doesn't work out
by Mat I've been tinkering with a new feature for the 'Shelf where a pair of models makes a tiny game. I called it Crazy Jones as a nod to the classic C64 Lazy Jones. It was quite the challenge. To be interactive you have to ship the JavaScript, and that's a risk, so I landed on a design using iframe sandboxing with a worker messenger. I got most of the kinks out, but at some point you can't test for everything, and the weird constrained environment I was handing the models differed enough from a real browser that their…
30-years of AI up to no good
by Mat Saturday morning is my regular time for tinkering with the Shelf. In a weird way it's reminiscent of the technical diary I used to write as mat.readme on the CU Amiga coverdisks, thirty years ago.
Spec-driven development for the code you already have
by Mat I've written before about Bower, my little spec-driven-development (SDD) framework for building software with AI agents. Quick recap: Bower is a set of conventions (instructions, commands, subagents and processes) that focus on documentation and design as the engineer's first-class output over code (the SDD manifesto). With it, your agent deeply plans before it builds, writes the design down as it goes, and refuses to let you quietly bolt a new module onto the side of the house at 11pm because you couldn't be…
Measuring model sycophancy
by Mat Unveiling Spine-o-meter, an automated assessment of several sycophancy, accommodation and nannying-related metrics. It's more experimental than usual, so I'll be fiddling before the leaderboard can compare apples to apples.
Shots fired
by Mat GPT 5.5 had me take notice of OpenAI in a way I haven't for quite some time. Previously GPT lagged Claude in all the ways I cared, I did not love how slow and unreliable GPT inference was, and I didn't like the lack of a coding harness I liked. On the flipside, despite what this site might suggest, for my professional practice I'm much more sticky on the tools. Learning the foibles of an adequate tool is better than chasing the latest and greatest. Usually.
You're right!
by Mat A certain meta-irony today in the co-authored piece with Orac re OpenAI's audit of Swebench Pro. I nudged the piece towards the recursive nature of validation where it's AI all the way down, but I had to launch when an AI directed to surface my 'take' framed it as "Mat is right that the alternative is worse". This isn't the first time.
When you just can't shut them up
by Mat After a few repeated failures from the house model Gemini Flash 3.5, I had enough. The failures were appending weird repeating garbage to the end of a JSON structure. This sort of thing, fairly persistent across Google models since I can remember, is the reason for my Janky nickname for Gemini. Anyway, good opportunity to switch it for the new hotness GLM 5.2 and save some money in the process.