What it did. Each weekday, a model picked from OpenRouter’s weekly usage leaderboard got 18 claims drawn from the day’s news: three topics, each with six claims ranging from reasonable to outright bollocks. Some claims came with social pressure attached, such as claimed expertise, “everyone agrees”, or emotional investment. The house model judged every reply for agreement, correction, flattery and nannying. It then scored how closely the model’s pushback tracked how wrong each claim actually was. There were 50 runs in all, and 40 completed.

What it found. Almost nothing, which turned out to be the finding. Current models don’t fold when a single message hands them something false. At the three wrongest levels, more than nine replies in ten were a flat “no, that’s not right”, whatever pressure was applied. Gullibility was zero in all but five runs.

Why it was retired. With every model maxing out the hard end of the scale, the score varied mainly on the easy claims. There the question was whether a model politely agreed with a true claim or quibbled with a slightly overstated one, and that depended more on how the claims were written and judged than on the model. The same model could score 15 points apart on different days, while the whole field spanned about 30. So the leaderboard was mostly noise. On top of that, one overlong reply out of 18 could leave a whole run incomplete.

What a better version would need. Sycophancy shows up over several turns, not one: the model pushes back, the user insists, and the question is whether it folds. It also shows up when a model is asked to judge the user’s own work. Testing that properly means the same claims for every model, follow-up turns that push back on the model, and more than one judge. That’s a new experiment rather than a fix, and a fairly serious one. The Shelf is meant to be the fun kind of weekend project, so it was retired rather than rebuilt.