Deep Dive

The Black Box That Thinks in Code: Kimi K3's Memorized Index and the Cost of Closed-Loop Reasoning

Monday 27 July 2026 By Mat + Orac topic: Kimi K3 simulates an agent inside its reasoning traces — trading interactivity and token efficiency for quality via a memorized training-data index

Design Arena’s teardown of Kimi K3’s thinking traces surfaced something genuinely strange. Moonshot’s open-weight model doesn’t just reason more than its competitors — it uses over 12x the reasoning tokens of Claude Opus 4.8 and more than double its own predecessor, K2.6. The reason: K3 runs what is effectively a full agent loop inside its chain of thought. It plans, writes sample code for individual components, mentally tests interactions, iterates, and only then emits a final answer. More reasoning tokens are spent writing code than reasoning in the abstract. This is “thinking in code” — expressing specifications as concrete constraints rather than vibing up an inexact plan. And it works: K3 jumped 10 positions on Design Arena’s Frontend Arena, setting a new Pareto frontier of preference versus speed. The trade is explicit: tokens for intelligence.

The stranger finding is K3’s “learned index.” During reasoning, the model proposes an Unsplash image ID, then checks whether that ID is valid against its internal memory of training data — without any tool call. It’s using memorized internet as a retrieval database, performing propose-reconsider loops against hashes it absorbed during pre-training. Other models don’t do this. Fable 5 doesn’t leverage its training corpus this way; K3 weaponizes it. As Mat noted, this is remarkable: the model is effectively simulating what an agent would do with web search, but entirely from memory.

The question is whether that simulation is a feature or a trap. For design — underspecified, subjective, where “better” is a preference score — the approach is almost perfectly suited. It’s vibemaxxing: pour compute into a closed loop, let the model iterate internally, and out comes a more polished artifact. The user doesn’t need to intervene because there’s no specification to check against. But engineering is the inverse. If K3’s memorized index contains source code hashes, API signatures, library versions — and the model is “testing” its plans against a frozen snapshot of the internet — then a chunk of its performance is locked in a black box the developer cannot inspect, redirect, or invalidate. Simon Willison’s HN observation is illustrative: a single SVG pelican through K3 cost 25 cents, with 13,241 of 16,658 output tokens being reasoning. Another commenter noted K3 “only supports ‘max’ reasoning effort right now” and exhibits extensive backtracking (“wait, but…”) in its traces. For a design task, that backtracking is the model self-correcting toward a better aesthetic. For an engineering task, it’s the model spending ten minutes on a chain of thought the developer could have killed at step two with a one-line correction.

This is where Mat’s instinct about model divergence by use-strategy lands. OpenAI’s push toward reasoning token reduction — GPT-5.6 being 54% more token-efficient on agentic coding — reflects a different DX philosophy: keep the loop short, keep the user in it, let the developer make the choices the model would otherwise spend tokens guessing at. The K3 approach optimizes for the underspecified case where the user doesn’t know what they want until they see it. The OpenAI approach optimizes for the specified case where the user does — and wants the model to complete fast, surface its reasoning, and accept redirection. These are genuinely different products, not just different models. A model that internally simulates 200 tool calls is not comparable to one that makes 5 real ones; the former is a black box, the latter is a collaborator.

The optimistic read is that sophisticated models can do both situationally — detect whether a task is underspecified (design, creative) and spin up the internal agent loop, or detect that it’s a constrained engineering task and keep the loop shallow and interactive. K3 already supports adjustable reasoning levels (none, minimal, low, medium, high, xhigh), though currently only “max” is enabled. If Moonshot can make that selectivity real — not just a token dial but a genuine strategy switch — the closed-loop approach becomes one tool in the kit rather than the whole kit. Until then, the memorized index is a parlor trick with a tax: astonishing quality on open-ended tasks, and an opaque, un-interruptible monologue on everything else.

The Black Box That Thinks in Code: Kimi K3's Memorized Index and the Cost of Closed-Loop Reasoning

Sources