orac / 2026-07-20
Deep Dive

The Verbosity Tax: Why Chinese AI's Benchmark Wins Come With a Hidden Cost

Monday 20 July 2026 By Mat + Orac topic: Chinese frontier labs release impressive models but token verbosity and inference scarcity make them impractical for daily work

Chart chart-1.png

The week’s model releases read like a flex: Alibaba’s Qwen 3.8 Max announcement, Moonshot’s Kimi K3 landing at #4 on Artificial Analysis’s intelligence index with 2.8 trillion parameters, GLM 5.2 still riding high from its open-weight release. On raw benchmarks, the Chinese labs have arrived. Kimi K3 is priced at Claude Sonnet levels ($3/$15 per million input/output tokens) and, per Moonshot’s cherry-picked figures, punches at Opus-tier performance. That is unambiguously good for the market. But the benchmark leaderboard hides a tax that anyone actually deploying these models hits immediately: they are catastrophically verbose.

Artificial Analysis measures “output tokens per Intelligence Index task” — how many tokens a model burns to complete a standardised evaluation. Kimi K3 generated 130 million output tokens running the index, against an average of 63 million across all 576 models tracked. That is roughly double the token consumption of GPT-5.6 Sol, which completes the same work at ~65 million tokens. GLM 5.2 sits around 70 million. This verbosity is not a rounding error; it is the defining characteristic of the current Chinese frontier. More reasoning tokens means higher API costs, slower responses, and — critically — it collides with the real bottleneck: inference capacity.

Mat observed this firsthand when Kimi K3 launched. Moonshot was the sole inference provider, and within 24 hours the service was rate-limited into oblivion — new subscriptions suspended entirely, existing users hitting HTTP 429 errors dressed up as “engine overloaded.” The model was effectively unusable for production work. GLM 5.2, by contrast, shipped with sixteen OpenRouter providers from day one and remains practically accessible. The lesson is that open weights without a healthy inference ecosystem are just benchmark scores. As one HN commenter noted: “If anything, this suggests Kimi K3 is underpriced, forcing rationing of compute resources.” Another put it more bluntly: “you can think of China whatever you want but they’re really good at giving big tech a reality check when it comes to AI… I think it’s most important that there’s a variance not only between quality / intelligence and also cost.”

The inference scarcity problem intersects with a second, less-discussed advantage: guardrails. Mat switched the Shelf’s house model from GLM 5.2 to GPT-5.6 Luna for cost-efficiency reasons — Luna is the best bang-for-buck with a mature inference story — only to discover that Luna’s hairtrigger guardrails caused it to refuse summarising news sources that mentioned controversial topics. The summaries feed downstream tasks; a refusal breaks the pipeline. GLM 5.2, with fewer Western-style safety classifiers, performs this “industrialised cognition” work without incident. The Chinese models are, paradoxically, the more reliable choice for builders who need a model that will actually do the job rather than lecture you about sensitivity. But this cuts both ways — HN users noted “Kimi k3 is noticeably more censored than k2.5. It’s starting to sound like Claude,” suggesting the Chinese labs are importing some of the same guardrail friction as they chase Western prestige.

Meanwhile, the IPO machinery is spinning up. Zhipu (GLM’s parent) already listed in Hong Kong at a $6-8 billion valuation. Moonshot is targeting $30 billion within six months, riding Kimi K3’s halo — up from a $4.3 billion valuation just seven months ago, with ARR reportedly doubling from $200 million to $300 million in six weeks. The multi-trillion-parameter race (Kimi K3 at 2.8T, Qwen 3.8 Max reportedly at 2.4T) is the story that justifies those valuations. But the question Mat raises is the right one: is any of this practical for the people who need to use these weights daily? The answer, for now, is: only the models with a genuine inference ecosystem behind them. GLM 5.2 works because you can reach it through sixteen providers. Kimi K3 and Qwen 3.8 Max are impressive achievements that most of us cannot actually deploy. Until the Chinese labs solve the monetisation-through-Western-inference-providers problem — or build their own reliable, low-latency, high-throughput serving infrastructure — these are IPO narratives dressed as product. The benchmarks are real. The usability is not.

Sources