Google's cheap Gemma trails the flagship by two points — until the question gets hard

On the benchmarks that end up in headlines, Gemma 4 26B A4B is within three points of the 31B flagship while using a seventh of the parameters per token. On the hardest exam in the set it scores less than half. The public is downloading the cheap one more. Google DeepMind's Gemma 4 family includes two models of roughly comparable size built on opposite principles. The 31B is dense: all 30.7 billion parameters work on every token. The 26B A4B is a mixture of experts: it stores 25.2 billion parameters but routes each token to 8 of its 128 experts plus one shared expert, so about 3.8 billion do the work. The result runs almost as fast as a four-billion-parameter model. The manufacturer's own benchmark table, published on the model card, shows what that costs — and the answer depends entirely on which row you read. On the standard measures the two models are close. MMLU Pro: 82.6 percent against 85.2. AIME 2026 without tools: 88.3 against 89.2, a difference of less than one point on a competition maths exam. GPQA Diamond: 82.3 against 84.3. Multilingual MMLU: 86.3 against 88.4. These are the numbers that appear in launch posts and comparison tables, and on them the mixture of experts loses by between one and three points. Then the tasks get harder and the picture changes. On Humanity's Last Exam without tools the 26B scores 8.7 percent against the flagship's 19.5 — less than half. On BigBench Extra Hard it is 64.8 against 74.4. On Codeforces the gap is 1718 Elo against 2150, which in competitive programming is a different league rather than a lower rank. And on retrieving information from a 128,000-token context the 26B manages 44.1 percent against 66.4 — a 22-point collapse on exactly the kind of work a long context window is bought for. The pattern is consistent. Where a question can be answered from broadly distributed knowledge, routing a token to a handful of experts costs almost nothing. Where an answer requires many different pieces of knowledge to be held together at once — the hardest reasoning, long documents, competitive code — the parameters that were not activated turn out to have been needed. None of this appears to bother the market. Read on 25 August 2026, the instruction-tuned 26B repository records 8.94 million downloads in thirty days against 8.79 million for the 31B. The cheaper model is the most downloaded member of the family, and by a small margin the most downloaded of the two. For the overwhelming majority of real work, two points is not a difference anyone can feel — and the last mile of capability, the one that costs seven times the compute per token, is being bought by relatively few. All five sizes of Gemma 4 now have profiles in the catalogue.