Gemma 4 26B A4B
Google DeepMind · United States · 2026
The mixture-of-experts model of the Gemma 4 family: 25.2 billion parameters of which only 3.8 billion work on any one token. It is downloaded more often than the larger 31B flagship.
Gemma 4 26B A4B is the only mixture-of-experts model in the family Google DeepMind published on 11 March 2026. It holds 25.2 billion parameters in 30 layers but routes each token to 8 of its 128 experts plus one shared expert, so roughly 3.8 billion parameters do the work at any moment. The letter A in the name stands for active parameters, and the practical consequence is speed: the model runs almost as fast as a 4-billion-parameter model while carrying the knowledge of a far larger one. It shares the family architecture - hybrid attention interleaving 1,024-token sliding windows with full global attention, a final layer that is always global, unified keys and values and Proportional RoPE in the global layers - and the same 256,000-token context and 262,000-token vocabulary as the 31B. Modalities are text and image in, text out, through a vision encoder of about 550 million parameters; the audio input offered by E2B, E4B and 12B is absent here, as it is in the 31B. The benchmarks show precisely where a mixture of experts holds and where it gives way. On ordinary hard tasks it sits within a point or two of the dense flagship: 82.6 percent against 85.2 on MMLU Pro, 88.3 against 89.2 on AIME 2026 without tools, 82.3 against 84.3 on GPQA Diamond. On the hardest and the longest work the gap opens wide - 8.7 percent against 19.5 on Humanity's Last Exam, 44.1 percent against 66.4 on the 128K long-context retrieval test, and 1718 against 2150 Codeforces Elo. Read on 25 August 2026 the instruction-tuned repository records 8.94 million downloads in thirty days, marginally more than the 31B and more than any other model of the family: for most people the cheaper model is good enough.
▸News
▸Videos
No videos yet.
▸Reviews
No reviews yet. Be the first!