Gemma 4 12B Unified

Gemma 4 12B Unified arrived on 23 May 2026, ten weeks after the rest of the family, and it is the architectural experiment of the release. The word Unified means encoder-free: where every other Gemma 4 runs images through a dedicated vision encoder and sound through an audio encoder before handing the result to the language model, this one projects raw image patches and raw audio waveforms directly into the decoder's embedding space through lightweight linear layers. Everything then flows through a single decoder-only transformer of 11.95 billion parameters in 48 layers. The point of the design is practical. Dropping the encoders shortens the path from input to answer, which lowers multimodal latency, and it lets the whole model be fine-tuned in one pass rather than in pieces - a real advantage for anyone adapting the model to their own images or recordings. The model keeps the family's hybrid attention with 1,024-token sliding windows, the 262,000-token vocabulary and the full 256,000-token context. It occupies a spot nothing else in the family fills. E2B and E4B also take audio, but they stop at a 128,000-token context and are built for phones; the larger 26B A4B and 31B reach 256,000 tokens but do not accept sound at all. Only this model does both. In exchange it gives up some raw ability: 77.2 percent on MMLU Pro and 77.5 on AIME 2026 without tools, against 85.2 and 89.2 for the 31B, and 43.4 percent on long-context retrieval at 128K against 66.4. On speech recognition it is the strongest of the three audio-capable Gemmas, at 38.5 on CoVoST excluding Chinese. Read on 25 August 2026 the instruction-tuned repository records 3.35 million downloads in thirty days.

▸Videos

No videos yet.

▸Reviews