DiffusionGemma 26B A4B

DiffusionGemma is an experimental open model that Google DeepMind announced on 10 June 2026 to test a different way of producing text. Ordinary language models work like a typewriter: one token, then the next, each conditioned on what came before. DiffusionGemma instead drafts a whole block of 256 tokens — the model card calls it a canvas — and then refines it over up to 48 denoising steps, so every token in the block can see every other token. The trick, borrowed from image diffusion and from Google's earlier Gemini Diffusion research, turns generation from a memory-bandwidth problem into a compute problem, which is exactly the shape of work a single idle consumer GPU is good at. Underneath it is the Gemma 4 26B A4B mixture of experts: 30 layers, 8 active experts out of 128 plus one shared, roughly 3.8 billion parameters working per token out of 25.2 billion in total, a 256,000-token context and a vision encoder of about 550 million parameters. What is new is the shape of inference. An autoregressive encoder reads the prompt and fills the key-value cache; a decoder then applies bidirectional attention across the canvas and hands each finished block back to the encoder. Google reports more than 1,000 tokens per second on a single NVIDIA H100 and more than 700 on a GeForce RTX 5090, with the quantised model fitting inside 18 GB of video memory. The unusual thing about this release is how openly the manufacturer states the cost. Google publishes DiffusionGemma side by side with the autoregressive Gemma 4 it is built on, and its own model loses nearly every comparison: 77.6 percent against 82.6 on MMLU Pro, 69.1 against 88.3 on AIME 2026 without tools, 73.2 against 82.3 on GPQA Diamond, 54.3 against 73.8 on MMMU Pro, Codeforces Elo 1429 against 1718. The single benchmark it wins is Humanity's Last Exam without tools, 11.0 against 8.7 percent. The blog post says it plainly: for work that demands maximum quality, deploy standard Gemma 4. DiffusionGemma is aimed at low-latency local uses where the block-at-once approach is a structural advantage rather than a compromise — in-line editing, code infilling, and problems whose tokens depend on each other in both directions. Google's own example is a fine-tune that solves Sudoku, a task autoregressive models struggle with because every square depends on squares not yet written.

▸Videos

No videos yet.

▸Reviews