IBM Granite 4.0 H Small

Granite-4.0-H-Small is the largest model of IBM's Granite 4.0 generation and the clearest expression of what made that generation unusual. Of its 40 layers only four use attention; the remaining 36 are Mamba-2 state-space layers, and the model carries no positional encoding at all — the NoPE design leaves order to be inferred from the recurrence itself. The published weights total 32.21 billion parameters, of which about 9 billion are active on any token. The mixture of experts holds 72 experts with ten routed per token plus a shared expert, embedding size is 4096, and the four attention layers use 32 heads with 8 key-value heads. Sequence length is 128K tokens. IBM trained the family on an NVIDIA GB200 NVL72 cluster at CoreWeave. The point of the design is memory. A conventional transformer keeps a key-value cache that grows with the length of the conversation; Mamba-2 layers keep a fixed-size state instead, so long inputs cost roughly the same as short ones. In a 32-billion-parameter model meant for enterprise servers that is the difference between one accelerator and several. On IBM's own runs H Small leads the generation across the board: MMLU 78.44, MMLU-Pro 55.47, BBH 81.62, GPQA 40.63, IFEval average 87.55, ArenaHard 46.48, HumanEval 88, MBPP 84, BigCodeBench 46.23 and BFCL v3 64.69 for tool calling. The one benchmark where it loses to its smaller siblings is multilingual grade-school maths (MGSM 38.72 against 45.36 for H Tiny). The experiment did not carry forward. Granite 4.1, released in April 2026, and Granite 4.2 after it are dense attention-only transformers with RoPE positions — closer to the conventional Micro model of this generation than to its hybrid flagship. Readers comparing our Granite profiles should treat 4.0 as an architectural branch rather than a step on a straight line. The licence is Apache 2.0 with no conditions attached, which in a year of licences carrying territorial carve-outs and user thresholds is worth stating plainly.

▸News

IBM built a generation of models that barely uses attention — then quietly went back

9/1/2026 · IBM (karty modeli i pliki konfiguracyjne ibm-granite na Hugging Face)

▸Videos

No videos yet.

▸Reviews