IBM built a generation of models that barely uses attention — then quietly went back

Nine out of every ten layers in IBM's Granite 4.0 models contain no attention mechanism at all. The generation, published on 2 October 2025, replaced 36 of its 40 layers with Mamba-2 state-space blocks and dropped positional encoding entirely — the configuration files name the design "nope", for no positional embedding. The models infer word order from the recurrence itself. The reason was memory. A conventional transformer holds a key-value cache that grows with every token of the conversation, which is why long documents get expensive faster than they get slow. A Mamba-2 layer keeps a state of fixed size instead, so a 128,000-token input costs roughly what a short one costs. For models meant to run on a single accelerator, that is the whole argument. The experiment did not survive its own family. IBM's Granite 4.1, published in April 2026, and Granite 4.2 after it are ordinary dense transformers: every layer uses attention, positions are encoded with RoPE, and the hybrid machinery is gone. The configuration files make the break plain — Granite 4.0 declares the model type granitemoehybrid, its successors declare plain granite. The detail that makes this more than a footnote is which model people are actually using. IBM hedged its own bet in October 2025 by shipping one non-hybrid model alongside the three hybrids, for inference software that could not yet run Mamba layers. That fallback, Granite 4.0 Micro, is the most downloaded model of the generation — about 59,500 pulls of its repository in the thirty days to 1 September 2026, against 19,100 for its hybrid twin of identical size. Its layer count, embedding size and head count are also what IBM carried into the smallest model of the next generation. There is a counterweight. The one Granite 4.0 model that API brokers still resell today is a hybrid — Granite 4.0 H Micro, at about 0.017 dollars per million input tokens, cheaper than IBM's own newer offerings. And the most downloaded hybrid, Granite 4.0 H Tiny, stores seven billion parameters but computes with roughly one, which is why it still finds homes on modest hardware nearly a year after two newer generations arrived. All four models are Apache 2.0 with no territorial exclusions and no user thresholds. Polish is not among the twelve languages IBM tested, in this generation or the two that followed. wujec.ai has added profiles of all four Granite 4.0 models, with the manufacturer's own benchmark figures and the architectural differences set side by side.