Language modelIn productionretires 11/26/2026 — in 71 days
Jamba 1.5 Large
AI21 Labs · Israel · 2024
The first hybrid Mamba-attention model released with open weights at flagship scale — and the first to hold quality across a genuine 256,000-token window.
Jamba 1.5 Large is the model that made AI21 Labs' architectural bet legible to the rest of the industry. Released on 22 August 2024 with open weights under the Jamba Open Model License, it interleaves Mamba state-space layers with conventional attention layers and a sparse mixture-of-experts, reaching 398 billion parameters of which 94 billion are active per token. The point of that shape is long context: attention alone gets quadratically expensive as input grows, while state-space layers stay cheap but historically forgot detail. AI21 published the RULER benchmark to show the hybrid kept both, and Jamba 1.5 Large was the only model in that table whose effective context matched its claimed context — a full 256K, scoring 93.9 at the longest setting, while Gemini 1.5 Pro, GPT-4 Turbo and Llama 3.1 70B all degraded well before their limits. The model is honest about its cost: it cannot be loaded in FP16 on a single node of eight 80GB GPUs, so AI21 shipped ExpertsInt8, its own quantisation for mixture-of-experts models in vLLM, to make one node enough. Its knowledge stops on 5 March 2024. AI21 has since moved on — this generation left its own API on 6 May 2025 — but Amazon Bedrock still sells it, with an end-of-life date of 26 November 2026.