Jamba Large

Jamba Large is the flagship model of AI21 Labs, the Tel Aviv laboratory founded in 2017 that predates most of the current generation of AI companies. Its interest lies in the architecture: Jamba interleaves Mamba state-space layers with conventional attention layers instead of relying on attention alone. Attention is what makes long documents expensive, because its cost grows with the square of the input and the key-value cache grows with it; state-space layers do not have that problem but historically recalled detail less reliably. The hybrid is an attempt to buy the cheapness of one and the precision of the other, and it is the reason a 398-billion-parameter model with 94 billion active can hold 256,000 tokens without the memory bill that shape would normally imply. AI21 sells it where that matters most — banks, insurers and health systems that want long internal documents processed inside their own walls — offering private cloud, on-premises and hybrid deployment alongside the API. It reads and writes in nine languages, costs two dollars per million input tokens and eight per million output, and its knowledge stops on 22 August 2024.

▸Videos

No videos yet.

▸Reviews