Tiny Aya Base

Tiny Aya Base is the model the rest of the family grows from. Cohere Labs pretrained it as a 3.35-billion-parameter multilingual trunk covering more than seventy languages, then built the instruction-following variants on top of it: tiny-aya-global with supervised fine-tuning and preference training, and the three regional models tuned for narrower language groups. What is published here is the stage before any of that alignment work, which is precisely why it exists as a separate release - a base model is the useful starting point for anyone who wants to fine-tune on their own data rather than inherit someone else's instruction behaviour. The architecture is the same throughout the family: an auto-regressive transformer of Cohere's second-generation design, in which three layers out of every four use sliding-window attention over 4096 tokens with rotary positional encoding, while the fourth uses global attention with no positional embeddings. The context window is 8K tokens in and 8K out. Because this is a base model, it should not be expected to follow instructions, hold a conversation or refuse anything: the safety and helpfulness training that produces those behaviours is what the derived variants add. The access terms are the family's terms, and they are stricter than the phrase "open weights" implies: CC BY-NC 4.0 with Cohere Labs' acceptable use policy, so research use only and no commercial use at any scale, plus a gate that requires an account and contact details before the files can be downloaded. By 17 August 2026 the repository had 1,515 downloads and 63 likes - a quiet number next to the instruction-tuned Global, which is the expected pattern for a base release.

▸Videos

No videos yet.

▸Reviews