TIPSv2

TIPSv2 — Text-Image Pre-training with Spatial awareness — is a family of vision-language encoders from Google DeepMind, published on Hugging Face on 9 April 2026 alongside a CVPR 2026 paper. It belongs to the same class as SigLIP: an image and a piece of text are pushed into a shared vector space, so labels can be supplied as text at inference time. The difference is what the model returns. Besides one global vector for the whole image it hands back a separate feature vector for every 14-by-14-pixel patch, aligned with the same text space. A 448-pixel image therefore yields 1,024 patch descriptions rather than a single one. That is what turns an encoder into a tool for tasks that need to know where something is, not merely whether it is there. Zero-shot segmentation follows directly: compare the text embedding of a word against every patch and the matching ones outline the object. Google also ships separate DPT variants for each size, which add a dense-prediction head for depth estimation on top of the same backbone. The patch features are useful even without any head — running principal component analysis over them and painting the first three components as colour channels produces a readable map of the image's structure, which is the standard way of showing that a model has understood a scene rather than merely labelled it. The family comes in four sizes, each counted separately for its two towers: B/14 at 86 million vision and 110 million text parameters, L/14 at 303 and 184 million, SO400m/14 at 412 and 448 million, and g/14 at 1.1 billion and 389 million, with embedding dimensions from 768 to 1,536. Both towers are 27-layer transformers. Images take any resolution and need no ImageNet normalisation; text goes through a SentencePiece tokeniser, lowercased, capped at 64 tokens. Everything is Apache 2.0 with no gate. Traffic splits almost evenly between SO400m and the small B/14 — 276,000 and 256,000 downloads a month respectively — which suggests the model is being used both as a research backbone and as a cheap component. Google publishes no benchmark table on the model card; the comparisons are in the paper.

▸Videos

No videos yet.

▸Reviews