PrismAudio

PrismAudio generates a soundtrack for silent video: footsteps, doors, rain, room tone, the whole layer that a film crew records or assembles by hand. It comes from the same Alibaba team as ThinkSound, which introduced step-by-step reasoning to the task, and it was accepted to the main conference of ICLR 2026. Its contribution is to stop treating that reasoning as one thing. Where ThinkSound planned a soundtrack in a single chain of thought, PrismAudio runs four: a semantic line deciding what is making the sound, a temporal line deciding when it happens, an aesthetic line deciding how it should sound, and a spatial line deciding where in the stereo field it sits. Each has its own reward function, and the model is tuned by reinforcement learning against all four at once — the producer describes it as the first video-to-audio system trained this way. The separation matters because these goals pull against each other: a sound that lands on the exact frame may be the wrong sound, and a beautiful sound may arrive late. The model does not stand on its own foundations. Its declared backbones are two Google models — the VideoPrism-large video encoder and the T5Gemma text encoder — which makes this an Alibaba research artifact built on a competitor's components. The download is 5.75 GiB in three parts: the generator itself, an audio autoencoder and a synchronisation module borrowed from the ThinkSound release. The producer publishes no parameter count, and the weights ship as PyTorch .ckpt files rather than the safer safetensors format. The licence deserves attention. The repository is labelled MIT, which normally permits commercial use without asking. The model card then states that code, weights and datasets are for research and education only, and that commercial use requires explicit authorisation from the authors. Anyone reading only the label would draw the wrong conclusion. The repository has 82 stars and no recorded downloads, but that counter should not be read as neglect: Hugging Face only registers standard model files, and this release ships PyTorch checkpoints. The rival MMAudio repository reports zero downloads the same way.

▸Videos

No videos yet.

▸Reviews