Netflix (Emma Yanyang Kong et al.) · Netflix multimodal embeddings

Multimodal embeddings for personalisation at Netflix

This paper from Netflix researchers describes how multimodal embeddings changed the production systems that personalise artwork images and video preview clips. Adding CLIP image embeddings to a two-tower model lets a single model serve all five Netflix artwork canvas types, replacing five separately trained per-canvas models and substantially improving cold-start performance. The authors also introduce MediaFM, an in-house tri-modal foundation model trained on shots from the Netflix show catalog, which fuses visual (SeqCLIP), audio (wav2vec 2.0) and timed-text signals. For video preview personalisation it outperforms strong visual-only baselines in both offline tests and online A/B tests. A simple offline proxy task, predicting the popularity-based winner from embeddings alone, is used to rank embedding models and versions, and it now gates every new MediaFM checkpoint. The paper also covers engineering decisions and failure modes, and was submitted to arXiv (cs.IR) on 18 August 2026; no code or licence is mentioned in the page.