Hugging Face (Tom Aarsen) · Pooling-aware embeddings

Token Pooling-Aware Training for Multi-Vector Embedding Models

Hugging Face shows how to train multi-vector embedding models to keep retrieval quality when document token vectors are merged by hierarchical token pooling. The method adds a pooling-aware loss to the Sentence Transformers library, averaging unpooled and pooled retrieval losses. At pool_factor=4, a normally trained model keeps 94.6% of its unpooled NanoBEIR nDCG@10 with a 4x reduction in document vectors, and pooling-aware training raises this to 97.8%.