Hugging Face tests a 4× reduction with pooling-aware training

What is changing?

Hugging Face has published a method that incorporates token pooling into the training of multi-vector embedding models. These architectures store a separate representation for each token instead of compressing a document into one vector, improving fine-grained matching while increasing index size and query cost. Token pooling reduces the number of stored vectors by clustering similar document vectors and replacing them with representative means.

In the NanoBEIR evaluation reported in the article, 4× pooling retained 94.6% of the unchanged nDCG@10 performance for a conventionally trained model. Pooling-aware training raised that figure to 97.8%. These are provider-run experiments and do not guarantee the same trade-off in every domain. The method offers a measurable optimization option for teams building smaller indexes for RAG, semantic search and visual document retrieval.