Séminaires

Encoding an entire document as a single vector for retrieval creates an information bottleneck. This has motivated late-interaction models such as ColBERT, which achieve competitive performance by encoding documents at the token level instead. However, these models must store a dense embedding for every document token, which leads to substantial index storage overhead. Past works prune low-importance token embeddings using statistical and empirical measures, but these approaches often lack formal grounding or prove ineffective. To address these shortcomings, we introduce a framework grounded in high-dimensional geometry that casts token pruning as a Voronoi cell estimation problem in the embedding space. We measure each token’s influence by the size of its Voronoi region, the part of the space where it is the closest match to a query. This enables principled pruning that preserves retrieval quality while reducing index size. Our experiments show that the approach is a competitive pruning strategy and also a useful tool for improving and interpreting token-level behavior in dense retrieval systems.