Hierarchical Retrieval at Scale: Bridging Interpretability and Efficiency

Shubham Gupta, Zichao Li, Tianyi Chen, Cem Subakan, Siva Reddy, Perouz Taslakian, Valentina Zantedeschi
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:38908-38943, 2026.

Abstract

Information retrieval is a core component of many intelligent systems as it enables conditioning of outputs on new and large-scale datasets. While effective, the standard practice of encoding data into high-dimensional representations for similarity search entails large memory and compute footprints, and also makes it hard to inspect the inner workings of the system. Hierarchical retrieval methods offer an interpretable alternative by organizing data at multiple granular levels, yet do not match the efficiency and performance of flat retrieval approaches. In this paper, we propose Retreever, a tree-based method that makes hierarchical retrieval viable at scale by directly optimizing its structure for retrieval performance while naturally providing transparency through meaningful semantic groupings. Our method offers the flexibility to balance cost and utility by indexing data using representations from any tree level. We show that Retreever delivers strong coarse (intermediate levels) and fine representations (terminal level), while achieving the highest retrieval accuracy at the lowest latency among hierarchical methods. These results demonstrate that this family of techniques is viable in practical applications.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-gupta26c, title = {Hierarchical Retrieval at Scale: Bridging Interpretability and Efficiency}, author = {Gupta, Shubham and Li, Zichao and Chen, Tianyi and Subakan, Cem and Reddy, Siva and Taslakian, Perouz and Zantedeschi, Valentina}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {38908--38943}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/gupta26c/gupta26c.pdf}, url = {https://proceedings.mlr.press/v306/gupta26c.html}, abstract = {Information retrieval is a core component of many intelligent systems as it enables conditioning of outputs on new and large-scale datasets. While effective, the standard practice of encoding data into high-dimensional representations for similarity search entails large memory and compute footprints, and also makes it hard to inspect the inner workings of the system. Hierarchical retrieval methods offer an interpretable alternative by organizing data at multiple granular levels, yet do not match the efficiency and performance of flat retrieval approaches. In this paper, we propose Retreever, a tree-based method that makes hierarchical retrieval viable at scale by directly optimizing its structure for retrieval performance while naturally providing transparency through meaningful semantic groupings. Our method offers the flexibility to balance cost and utility by indexing data using representations from any tree level. We show that Retreever delivers strong coarse (intermediate levels) and fine representations (terminal level), while achieving the highest retrieval accuracy at the lowest latency among hierarchical methods. These results demonstrate that this family of techniques is viable in practical applications.} }
Endnote
%0 Conference Paper %T Hierarchical Retrieval at Scale: Bridging Interpretability and Efficiency %A Shubham Gupta %A Zichao Li %A Tianyi Chen %A Cem Subakan %A Siva Reddy %A Perouz Taslakian %A Valentina Zantedeschi %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-gupta26c %I PMLR %P 38908--38943 %U https://proceedings.mlr.press/v306/gupta26c.html %V 306 %X Information retrieval is a core component of many intelligent systems as it enables conditioning of outputs on new and large-scale datasets. While effective, the standard practice of encoding data into high-dimensional representations for similarity search entails large memory and compute footprints, and also makes it hard to inspect the inner workings of the system. Hierarchical retrieval methods offer an interpretable alternative by organizing data at multiple granular levels, yet do not match the efficiency and performance of flat retrieval approaches. In this paper, we propose Retreever, a tree-based method that makes hierarchical retrieval viable at scale by directly optimizing its structure for retrieval performance while naturally providing transparency through meaningful semantic groupings. Our method offers the flexibility to balance cost and utility by indexing data using representations from any tree level. We show that Retreever delivers strong coarse (intermediate levels) and fine representations (terminal level), while achieving the highest retrieval accuracy at the lowest latency among hierarchical methods. These results demonstrate that this family of techniques is viable in practical applications.
APA
Gupta, S., Li, Z., Chen, T., Subakan, C., Reddy, S., Taslakian, P. & Zantedeschi, V.. (2026). Hierarchical Retrieval at Scale: Bridging Interpretability and Efficiency. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:38908-38943 Available from https://proceedings.mlr.press/v306/gupta26c.html.

Related Material