LapSum - One Method to Differentiate Them All: Ranking, Sorting and Top-k Selection

Łukasz Struski, Michal B. Bednarczyk, Igor T. Podolak, Jacek Tabor
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:56990-57007, 2025.

Abstract

We present a novel technique for constructing differentiable order-type operations, including soft ranking, soft top-k selection, and soft permutations. Our approach leverages an efficient closed-form formula for the inverse of the function LapSum, defined as the sum of Laplace distributions. This formulation ensures low computational and memory complexity in selecting the highest activations, enabling losses and gradients to be computed in $O(n \log n)$ time. Through extensive experiments, we demonstrate that our method outperforms state-of-the-art techniques for high-dimensional vectors and large $k$ values. Furthermore, we provide efficient implementations for both CPU and CUDA environments, underscoring the practicality and scalability of our method for large-scale ranking and differentiable ordering problems.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-struski25a, title = {{L}ap{S}um - One Method to Differentiate Them All: Ranking, Sorting and Top-k Selection}, author = {Struski, {\L}ukasz and Bednarczyk, Michal B. and Podolak, Igor T. and Tabor, Jacek}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {56990--57007}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/struski25a/struski25a.pdf}, url = {https://proceedings.mlr.press/v267/struski25a.html}, abstract = {We present a novel technique for constructing differentiable order-type operations, including soft ranking, soft top-k selection, and soft permutations. Our approach leverages an efficient closed-form formula for the inverse of the function LapSum, defined as the sum of Laplace distributions. This formulation ensures low computational and memory complexity in selecting the highest activations, enabling losses and gradients to be computed in $O(n \log n)$ time. Through extensive experiments, we demonstrate that our method outperforms state-of-the-art techniques for high-dimensional vectors and large $k$ values. Furthermore, we provide efficient implementations for both CPU and CUDA environments, underscoring the practicality and scalability of our method for large-scale ranking and differentiable ordering problems.} }
Endnote
%0 Conference Paper %T LapSum - One Method to Differentiate Them All: Ranking, Sorting and Top-k Selection %A Łukasz Struski %A Michal B. Bednarczyk %A Igor T. Podolak %A Jacek Tabor %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-struski25a %I PMLR %P 56990--57007 %U https://proceedings.mlr.press/v267/struski25a.html %V 267 %X We present a novel technique for constructing differentiable order-type operations, including soft ranking, soft top-k selection, and soft permutations. Our approach leverages an efficient closed-form formula for the inverse of the function LapSum, defined as the sum of Laplace distributions. This formulation ensures low computational and memory complexity in selecting the highest activations, enabling losses and gradients to be computed in $O(n \log n)$ time. Through extensive experiments, we demonstrate that our method outperforms state-of-the-art techniques for high-dimensional vectors and large $k$ values. Furthermore, we provide efficient implementations for both CPU and CUDA environments, underscoring the practicality and scalability of our method for large-scale ranking and differentiable ordering problems.
APA
Struski, Ł., Bednarczyk, M.B., Podolak, I.T. & Tabor, J.. (2025). LapSum - One Method to Differentiate Them All: Ranking, Sorting and Top-k Selection. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:56990-57007 Available from https://proceedings.mlr.press/v267/struski25a.html.

Related Material