Reparameterizing the Birkhoff Polytope for Variational Permutation Inference

Scott Linderman, Gonzalo Mena, Hal Cooper, Liam Paninski, John Cunningham
Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, PMLR 84:1618-1627, 2018.

Abstract

Many matching, tracking, sorting, and ranking problems require probabilistic reasoning about possible permutations, a set that grows factorially with dimension. Combinatorial optimization algorithms may enable efficient point estimation, but fully Bayesian inference poses a severe challenge in this high-dimensional, discrete space. To surmount this challenge, we start by relaxing the discrete set of permutation matrices to its convex hull the Birkhoff polytope, the set of doubly-stochastic matrices. We then introduce two novel transformations: an invertible and differentiable stick-breaking procedure that maps unconstrained space to the Birkhoff polytope, and a map that rounds points toward the vertices of the polytope. Both transformations include a temperature parameter that, in the limit, concentrates the densities on permutation matrices. We exploit these transformations and reparameterization gradients to introduce variational inference over permutation matrices, and we demonstrate its utility in a series of experiments.

Cite this Paper

BibTeX
@InProceedings{pmlr-v84-linderman18a, title = {Reparameterizing the Birkhoff Polytope for Variational Permutation Inference}, author = {Linderman, Scott and Mena, Gonzalo and Cooper, Hal and Paninski, Liam and Cunningham, John}, booktitle = {Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics}, pages = {1618--1627}, year = {2018}, editor = {Storkey, Amos and Perez-Cruz, Fernando}, volume = {84}, series = {Proceedings of Machine Learning Research}, month = {09--11 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v84/linderman18a/linderman18a.pdf}, url = {https://proceedings.mlr.press/v84/linderman18a.html}, abstract = { Many matching, tracking, sorting, and ranking problems require probabilistic reasoning about possible permutations, a set that grows factorially with dimension. Combinatorial optimization algorithms may enable efficient point estimation, but fully Bayesian inference poses a severe challenge in this high-dimensional, discrete space. To surmount this challenge, we start by relaxing the discrete set of permutation matrices to its convex hull the Birkhoff polytope, the set of doubly-stochastic matrices. We then introduce two novel transformations: an invertible and differentiable stick-breaking procedure that maps unconstrained space to the Birkhoff polytope, and a map that rounds points toward the vertices of the polytope. Both transformations include a temperature parameter that, in the limit, concentrates the densities on permutation matrices. We exploit these transformations and reparameterization gradients to introduce variational inference over permutation matrices, and we demonstrate its utility in a series of experiments. } }
Endnote
%0 Conference Paper %T Reparameterizing the Birkhoff Polytope for Variational Permutation Inference %A Scott Linderman %A Gonzalo Mena %A Hal Cooper %A Liam Paninski %A John Cunningham %B Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2018 %E Amos Storkey %E Fernando Perez-Cruz %F pmlr-v84-linderman18a %I PMLR %P 1618--1627 %U https://proceedings.mlr.press/v84/linderman18a.html %V 84 %X Many matching, tracking, sorting, and ranking problems require probabilistic reasoning about possible permutations, a set that grows factorially with dimension. Combinatorial optimization algorithms may enable efficient point estimation, but fully Bayesian inference poses a severe challenge in this high-dimensional, discrete space. To surmount this challenge, we start by relaxing the discrete set of permutation matrices to its convex hull the Birkhoff polytope, the set of doubly-stochastic matrices. We then introduce two novel transformations: an invertible and differentiable stick-breaking procedure that maps unconstrained space to the Birkhoff polytope, and a map that rounds points toward the vertices of the polytope. Both transformations include a temperature parameter that, in the limit, concentrates the densities on permutation matrices. We exploit these transformations and reparameterization gradients to introduce variational inference over permutation matrices, and we demonstrate its utility in a series of experiments.
APA
Linderman, S., Mena, G., Cooper, H., Paninski, L. & Cunningham, J.. (2018). Reparameterizing the Birkhoff Polytope for Variational Permutation Inference. Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 84:1618-1627 Available from https://proceedings.mlr.press/v84/linderman18a.html.

Related Material