Learning functions on multiple sets using multi-set transformers

Kira A. Selby, Ahmad Rashid, Ivan Kobyzev, Mehdi Rezagholizadeh, Pascal Poupart
Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, PMLR 180:1760-1770, 2022.

Abstract

We propose a general deep architecture for learning functions on multiple permutation-invariant sets. We also show how to generalize this architecture to sets of elements of any dimension by dimension equivariance. We demonstrate that our architecture is a universal approximator of these functions, and show superior results to existing methods on a variety of tasks including counting tasks, alignment tasks, distinguishability tasks and statistical distance measurements. This last task is quite important in Machine Learning. Although our approach is quite general, we demonstrate that it can generate approximate estimates of KL divergence and mutual information that are more accurate than previous techniques that are specifically designed to approximate those statistical distances.

Cite this Paper

BibTeX
@InProceedings{pmlr-v180-selby22a, title = {Learning functions on multiple sets using multi-set transformers}, author = {Selby, Kira A. and Rashid, Ahmad and Kobyzev, Ivan and Rezagholizadeh, Mehdi and Poupart, Pascal}, booktitle = {Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence}, pages = {1760--1770}, year = {2022}, editor = {Cussens, James and Zhang, Kun}, volume = {180}, series = {Proceedings of Machine Learning Research}, month = {01--05 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v180/selby22a/selby22a.pdf}, url = {https://proceedings.mlr.press/v180/selby22a.html}, abstract = {We propose a general deep architecture for learning functions on multiple permutation-invariant sets. We also show how to generalize this architecture to sets of elements of any dimension by dimension equivariance. We demonstrate that our architecture is a universal approximator of these functions, and show superior results to existing methods on a variety of tasks including counting tasks, alignment tasks, distinguishability tasks and statistical distance measurements. This last task is quite important in Machine Learning. Although our approach is quite general, we demonstrate that it can generate approximate estimates of KL divergence and mutual information that are more accurate than previous techniques that are specifically designed to approximate those statistical distances.} }
Endnote
%0 Conference Paper %T Learning functions on multiple sets using multi-set transformers %A Kira A. Selby %A Ahmad Rashid %A Ivan Kobyzev %A Mehdi Rezagholizadeh %A Pascal Poupart %B Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2022 %E James Cussens %E Kun Zhang %F pmlr-v180-selby22a %I PMLR %P 1760--1770 %U https://proceedings.mlr.press/v180/selby22a.html %V 180 %X We propose a general deep architecture for learning functions on multiple permutation-invariant sets. We also show how to generalize this architecture to sets of elements of any dimension by dimension equivariance. We demonstrate that our architecture is a universal approximator of these functions, and show superior results to existing methods on a variety of tasks including counting tasks, alignment tasks, distinguishability tasks and statistical distance measurements. This last task is quite important in Machine Learning. Although our approach is quite general, we demonstrate that it can generate approximate estimates of KL divergence and mutual information that are more accurate than previous techniques that are specifically designed to approximate those statistical distances.
APA
Selby, K.A., Rashid, A., Kobyzev, I., Rezagholizadeh, M. & Poupart, P.. (2022). Learning functions on multiple sets using multi-set transformers. Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 180:1760-1770 Available from https://proceedings.mlr.press/v180/selby22a.html.

Related Material