Deep Models of Interactions Across Sets

Jason Hartford, Devon Graham, Kevin Leyton-Brown, Siamak Ravanbakhsh
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:1909-1918, 2018.

Abstract

We use deep learning to model interactions across two or more sets of objects, such as user{–}movie ratings or protein{–}drug bindings. The canonical representation of such interactions is a matrix (or tensor) with an exchangeability property: the encoding’s meaning is not changed by permuting rows or columns. We argue that models should hence be Permutation Equivariant (PE): constrained to make the same predictions across such permutations. We present a parameter-sharing scheme and prove that it is maximally expressive under the PE constraint. This scheme yields three benefits. First, we demonstrate performance competitive with the state of the art on multiple matrix completion benchmarks. Second, our models require a number of parameters independent of the numbers of objects and thus scale well to large datasets. Third, models can be queried about new objects that were not available at training time, but for which interactions have since been observed. We observed surprisingly good generalization performance on this matrix extrapolation task, both within domains (e.g., new users and new movies drawn from the same distribution used for training) and even across domains (e.g., predicting music ratings after training on movie ratings).

Cite this Paper

BibTeX
@InProceedings{pmlr-v80-hartford18a, title = {Deep Models of Interactions Across Sets}, author = {Hartford, Jason and Graham, Devon and Leyton-Brown, Kevin and Ravanbakhsh, Siamak}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {1909--1918}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/hartford18a/hartford18a.pdf}, url = {https://proceedings.mlr.press/v80/hartford18a.html}, abstract = {We use deep learning to model interactions across two or more sets of objects, such as user{–}movie ratings or protein{–}drug bindings. The canonical representation of such interactions is a matrix (or tensor) with an exchangeability property: the encoding’s meaning is not changed by permuting rows or columns. We argue that models should hence be Permutation Equivariant (PE): constrained to make the same predictions across such permutations. We present a parameter-sharing scheme and prove that it is maximally expressive under the PE constraint. This scheme yields three benefits. First, we demonstrate performance competitive with the state of the art on multiple matrix completion benchmarks. Second, our models require a number of parameters independent of the numbers of objects and thus scale well to large datasets. Third, models can be queried about new objects that were not available at training time, but for which interactions have since been observed. We observed surprisingly good generalization performance on this matrix extrapolation task, both within domains (e.g., new users and new movies drawn from the same distribution used for training) and even across domains (e.g., predicting music ratings after training on movie ratings).} }
Endnote
%0 Conference Paper %T Deep Models of Interactions Across Sets %A Jason Hartford %A Devon Graham %A Kevin Leyton-Brown %A Siamak Ravanbakhsh %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-hartford18a %I PMLR %P 1909--1918 %U https://proceedings.mlr.press/v80/hartford18a.html %V 80 %X We use deep learning to model interactions across two or more sets of objects, such as user{–}movie ratings or protein{–}drug bindings. The canonical representation of such interactions is a matrix (or tensor) with an exchangeability property: the encoding’s meaning is not changed by permuting rows or columns. We argue that models should hence be Permutation Equivariant (PE): constrained to make the same predictions across such permutations. We present a parameter-sharing scheme and prove that it is maximally expressive under the PE constraint. This scheme yields three benefits. First, we demonstrate performance competitive with the state of the art on multiple matrix completion benchmarks. Second, our models require a number of parameters independent of the numbers of objects and thus scale well to large datasets. Third, models can be queried about new objects that were not available at training time, but for which interactions have since been observed. We observed surprisingly good generalization performance on this matrix extrapolation task, both within domains (e.g., new users and new movies drawn from the same distribution used for training) and even across domains (e.g., predicting music ratings after training on movie ratings).
APA
Hartford, J., Graham, D., Leyton-Brown, K. & Ravanbakhsh, S.. (2018). Deep Models of Interactions Across Sets. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:1909-1918 Available from https://proceedings.mlr.press/v80/hartford18a.html.

Related Material