My House, My Rules: Learning Tidying Preferences with Graph Neural Networks

Ivan Kapelyukh, Edward Johns
Proceedings of the 5th Conference on Robot Learning, PMLR 164:740-749, 2022.

Abstract

Robots that arrange household objects should do so according to the user’s preferences, which are inherently subjective and difficult to model. We present NeatNet: a novel Variational Autoencoder architecture using Graph Neural Network layers, which can extract a low-dimensional latent preference vector from a user by observing how they arrange scenes. Given any set of objects, this vector can then be used to generate an arrangement which is tailored to that user’s spatial preferences, with word embeddings used for generalisation to new objects. We develop a tidying simulator to gather rearrangement examples from 75 users, and demonstrate empirically that our method consistently produces neat and personalised arrangements across a variety of rearrangement scenarios.

Cite this Paper

BibTeX
@InProceedings{pmlr-v164-kapelyukh22a, title = {My House, My Rules: Learning Tidying Preferences with Graph Neural Networks}, author = {Kapelyukh, Ivan and Johns, Edward}, booktitle = {Proceedings of the 5th Conference on Robot Learning}, pages = {740--749}, year = {2022}, editor = {Faust, Aleksandra and Hsu, David and Neumann, Gerhard}, volume = {164}, series = {Proceedings of Machine Learning Research}, month = {08--11 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v164/kapelyukh22a/kapelyukh22a.pdf}, url = {https://proceedings.mlr.press/v164/kapelyukh22a.html}, abstract = {Robots that arrange household objects should do so according to the user’s preferences, which are inherently subjective and difficult to model. We present NeatNet: a novel Variational Autoencoder architecture using Graph Neural Network layers, which can extract a low-dimensional latent preference vector from a user by observing how they arrange scenes. Given any set of objects, this vector can then be used to generate an arrangement which is tailored to that user’s spatial preferences, with word embeddings used for generalisation to new objects. We develop a tidying simulator to gather rearrangement examples from 75 users, and demonstrate empirically that our method consistently produces neat and personalised arrangements across a variety of rearrangement scenarios.} }
Endnote
%0 Conference Paper %T My House, My Rules: Learning Tidying Preferences with Graph Neural Networks %A Ivan Kapelyukh %A Edward Johns %B Proceedings of the 5th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2022 %E Aleksandra Faust %E David Hsu %E Gerhard Neumann %F pmlr-v164-kapelyukh22a %I PMLR %P 740--749 %U https://proceedings.mlr.press/v164/kapelyukh22a.html %V 164 %X Robots that arrange household objects should do so according to the user’s preferences, which are inherently subjective and difficult to model. We present NeatNet: a novel Variational Autoencoder architecture using Graph Neural Network layers, which can extract a low-dimensional latent preference vector from a user by observing how they arrange scenes. Given any set of objects, this vector can then be used to generate an arrangement which is tailored to that user’s spatial preferences, with word embeddings used for generalisation to new objects. We develop a tidying simulator to gather rearrangement examples from 75 users, and demonstrate empirically that our method consistently produces neat and personalised arrangements across a variety of rearrangement scenarios.
APA
Kapelyukh, I. & Johns, E.. (2022). My House, My Rules: Learning Tidying Preferences with Graph Neural Networks. Proceedings of the 5th Conference on Robot Learning, in Proceedings of Machine Learning Research 164:740-749 Available from https://proceedings.mlr.press/v164/kapelyukh22a.html.

Related Material