Unsupervised Learning of Latent Physical Properties Using Perception-Prediction Networks

David Zheng, Vinson Luo, Jiajun Wu, Joshua Tenenbaum
Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, PMLR R16:496-506, 2018.

Abstract

We propose a framework for the completely unsupervised learning of latent object prop- erties from their interactions: the perception- prediction network (PPN). Consisting of a per- ception module that extracts representations of latent object properties and a prediction module that uses those extracted properties to simulate system dynamics, the PPN can be trained in an end-to-end fashion purely from samples of object dynamics. The representations of latent object properties learned by PPNs not only are sufficient to accurately simulate the dynamics of systems comprised of previously unseen ob- jects, but also can be translated directly into human-interpretable properties (e.g. mass, co- efficient of restitution) in an entirely unsuper- vised manner. Crucially, PPNs also generalize to novel scenarios: their gradient-based training can be applied to many dynamical systems and their graph-based structure functions over sys- tems comprised of different numbers of objects. Our results demonstrate the efficacy of graph- based neural architectures in object-centric in- ference and prediction tasks, and our model has the potential to discover relevant object proper- ties in systems that are not yet well understood.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR16-zheng18b, title = {Unsupervised Learning of Latent Physical Properties Using Perception-Prediction Networks}, author = {Zheng, David and Luo, Vinson and Wu, Jiajun and Tenenbaum, Joshua}, booktitle = {Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence}, pages = {496--506}, year = {2018}, editor = {Globerson, Amir and Silva, Ricardo}, volume = {R16}, series = {Proceedings of Machine Learning Research}, month = {06--10 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r16/main/assets/zheng18b/zheng18b.pdf}, url = {https://proceedings.mlr.press/r16/zheng18b.html}, abstract = {We propose a framework for the completely unsupervised learning of latent object prop- erties from their interactions: the perception- prediction network (PPN). Consisting of a per- ception module that extracts representations of latent object properties and a prediction module that uses those extracted properties to simulate system dynamics, the PPN can be trained in an end-to-end fashion purely from samples of object dynamics. The representations of latent object properties learned by PPNs not only are sufficient to accurately simulate the dynamics of systems comprised of previously unseen ob- jects, but also can be translated directly into human-interpretable properties (e.g. mass, co- efficient of restitution) in an entirely unsuper- vised manner. Crucially, PPNs also generalize to novel scenarios: their gradient-based training can be applied to many dynamical systems and their graph-based structure functions over sys- tems comprised of different numbers of objects. Our results demonstrate the efficacy of graph- based neural architectures in object-centric in- ference and prediction tasks, and our model has the potential to discover relevant object proper- ties in systems that are not yet well understood.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Unsupervised Learning of Latent Physical Properties Using Perception-Prediction Networks %A David Zheng %A Vinson Luo %A Jiajun Wu %A Joshua Tenenbaum %B Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2018 %E Amir Globerson %E Ricardo Silva %F pmlr-vR16-zheng18b %I PMLR %P 496--506 %U https://proceedings.mlr.press/r16/zheng18b.html %V R16 %X We propose a framework for the completely unsupervised learning of latent object prop- erties from their interactions: the perception- prediction network (PPN). Consisting of a per- ception module that extracts representations of latent object properties and a prediction module that uses those extracted properties to simulate system dynamics, the PPN can be trained in an end-to-end fashion purely from samples of object dynamics. The representations of latent object properties learned by PPNs not only are sufficient to accurately simulate the dynamics of systems comprised of previously unseen ob- jects, but also can be translated directly into human-interpretable properties (e.g. mass, co- efficient of restitution) in an entirely unsuper- vised manner. Crucially, PPNs also generalize to novel scenarios: their gradient-based training can be applied to many dynamical systems and their graph-based structure functions over sys- tems comprised of different numbers of objects. Our results demonstrate the efficacy of graph- based neural architectures in object-centric in- ference and prediction tasks, and our model has the potential to discover relevant object proper- ties in systems that are not yet well understood. %Z Reissued by PMLR on 04 October 2026.
APA
Zheng, D., Luo, V., Wu, J. & Tenenbaum, J.. (2018). Unsupervised Learning of Latent Physical Properties Using Perception-Prediction Networks. Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R16:496-506 Available from https://proceedings.mlr.press/r16/zheng18b.html. Reissued by PMLR on 04 October 2026.

Related Material