Unsupervised Image Representation Learning with Deep Latent Particles

Tal Daniel, Aviv Tamar
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:4644-4665, 2022.

Abstract

We propose a new representation of visual data that disentangles object position from appearance. Our method, termed Deep Latent Particles (DLP), decomposes the visual input into low-dimensional latent “particles”, where each particle is described by its spatial location and features of its surrounding region. To drive learning of such representations, we follow a VAE-based based approach and introduce a prior for particle positions based on a spatial-Softmax architecture, and a modification of the evidence lower bound loss inspired by the Chamfer distance between particles. We demonstrate that our DLP representations are useful for downstream tasks such as unsupervised keypoint (KP) detection, image manipulation, and video prediction for scenes composed of multiple dynamic objects. In addition, we show that our probabilistic interpretation of the problem naturally provides uncertainty estimates for particle locations, which can be used for model selection, among other tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-daniel22a, title = {Unsupervised Image Representation Learning with Deep Latent Particles}, author = {Daniel, Tal and Tamar, Aviv}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {4644--4665}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/daniel22a/daniel22a.pdf}, url = {https://proceedings.mlr.press/v162/daniel22a.html}, abstract = {We propose a new representation of visual data that disentangles object position from appearance. Our method, termed Deep Latent Particles (DLP), decomposes the visual input into low-dimensional latent “particles”, where each particle is described by its spatial location and features of its surrounding region. To drive learning of such representations, we follow a VAE-based based approach and introduce a prior for particle positions based on a spatial-Softmax architecture, and a modification of the evidence lower bound loss inspired by the Chamfer distance between particles. We demonstrate that our DLP representations are useful for downstream tasks such as unsupervised keypoint (KP) detection, image manipulation, and video prediction for scenes composed of multiple dynamic objects. In addition, we show that our probabilistic interpretation of the problem naturally provides uncertainty estimates for particle locations, which can be used for model selection, among other tasks.} }
Endnote
%0 Conference Paper %T Unsupervised Image Representation Learning with Deep Latent Particles %A Tal Daniel %A Aviv Tamar %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-daniel22a %I PMLR %P 4644--4665 %U https://proceedings.mlr.press/v162/daniel22a.html %V 162 %X We propose a new representation of visual data that disentangles object position from appearance. Our method, termed Deep Latent Particles (DLP), decomposes the visual input into low-dimensional latent “particles”, where each particle is described by its spatial location and features of its surrounding region. To drive learning of such representations, we follow a VAE-based based approach and introduce a prior for particle positions based on a spatial-Softmax architecture, and a modification of the evidence lower bound loss inspired by the Chamfer distance between particles. We demonstrate that our DLP representations are useful for downstream tasks such as unsupervised keypoint (KP) detection, image manipulation, and video prediction for scenes composed of multiple dynamic objects. In addition, we show that our probabilistic interpretation of the problem naturally provides uncertainty estimates for particle locations, which can be used for model selection, among other tasks.
APA
Daniel, T. & Tamar, A.. (2022). Unsupervised Image Representation Learning with Deep Latent Particles. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:4644-4665 Available from https://proceedings.mlr.press/v162/daniel22a.html.

Related Material