Unsupervised Part Representation by Flow Capsules

Sara Sabour, Andrea Tagliasacchi, Soroosh Yazdani, Geoffrey Hinton, David J Fleet
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:9213-9223, 2021.

Abstract

Capsule networks aim to parse images into a hierarchy of objects, parts and relations. While promising, they remain limited by an inability to learn effective low level part descriptions. To address this issue we propose a way to learn primary capsule encoders that detect atomic parts from a single image. During training we exploit motion as a powerful perceptual cue for part definition, with an expressive decoder for part generation within a layered image model with occlusion. Experiments demonstrate robust part discovery in the presence of multiple objects, cluttered backgrounds, and occlusion. The learned part decoder is shown to infer the underlying shape masks, effectively filling in occluded regions of the detected shapes. We evaluate FlowCapsules on unsupervised part segmentation and unsupervised image classification.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-sabour21a, title = {Unsupervised Part Representation by Flow Capsules}, author = {Sabour, Sara and Tagliasacchi, Andrea and Yazdani, Soroosh and Hinton, Geoffrey and Fleet, David J}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {9213--9223}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/sabour21a/sabour21a.pdf}, url = {https://proceedings.mlr.press/v139/sabour21a.html}, abstract = {Capsule networks aim to parse images into a hierarchy of objects, parts and relations. While promising, they remain limited by an inability to learn effective low level part descriptions. To address this issue we propose a way to learn primary capsule encoders that detect atomic parts from a single image. During training we exploit motion as a powerful perceptual cue for part definition, with an expressive decoder for part generation within a layered image model with occlusion. Experiments demonstrate robust part discovery in the presence of multiple objects, cluttered backgrounds, and occlusion. The learned part decoder is shown to infer the underlying shape masks, effectively filling in occluded regions of the detected shapes. We evaluate FlowCapsules on unsupervised part segmentation and unsupervised image classification.} }
Endnote
%0 Conference Paper %T Unsupervised Part Representation by Flow Capsules %A Sara Sabour %A Andrea Tagliasacchi %A Soroosh Yazdani %A Geoffrey Hinton %A David J Fleet %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-sabour21a %I PMLR %P 9213--9223 %U https://proceedings.mlr.press/v139/sabour21a.html %V 139 %X Capsule networks aim to parse images into a hierarchy of objects, parts and relations. While promising, they remain limited by an inability to learn effective low level part descriptions. To address this issue we propose a way to learn primary capsule encoders that detect atomic parts from a single image. During training we exploit motion as a powerful perceptual cue for part definition, with an expressive decoder for part generation within a layered image model with occlusion. Experiments demonstrate robust part discovery in the presence of multiple objects, cluttered backgrounds, and occlusion. The learned part decoder is shown to infer the underlying shape masks, effectively filling in occluded regions of the detected shapes. We evaluate FlowCapsules on unsupervised part segmentation and unsupervised image classification.
APA
Sabour, S., Tagliasacchi, A., Yazdani, S., Hinton, G. & Fleet, D.J.. (2021). Unsupervised Part Representation by Flow Capsules. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:9213-9223 Available from https://proceedings.mlr.press/v139/sabour21a.html.

Related Material