PixelTransformer: Sample Conditioned Signal Generation

Shubham Tulsiani, Abhinav Gupta
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:10455-10464, 2021.

Abstract

We propose a generative model that can infer a distribution for the underlying spatial signal conditioned on sparse samples e.g. plausible images given a few observed pixels. In contrast to sequential autoregressive generative models, our model allows conditioning on arbitrary samples and can answer distributional queries for any location. We empirically validate our approach across three image datasets and show that we learn to generate diverse and meaningful samples, with the distribution variance reducing given more observed pixels. We also show that our approach is applicable beyond images and can allow generating other types of spatial outputs e.g. polynomials, 3D shapes, and videos.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-tulsiani21a, title = {PixelTransformer: Sample Conditioned Signal Generation}, author = {Tulsiani, Shubham and Gupta, Abhinav}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {10455--10464}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/tulsiani21a/tulsiani21a.pdf}, url = {https://proceedings.mlr.press/v139/tulsiani21a.html}, abstract = {We propose a generative model that can infer a distribution for the underlying spatial signal conditioned on sparse samples e.g. plausible images given a few observed pixels. In contrast to sequential autoregressive generative models, our model allows conditioning on arbitrary samples and can answer distributional queries for any location. We empirically validate our approach across three image datasets and show that we learn to generate diverse and meaningful samples, with the distribution variance reducing given more observed pixels. We also show that our approach is applicable beyond images and can allow generating other types of spatial outputs e.g. polynomials, 3D shapes, and videos.} }
Endnote
%0 Conference Paper %T PixelTransformer: Sample Conditioned Signal Generation %A Shubham Tulsiani %A Abhinav Gupta %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-tulsiani21a %I PMLR %P 10455--10464 %U https://proceedings.mlr.press/v139/tulsiani21a.html %V 139 %X We propose a generative model that can infer a distribution for the underlying spatial signal conditioned on sparse samples e.g. plausible images given a few observed pixels. In contrast to sequential autoregressive generative models, our model allows conditioning on arbitrary samples and can answer distributional queries for any location. We empirically validate our approach across three image datasets and show that we learn to generate diverse and meaningful samples, with the distribution variance reducing given more observed pixels. We also show that our approach is applicable beyond images and can allow generating other types of spatial outputs e.g. polynomials, 3D shapes, and videos.
APA
Tulsiani, S. & Gupta, A.. (2021). PixelTransformer: Sample Conditioned Signal Generation. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:10455-10464 Available from https://proceedings.mlr.press/v139/tulsiani21a.html.

Related Material