Stochastic Video Generation with a Learned Prior

Emily Denton, Rob Fergus
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:1174-1183, 2018.

Abstract

Generating video frames that accurately predict future world states is challenging. Existing approaches either fail to capture the full distribution of outcomes, or yield blurry generations, or both. In this paper we introduce a video generation model with a learned prior over stochastic latent variables at each time step. Video frames are generated by drawing samples from this prior and combining them with a deterministic estimate of the future frame. The approach is simple and easily trained end-to-end on a variety of datasets. Sample generations are both varied and sharp, even many frames into the future, and compare favorably to those from existing approaches.

Cite this Paper

BibTeX
@InProceedings{pmlr-v80-denton18a, title = {Stochastic Video Generation with a Learned Prior}, author = {Denton, Emily and Fergus, Rob}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {1174--1183}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/denton18a/denton18a.pdf}, url = {https://proceedings.mlr.press/v80/denton18a.html}, abstract = {Generating video frames that accurately predict future world states is challenging. Existing approaches either fail to capture the full distribution of outcomes, or yield blurry generations, or both. In this paper we introduce a video generation model with a learned prior over stochastic latent variables at each time step. Video frames are generated by drawing samples from this prior and combining them with a deterministic estimate of the future frame. The approach is simple and easily trained end-to-end on a variety of datasets. Sample generations are both varied and sharp, even many frames into the future, and compare favorably to those from existing approaches.} }
Endnote
%0 Conference Paper %T Stochastic Video Generation with a Learned Prior %A Emily Denton %A Rob Fergus %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-denton18a %I PMLR %P 1174--1183 %U https://proceedings.mlr.press/v80/denton18a.html %V 80 %X Generating video frames that accurately predict future world states is challenging. Existing approaches either fail to capture the full distribution of outcomes, or yield blurry generations, or both. In this paper we introduce a video generation model with a learned prior over stochastic latent variables at each time step. Video frames are generated by drawing samples from this prior and combining them with a deterministic estimate of the future frame. The approach is simple and easily trained end-to-end on a variety of datasets. Sample generations are both varied and sharp, even many frames into the future, and compare favorably to those from existing approaches.
APA
Denton, E. & Fergus, R.. (2018). Stochastic Video Generation with a Learned Prior. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:1174-1183 Available from https://proceedings.mlr.press/v80/denton18a.html.

Related Material