Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics

Ken Kansky, Tom Silver, David A. Mély, Mohamed Eldawy, Miguel Lázaro-Gredilla, Xinghua Lou, Nimrod Dorfman, Szymon Sidor, Scott Phoenix, Dileep George
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:1809-1818, 2017.

Abstract

The recent adaptation of deep neural network-based methods to reinforcement learning and planning domains has yielded remarkable progress on individual tasks. Nonetheless, progress on task-to-task transfer remains limited. In pursuit of efficient and robust generalization, we introduce the Schema Network, an object-oriented generative physics simulator capable of disentangling multiple causes of events and reasoning backward through causes to achieve goals. The richly structured architecture of the Schema Network can learn the dynamics of an environment directly from data. We compare Schema Networks with Asynchronous Advantage Actor-Critic and Progressive Networks on a suite of Breakout variations, reporting results on training efficiency and zero-shot generalization, consistently demonstrating faster, more robust learning and better transfer. We argue that generalizing from limited data and learning causal relationships are essential abilities on the path toward generally intelligent systems.

Cite this Paper

BibTeX
@InProceedings{pmlr-v70-kansky17a, title = {Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics}, author = {Ken Kansky and Tom Silver and David A. M{\'e}ly and Mohamed Eldawy and Miguel L{\'a}zaro-Gredilla and Xinghua Lou and Nimrod Dorfman and Szymon Sidor and Scott Phoenix and Dileep George}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {1809--1818}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, month = {06--11 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/kansky17a/kansky17a.pdf}, url = {https://proceedings.mlr.press/v70/kansky17a.html}, abstract = {The recent adaptation of deep neural network-based methods to reinforcement learning and planning domains has yielded remarkable progress on individual tasks. Nonetheless, progress on task-to-task transfer remains limited. In pursuit of efficient and robust generalization, we introduce the Schema Network, an object-oriented generative physics simulator capable of disentangling multiple causes of events and reasoning backward through causes to achieve goals. The richly structured architecture of the Schema Network can learn the dynamics of an environment directly from data. We compare Schema Networks with Asynchronous Advantage Actor-Critic and Progressive Networks on a suite of Breakout variations, reporting results on training efficiency and zero-shot generalization, consistently demonstrating faster, more robust learning and better transfer. We argue that generalizing from limited data and learning causal relationships are essential abilities on the path toward generally intelligent systems.} }
Endnote
%0 Conference Paper %T Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics %A Ken Kansky %A Tom Silver %A David A. Mély %A Mohamed Eldawy %A Miguel Lázaro-Gredilla %A Xinghua Lou %A Nimrod Dorfman %A Szymon Sidor %A Scott Phoenix %A Dileep George %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F pmlr-v70-kansky17a %I PMLR %P 1809--1818 %U https://proceedings.mlr.press/v70/kansky17a.html %V 70 %X The recent adaptation of deep neural network-based methods to reinforcement learning and planning domains has yielded remarkable progress on individual tasks. Nonetheless, progress on task-to-task transfer remains limited. In pursuit of efficient and robust generalization, we introduce the Schema Network, an object-oriented generative physics simulator capable of disentangling multiple causes of events and reasoning backward through causes to achieve goals. The richly structured architecture of the Schema Network can learn the dynamics of an environment directly from data. We compare Schema Networks with Asynchronous Advantage Actor-Critic and Progressive Networks on a suite of Breakout variations, reporting results on training efficiency and zero-shot generalization, consistently demonstrating faster, more robust learning and better transfer. We argue that generalizing from limited data and learning causal relationships are essential abilities on the path toward generally intelligent systems.
APA
Kansky, K., Silver, T., Mély, D.A., Eldawy, M., Lázaro-Gredilla, M., Lou, X., Dorfman, N., Sidor, S., Phoenix, S. & George, D.. (2017). Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:1809-1818 Available from https://proceedings.mlr.press/v70/kansky17a.html.

Related Material