Toward Compositional Generalization in Object-Oriented World Modeling

Linfeng Zhao, Lingzhi Kong, Robin Walters, Lawson L.S. Wong
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:26841-26864, 2022.

Abstract

Compositional generalization is a critical ability in learning and decision-making. We focus on the setting of reinforcement learning in object-oriented environments to study compositional generalization in world modeling. We (1) formalize the compositional generalization problem with an algebraic approach and (2) study how a world model can achieve that. We introduce a conceptual environment, Object Library, and two instances, and deploy a principled pipeline to measure the generalization ability. Motivated by the formulation, we analyze several methods with exact or no compositional generalization ability using our framework, and design a differentiable approach, Homomorphic Object-oriented World Model (HOWM), that achieves soft but more efficient compositional generalization.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-zhao22b, title = {Toward Compositional Generalization in Object-Oriented World Modeling}, author = {Zhao, Linfeng and Kong, Lingzhi and Walters, Robin and Wong, Lawson L.S.}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {26841--26864}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/zhao22b/zhao22b.pdf}, url = {https://proceedings.mlr.press/v162/zhao22b.html}, abstract = {Compositional generalization is a critical ability in learning and decision-making. We focus on the setting of reinforcement learning in object-oriented environments to study compositional generalization in world modeling. We (1) formalize the compositional generalization problem with an algebraic approach and (2) study how a world model can achieve that. We introduce a conceptual environment, Object Library, and two instances, and deploy a principled pipeline to measure the generalization ability. Motivated by the formulation, we analyze several methods with exact or no compositional generalization ability using our framework, and design a differentiable approach, Homomorphic Object-oriented World Model (HOWM), that achieves soft but more efficient compositional generalization.} }
Endnote
%0 Conference Paper %T Toward Compositional Generalization in Object-Oriented World Modeling %A Linfeng Zhao %A Lingzhi Kong %A Robin Walters %A Lawson L.S. Wong %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-zhao22b %I PMLR %P 26841--26864 %U https://proceedings.mlr.press/v162/zhao22b.html %V 162 %X Compositional generalization is a critical ability in learning and decision-making. We focus on the setting of reinforcement learning in object-oriented environments to study compositional generalization in world modeling. We (1) formalize the compositional generalization problem with an algebraic approach and (2) study how a world model can achieve that. We introduce a conceptual environment, Object Library, and two instances, and deploy a principled pipeline to measure the generalization ability. Motivated by the formulation, we analyze several methods with exact or no compositional generalization ability using our framework, and design a differentiable approach, Homomorphic Object-oriented World Model (HOWM), that achieves soft but more efficient compositional generalization.
APA
Zhao, L., Kong, L., Walters, R. & Wong, L.L.. (2022). Toward Compositional Generalization in Object-Oriented World Modeling. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:26841-26864 Available from https://proceedings.mlr.press/v162/zhao22b.html.

Related Material