Inverse Decision Modeling: Learning Interpretable Representations of Behavior

Daniel Jarrett, Alihan Hüyük, Mihaela Van Der Schaar
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:4755-4771, 2021.

Abstract

Decision analysis deals with modeling and enhancing decision processes. A principal challenge in improving behavior is in obtaining a transparent *description* of existing behavior in the first place. In this paper, we develop an expressive, unifying perspective on *inverse decision modeling*: a framework for learning parameterized representations of sequential decision behavior. First, we formalize the *forward* problem (as a normative standard), subsuming common classes of control behavior. Second, we use this to formalize the *inverse* problem (as a descriptive model), generalizing existing work on imitation/reward learning—while opening up a much broader class of research problems in behavior representation. Finally, we instantiate this approach with an example (*inverse bounded rational control*), illustrating how this structure enables learning (interpretable) representations of (bounded) rationality—while naturally capturing intuitive notions of suboptimal actions, biased beliefs, and imperfect knowledge of environments.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-jarrett21a, title = {Inverse Decision Modeling: Learning Interpretable Representations of Behavior}, author = {Jarrett, Daniel and H{\"u}y{\"u}k, Alihan and Van Der Schaar, Mihaela}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {4755--4771}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/jarrett21a/jarrett21a.pdf}, url = {https://proceedings.mlr.press/v139/jarrett21a.html}, abstract = {Decision analysis deals with modeling and enhancing decision processes. A principal challenge in improving behavior is in obtaining a transparent *description* of existing behavior in the first place. In this paper, we develop an expressive, unifying perspective on *inverse decision modeling*: a framework for learning parameterized representations of sequential decision behavior. First, we formalize the *forward* problem (as a normative standard), subsuming common classes of control behavior. Second, we use this to formalize the *inverse* problem (as a descriptive model), generalizing existing work on imitation/reward learning—while opening up a much broader class of research problems in behavior representation. Finally, we instantiate this approach with an example (*inverse bounded rational control*), illustrating how this structure enables learning (interpretable) representations of (bounded) rationality—while naturally capturing intuitive notions of suboptimal actions, biased beliefs, and imperfect knowledge of environments.} }
Endnote
%0 Conference Paper %T Inverse Decision Modeling: Learning Interpretable Representations of Behavior %A Daniel Jarrett %A Alihan Hüyük %A Mihaela Van Der Schaar %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-jarrett21a %I PMLR %P 4755--4771 %U https://proceedings.mlr.press/v139/jarrett21a.html %V 139 %X Decision analysis deals with modeling and enhancing decision processes. A principal challenge in improving behavior is in obtaining a transparent *description* of existing behavior in the first place. In this paper, we develop an expressive, unifying perspective on *inverse decision modeling*: a framework for learning parameterized representations of sequential decision behavior. First, we formalize the *forward* problem (as a normative standard), subsuming common classes of control behavior. Second, we use this to formalize the *inverse* problem (as a descriptive model), generalizing existing work on imitation/reward learning—while opening up a much broader class of research problems in behavior representation. Finally, we instantiate this approach with an example (*inverse bounded rational control*), illustrating how this structure enables learning (interpretable) representations of (bounded) rationality—while naturally capturing intuitive notions of suboptimal actions, biased beliefs, and imperfect knowledge of environments.
APA
Jarrett, D., Hüyük, A. & Van Der Schaar, M.. (2021). Inverse Decision Modeling: Learning Interpretable Representations of Behavior. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:4755-4771 Available from https://proceedings.mlr.press/v139/jarrett21a.html.

Related Material