Global Concavity and Optimization in a Class of Dynamic Discrete Choice Models

Yiding Feng, Ekaterina Khmelnitskaya, Denis Nekipelov
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:3082-3091, 2020.

Abstract

Discrete choice models with unobserved heterogeneity are commonly used Econometric models for dynamic Economic behavior which have been adopted in practice to predict behavior of individuals and firms from schooling and job choices to strategic decisions in market competition. These models feature optimizing agents who choose among a finite set of options in a sequence of periods and receive choice-specific payoffs that depend on both variables that are observed by the agent and recorded in the data and variables that are only observed by the agent but not recorded in the data. Existing work in Econometrics assumes that optimizing agents are fully rational and requires finding a functional fixed point to find the optimal policy. We show that in an important class of discrete choice models the value function is globally concave in the policy. That means that simple algorithms that do not require fixed point computation, such as the policy gradient algorithm, globally converge to the optimal policy. This finding can both be used to relax behavioral assumption regarding the optimizing agents and to facilitate Econometric analysis of dynamic behavior. In particular, we demonstrate significant computational advantages in using a simple implementation policy gradient algorithm over existing “nested fixed point” algorithms used in Econometrics.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-feng20b, title = {Global Concavity and Optimization in a Class of Dynamic Discrete Choice Models}, author = {Feng, Yiding and Khmelnitskaya, Ekaterina and Nekipelov, Denis}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {3082--3091}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/feng20b/feng20b.pdf}, url = {https://proceedings.mlr.press/v119/feng20b.html}, abstract = {Discrete choice models with unobserved heterogeneity are commonly used Econometric models for dynamic Economic behavior which have been adopted in practice to predict behavior of individuals and firms from schooling and job choices to strategic decisions in market competition. These models feature optimizing agents who choose among a finite set of options in a sequence of periods and receive choice-specific payoffs that depend on both variables that are observed by the agent and recorded in the data and variables that are only observed by the agent but not recorded in the data. Existing work in Econometrics assumes that optimizing agents are fully rational and requires finding a functional fixed point to find the optimal policy. We show that in an important class of discrete choice models the value function is globally concave in the policy. That means that simple algorithms that do not require fixed point computation, such as the policy gradient algorithm, globally converge to the optimal policy. This finding can both be used to relax behavioral assumption regarding the optimizing agents and to facilitate Econometric analysis of dynamic behavior. In particular, we demonstrate significant computational advantages in using a simple implementation policy gradient algorithm over existing “nested fixed point” algorithms used in Econometrics.} }
Endnote
%0 Conference Paper %T Global Concavity and Optimization in a Class of Dynamic Discrete Choice Models %A Yiding Feng %A Ekaterina Khmelnitskaya %A Denis Nekipelov %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-feng20b %I PMLR %P 3082--3091 %U https://proceedings.mlr.press/v119/feng20b.html %V 119 %X Discrete choice models with unobserved heterogeneity are commonly used Econometric models for dynamic Economic behavior which have been adopted in practice to predict behavior of individuals and firms from schooling and job choices to strategic decisions in market competition. These models feature optimizing agents who choose among a finite set of options in a sequence of periods and receive choice-specific payoffs that depend on both variables that are observed by the agent and recorded in the data and variables that are only observed by the agent but not recorded in the data. Existing work in Econometrics assumes that optimizing agents are fully rational and requires finding a functional fixed point to find the optimal policy. We show that in an important class of discrete choice models the value function is globally concave in the policy. That means that simple algorithms that do not require fixed point computation, such as the policy gradient algorithm, globally converge to the optimal policy. This finding can both be used to relax behavioral assumption regarding the optimizing agents and to facilitate Econometric analysis of dynamic behavior. In particular, we demonstrate significant computational advantages in using a simple implementation policy gradient algorithm over existing “nested fixed point” algorithms used in Econometrics.
APA
Feng, Y., Khmelnitskaya, E. & Nekipelov, D.. (2020). Global Concavity and Optimization in a Class of Dynamic Discrete Choice Models. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:3082-3091 Available from https://proceedings.mlr.press/v119/feng20b.html.

Related Material