An Approximate Solution Method for Large Risk-Averse Markov Decision Processes

Marek Petrik, Dharmashankar Subramanian
Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence, PMLR R10:804-813, 2012.

Abstract

Stochastic domains often involve risk-averse decision makers. While recent work has focused on how to model risk in Markov decision processes using risk measures, it has not addressed the problem of solving large risk-averse formulations. In this paper, we propose and analyze a new method for solving large risk-averse MDPs with hybrid continuous-discrete state spaces and continuous action spaces. The proposed method iteratively improves a bound on the value function using a linearity structure of the MDP. We demonstrate the utility and properties of the method on a portfolio optimization problem.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR10-petrik12a, title = {An Approximate Solution Method for Large Risk-Averse {M}arkov Decision Processes}, author = {Petrik, Marek and Subramanian, Dharmashankar}, booktitle = {Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence}, pages = {804--813}, year = {2012}, editor = {de Freitas, Nando and Murphy, Kevin}, volume = {R10}, series = {Proceedings of Machine Learning Research}, month = {14--18 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r10/main/assets/petrik12a/petrik12a.pdf}, url = {https://proceedings.mlr.press/r10/petrik12a.html}, abstract = {Stochastic domains often involve risk-averse decision makers. While recent work has focused on how to model risk in Markov decision processes using risk measures, it has not addressed the problem of solving large risk-averse formulations. In this paper, we propose and analyze a new method for solving large risk-averse MDPs with hybrid continuous-discrete state spaces and continuous action spaces. The proposed method iteratively improves a bound on the value function using a linearity structure of the MDP. We demonstrate the utility and properties of the method on a portfolio optimization problem.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T An Approximate Solution Method for Large Risk-Averse Markov Decision Processes %A Marek Petrik %A Dharmashankar Subramanian %B Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2012 %E Nando de Freitas %E Kevin Murphy %F pmlr-vR10-petrik12a %I PMLR %P 804--813 %U https://proceedings.mlr.press/r10/petrik12a.html %V R10 %X Stochastic domains often involve risk-averse decision makers. While recent work has focused on how to model risk in Markov decision processes using risk measures, it has not addressed the problem of solving large risk-averse formulations. In this paper, we propose and analyze a new method for solving large risk-averse MDPs with hybrid continuous-discrete state spaces and continuous action spaces. The proposed method iteratively improves a bound on the value function using a linearity structure of the MDP. We demonstrate the utility and properties of the method on a portfolio optimization problem. %Z Reissued by PMLR on 04 October 2026.
APA
Petrik, M. & Subramanian, D.. (2012). An Approximate Solution Method for Large Risk-Averse Markov Decision Processes. Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R10:804-813 Available from https://proceedings.mlr.press/r10/petrik12a.html. Reissued by PMLR on 04 October 2026.

Related Material