A General Approach to MultiArmed Bandits Under Risk Criteria
[edit]
Proceedings of the 31st Conference On Learning Theory, PMLR 75:12951306, 2018.
Abstract
Different riskrelated criteria have received recent interest in learning problems, where typically each case is treated in a customized manner. In this paper we provide a more systematic approach to analyzing such risk criteria within a stochastic multiarmed bandit (MAB) formulation. We identify a set of general conditions that yield a simple characterization of the oracle rule (which serves as the regret benchmark), and facilitate the design of upper confidence bound (UCB) learning policies. The conditions are derived from problem primitives, primarily focusing on the relation between the arm reward distributions and the (risk criteria) performance metric. Among other things, the work highlights some (possibly nonintuitive) subtleties that differentiate various criteria in conjunction with statistical properties of the arms. Our main findings are illustrated on several widely used objectives such as conditional valueatrisk, meanvariance, Sharperatio, and more.
Related Material


