Stochastic Optimization for Spectral Risk Measures

Ronak Mehta, Vincent Roulet, Krishna Pillutla, Lang Liu, Zaid Harchaoui
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:10112-10159, 2023.

Abstract

Spectral risk objectives – also called L-risks – allow for learning systems to interpolate between optimizing average-case performance (as in empirical risk minimization) and worst-case performance on a task. We develop LSVRG, a stochastic algorithm to optimize these quantities by characterizing their subdifferential and addressing challenges such as biasedness of subgradient estimates and non-smoothness of the objective. We show theoretically and experimentally that out-of-the-box approaches such as stochastic subgradient and dual averaging can be hindered by bias, whereas our approach exhibits linear convergence.

Cite this Paper

BibTeX
@InProceedings{pmlr-v206-mehta23b, title = {Stochastic Optimization for Spectral Risk Measures}, author = {Mehta, Ronak and Roulet, Vincent and Pillutla, Krishna and Liu, Lang and Harchaoui, Zaid}, booktitle = {Proceedings of The 26th International Conference on Artificial Intelligence and Statistics}, pages = {10112--10159}, year = {2023}, editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}, volume = {206}, series = {Proceedings of Machine Learning Research}, month = {25--27 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v206/mehta23b/mehta23b.pdf}, url = {https://proceedings.mlr.press/v206/mehta23b.html}, abstract = {Spectral risk objectives – also called L-risks – allow for learning systems to interpolate between optimizing average-case performance (as in empirical risk minimization) and worst-case performance on a task. We develop LSVRG, a stochastic algorithm to optimize these quantities by characterizing their subdifferential and addressing challenges such as biasedness of subgradient estimates and non-smoothness of the objective. We show theoretically and experimentally that out-of-the-box approaches such as stochastic subgradient and dual averaging can be hindered by bias, whereas our approach exhibits linear convergence.} }
Endnote
%0 Conference Paper %T Stochastic Optimization for Spectral Risk Measures %A Ronak Mehta %A Vincent Roulet %A Krishna Pillutla %A Lang Liu %A Zaid Harchaoui %B Proceedings of The 26th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2023 %E Francisco Ruiz %E Jennifer Dy %E Jan-Willem van de Meent %F pmlr-v206-mehta23b %I PMLR %P 10112--10159 %U https://proceedings.mlr.press/v206/mehta23b.html %V 206 %X Spectral risk objectives – also called L-risks – allow for learning systems to interpolate between optimizing average-case performance (as in empirical risk minimization) and worst-case performance on a task. We develop LSVRG, a stochastic algorithm to optimize these quantities by characterizing their subdifferential and addressing challenges such as biasedness of subgradient estimates and non-smoothness of the objective. We show theoretically and experimentally that out-of-the-box approaches such as stochastic subgradient and dual averaging can be hindered by bias, whereas our approach exhibits linear convergence.
APA
Mehta, R., Roulet, V., Pillutla, K., Liu, L. & Harchaoui, Z.. (2023). Stochastic Optimization for Spectral Risk Measures. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 206:10112-10159 Available from https://proceedings.mlr.press/v206/mehta23b.html.

Related Material