Non-Parametric Optimization for Scalable Learning in Stochastic Decision Problems

Mohsen Amidzadeh, Lauri Viitasaari, Mario Di Francesco
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:2332-2346, 2026.

Abstract

Stochastic optimization (SO) plays a central role in decision-making under uncertainty. Among SO problems, time-varying stochastic optimization (TV-SO) is particularly important due to its applications in adaptive control and machine learning. Non-parametric approaches have been proposed for time-varying deterministic optimization; however, they have not been developed for their stochastic counterparts. This work addresses that gap by developing a stochastic variational framework based on Malliavin calculus. This framework yields non-parametric optimality conditions for SO problems with stochastic decisions and supports the design of a scalable deep-learning algorithm that is insensitive to the parameterization dimension. This algorithm, called the Stochastic Path Follower (SPF), is applied to two important problems under distribution drift, namely least-squares recovery and logistic regression. Experimental results show that the proposed approach outperforms state-of-the-art learning-based and gradient-based methods in both performance and scalability.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-amidzadeh26a, title = {Non-Parametric Optimization for Scalable Learning in Stochastic Decision Problems}, author = {Amidzadeh, Mohsen and Viitasaari, Lauri and Di Francesco, Mario}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {2332--2346}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/amidzadeh26a/amidzadeh26a.pdf}, url = {https://proceedings.mlr.press/v306/amidzadeh26a.html}, abstract = {Stochastic optimization (SO) plays a central role in decision-making under uncertainty. Among SO problems, time-varying stochastic optimization (TV-SO) is particularly important due to its applications in adaptive control and machine learning. Non-parametric approaches have been proposed for time-varying deterministic optimization; however, they have not been developed for their stochastic counterparts. This work addresses that gap by developing a stochastic variational framework based on Malliavin calculus. This framework yields non-parametric optimality conditions for SO problems with stochastic decisions and supports the design of a scalable deep-learning algorithm that is insensitive to the parameterization dimension. This algorithm, called the Stochastic Path Follower (SPF), is applied to two important problems under distribution drift, namely least-squares recovery and logistic regression. Experimental results show that the proposed approach outperforms state-of-the-art learning-based and gradient-based methods in both performance and scalability.} }
Endnote
%0 Conference Paper %T Non-Parametric Optimization for Scalable Learning in Stochastic Decision Problems %A Mohsen Amidzadeh %A Lauri Viitasaari %A Mario Di Francesco %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-amidzadeh26a %I PMLR %P 2332--2346 %U https://proceedings.mlr.press/v306/amidzadeh26a.html %V 306 %X Stochastic optimization (SO) plays a central role in decision-making under uncertainty. Among SO problems, time-varying stochastic optimization (TV-SO) is particularly important due to its applications in adaptive control and machine learning. Non-parametric approaches have been proposed for time-varying deterministic optimization; however, they have not been developed for their stochastic counterparts. This work addresses that gap by developing a stochastic variational framework based on Malliavin calculus. This framework yields non-parametric optimality conditions for SO problems with stochastic decisions and supports the design of a scalable deep-learning algorithm that is insensitive to the parameterization dimension. This algorithm, called the Stochastic Path Follower (SPF), is applied to two important problems under distribution drift, namely least-squares recovery and logistic regression. Experimental results show that the proposed approach outperforms state-of-the-art learning-based and gradient-based methods in both performance and scalability.
APA
Amidzadeh, M., Viitasaari, L. & Di Francesco, M.. (2026). Non-Parametric Optimization for Scalable Learning in Stochastic Decision Problems. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:2332-2346 Available from https://proceedings.mlr.press/v306/amidzadeh26a.html.

Related Material