Universal Representation of Generalized Convex Functions and their Gradients

Moeen Nehzati
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:92155-92169, 2026.

Abstract

A wide range of optimization problems can often be written in terms of generalized convex functions (GCFs). When this structure is present, it can convert certain nested bilevel objectives into single-level problems amenable to standard first-order optimization methods. We provide a new differentiable layer with a convex parameter space and show (Theorems 5.1 and 5.2) that it and its gradient are universal approximators for GCFs and their gradients. We demonstrate how this parameterization can be leveraged in practice by (i) learning optimal transport maps with general cost functions and (ii) learning optimal auctions of multiple goods. In both these cases, we show how our layer can be used to convert the existing bilevel or min-max formulations into single-level problems that can be solved efficiently with first-order methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-nehzati26a, title = {Universal Representation of Generalized Convex Functions and their Gradients}, author = {Nehzati, Moeen}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {92155--92169}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/nehzati26a/nehzati26a.pdf}, url = {https://proceedings.mlr.press/v306/nehzati26a.html}, abstract = {A wide range of optimization problems can often be written in terms of generalized convex functions (GCFs). When this structure is present, it can convert certain nested bilevel objectives into single-level problems amenable to standard first-order optimization methods. We provide a new differentiable layer with a convex parameter space and show (Theorems 5.1 and 5.2) that it and its gradient are universal approximators for GCFs and their gradients. We demonstrate how this parameterization can be leveraged in practice by (i) learning optimal transport maps with general cost functions and (ii) learning optimal auctions of multiple goods. In both these cases, we show how our layer can be used to convert the existing bilevel or min-max formulations into single-level problems that can be solved efficiently with first-order methods.} }
Endnote
%0 Conference Paper %T Universal Representation of Generalized Convex Functions and their Gradients %A Moeen Nehzati %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-nehzati26a %I PMLR %P 92155--92169 %U https://proceedings.mlr.press/v306/nehzati26a.html %V 306 %X A wide range of optimization problems can often be written in terms of generalized convex functions (GCFs). When this structure is present, it can convert certain nested bilevel objectives into single-level problems amenable to standard first-order optimization methods. We provide a new differentiable layer with a convex parameter space and show (Theorems 5.1 and 5.2) that it and its gradient are universal approximators for GCFs and their gradients. We demonstrate how this parameterization can be leveraged in practice by (i) learning optimal transport maps with general cost functions and (ii) learning optimal auctions of multiple goods. In both these cases, we show how our layer can be used to convert the existing bilevel or min-max formulations into single-level problems that can be solved efficiently with first-order methods.
APA
Nehzati, M.. (2026). Universal Representation of Generalized Convex Functions and their Gradients. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:92155-92169 Available from https://proceedings.mlr.press/v306/nehzati26a.html.

Related Material