Global Optimization Networks

Sen Zhao, Erez Louidor, Maya Gupta
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:26927-26957, 2022.

Abstract

We consider the problem of estimating a good maximizer of a black-box function given noisy examples. We propose to fit a new type of function called a global optimization network (GON), defined as any composition of an invertible function and a unimodal function, whose unique global maximizer can be inferred in $\mathcal{O}(D)$ time, and used as the estimate. As an example way to construct GON functions, and interesting in its own right, we give new results for specifying multi-dimensional unimodal functions using lattice models with linear inequality constraints. We extend to conditional GONs that find a global maximizer conditioned on specified inputs of other dimensions. Experiments show the GON maximizers are statistically significantly better predictions than those produced by convex fits, GPR, or DNNs, and form more reasonable predictions for real-world problems.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-zhao22f, title = {Global Optimization Networks}, author = {Zhao, Sen and Louidor, Erez and Gupta, Maya}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {26927--26957}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/zhao22f/zhao22f.pdf}, url = {https://proceedings.mlr.press/v162/zhao22f.html}, abstract = {We consider the problem of estimating a good maximizer of a black-box function given noisy examples. We propose to fit a new type of function called a global optimization network (GON), defined as any composition of an invertible function and a unimodal function, whose unique global maximizer can be inferred in $\mathcal{O}(D)$ time, and used as the estimate. As an example way to construct GON functions, and interesting in its own right, we give new results for specifying multi-dimensional unimodal functions using lattice models with linear inequality constraints. We extend to conditional GONs that find a global maximizer conditioned on specified inputs of other dimensions. Experiments show the GON maximizers are statistically significantly better predictions than those produced by convex fits, GPR, or DNNs, and form more reasonable predictions for real-world problems.} }
Endnote
%0 Conference Paper %T Global Optimization Networks %A Sen Zhao %A Erez Louidor %A Maya Gupta %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-zhao22f %I PMLR %P 26927--26957 %U https://proceedings.mlr.press/v162/zhao22f.html %V 162 %X We consider the problem of estimating a good maximizer of a black-box function given noisy examples. We propose to fit a new type of function called a global optimization network (GON), defined as any composition of an invertible function and a unimodal function, whose unique global maximizer can be inferred in $\mathcal{O}(D)$ time, and used as the estimate. As an example way to construct GON functions, and interesting in its own right, we give new results for specifying multi-dimensional unimodal functions using lattice models with linear inequality constraints. We extend to conditional GONs that find a global maximizer conditioned on specified inputs of other dimensions. Experiments show the GON maximizers are statistically significantly better predictions than those produced by convex fits, GPR, or DNNs, and form more reasonable predictions for real-world problems.
APA
Zhao, S., Louidor, E. & Gupta, M.. (2022). Global Optimization Networks. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:26927-26957 Available from https://proceedings.mlr.press/v162/zhao22f.html.

Related Material