Adaptive multi-fidelity optimization with fast learning rates

Côme Fiegel, Victor Gabillon, Michal Valko
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, PMLR 108:3493-3502, 2020.

Abstract

In multi-fidelity optimization, biased approximations of varying costs of the target function are available. This paper studies the problem of optimizing a locally smooth function with a limited budget, where the learner has to make a tradeoff between the cost and the bias of these approximations. We first prove lower bounds for the simple regret under different assumptions on the fidelities, based on a cost-to-bias function. We then present the Kometo algorithm which achieves, with additional logarithmic factors, the same rates without any knowledge of the function smoothness and fidelity assumptions, and improves previously proven guarantees. We finally empirically show that our algorithm outperforms previous multi-fidelity optimization methods without the knowledge of problem-dependent parameters.

Cite this Paper

BibTeX
@InProceedings{pmlr-v108-fiegel20a, title = {Adaptive multi-fidelity optimization with fast learning rates}, author = {Fiegel, C\^ome and Gabillon, Victor and Valko, Michal}, booktitle = {Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics}, pages = {3493--3502}, year = {2020}, editor = {Chiappa, Silvia and Calandra, Roberto}, volume = {108}, series = {Proceedings of Machine Learning Research}, month = {26--28 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v108/fiegel20a/fiegel20a.pdf}, url = {https://proceedings.mlr.press/v108/fiegel20a.html}, abstract = {In multi-fidelity optimization, biased approximations of varying costs of the target function are available. This paper studies the problem of optimizing a locally smooth function with a limited budget, where the learner has to make a tradeoff between the cost and the bias of these approximations. We first prove lower bounds for the simple regret under different assumptions on the fidelities, based on a cost-to-bias function. We then present the Kometo algorithm which achieves, with additional logarithmic factors, the same rates without any knowledge of the function smoothness and fidelity assumptions, and improves previously proven guarantees. We finally empirically show that our algorithm outperforms previous multi-fidelity optimization methods without the knowledge of problem-dependent parameters.} }
Endnote
%0 Conference Paper %T Adaptive multi-fidelity optimization with fast learning rates %A Côme Fiegel %A Victor Gabillon %A Michal Valko %B Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2020 %E Silvia Chiappa %E Roberto Calandra %F pmlr-v108-fiegel20a %I PMLR %P 3493--3502 %U https://proceedings.mlr.press/v108/fiegel20a.html %V 108 %X In multi-fidelity optimization, biased approximations of varying costs of the target function are available. This paper studies the problem of optimizing a locally smooth function with a limited budget, where the learner has to make a tradeoff between the cost and the bias of these approximations. We first prove lower bounds for the simple regret under different assumptions on the fidelities, based on a cost-to-bias function. We then present the Kometo algorithm which achieves, with additional logarithmic factors, the same rates without any knowledge of the function smoothness and fidelity assumptions, and improves previously proven guarantees. We finally empirically show that our algorithm outperforms previous multi-fidelity optimization methods without the knowledge of problem-dependent parameters.
APA
Fiegel, C., Gabillon, V. & Valko, M.. (2020). Adaptive multi-fidelity optimization with fast learning rates. Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 108:3493-3502 Available from https://proceedings.mlr.press/v108/fiegel20a.html.

Related Material