GLASSES: Relieving The Myopia Of Bayesian Optimisation

Javier Gonzalez, Michael Osborne, Neil Lawrence
Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, PMLR 51:790-799, 2016.

Abstract

We present GLASSES: Global optimisation with Look-Ahead through Stochastic Simulation and Expected-loss Search. The majority of global optimisation approaches in use are myopic, in only considering the impact of the next function value; the non-myopic approaches that do exist are able to consider only a handful of future evaluations. Our novel algorithm, GLASSES, permits the consideration of dozens of evaluations into the future. This is done by approximating the ideal look-ahead loss function, which is expensive to evaluate, by a cheaper alternative in which the future steps of the algorithm are simulated beforehand. An Expectation Propagation algorithm is used to compute the expected value of the loss. We show that the far-horizon planning thus enabled leads to substantive performance gains in empirical tests.

Cite this Paper

BibTeX
@InProceedings{pmlr-v51-gonzalez16b, title = {GLASSES: Relieving The Myopia Of Bayesian Optimisation}, author = {Gonzalez, Javier and Osborne, Michael and Lawrence, Neil}, booktitle = {Proceedings of the 19th International Conference on Artificial Intelligence and Statistics}, pages = {790--799}, year = {2016}, editor = {Gretton, Arthur and Robert, Christian C.}, volume = {51}, series = {Proceedings of Machine Learning Research}, address = {Cadiz, Spain}, month = {09--11 May}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v51/gonzalez16b.pdf}, url = {https://proceedings.mlr.press/v51/gonzalez16b.html}, abstract = {We present GLASSES: Global optimisation with Look-Ahead through Stochastic Simulation and Expected-loss Search. The majority of global optimisation approaches in use are myopic, in only considering the impact of the next function value; the non-myopic approaches that do exist are able to consider only a handful of future evaluations. Our novel algorithm, GLASSES, permits the consideration of dozens of evaluations into the future. This is done by approximating the ideal look-ahead loss function, which is expensive to evaluate, by a cheaper alternative in which the future steps of the algorithm are simulated beforehand. An Expectation Propagation algorithm is used to compute the expected value of the loss. We show that the far-horizon planning thus enabled leads to substantive performance gains in empirical tests.} }
Endnote
%0 Conference Paper %T GLASSES: Relieving The Myopia Of Bayesian Optimisation %A Javier Gonzalez %A Michael Osborne %A Neil Lawrence %B Proceedings of the 19th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2016 %E Arthur Gretton %E Christian C. Robert %F pmlr-v51-gonzalez16b %I PMLR %P 790--799 %U https://proceedings.mlr.press/v51/gonzalez16b.html %V 51 %X We present GLASSES: Global optimisation with Look-Ahead through Stochastic Simulation and Expected-loss Search. The majority of global optimisation approaches in use are myopic, in only considering the impact of the next function value; the non-myopic approaches that do exist are able to consider only a handful of future evaluations. Our novel algorithm, GLASSES, permits the consideration of dozens of evaluations into the future. This is done by approximating the ideal look-ahead loss function, which is expensive to evaluate, by a cheaper alternative in which the future steps of the algorithm are simulated beforehand. An Expectation Propagation algorithm is used to compute the expected value of the loss. We show that the far-horizon planning thus enabled leads to substantive performance gains in empirical tests.
RIS
TY - CPAPER TI - GLASSES: Relieving The Myopia Of Bayesian Optimisation AU - Javier Gonzalez AU - Michael Osborne AU - Neil Lawrence BT - Proceedings of the 19th International Conference on Artificial Intelligence and Statistics DA - 2016/05/02 ED - Arthur Gretton ED - Christian C. Robert ID - pmlr-v51-gonzalez16b PB - PMLR DP - Proceedings of Machine Learning Research VL - 51 SP - 790 EP - 799 L1 - http://proceedings.mlr.press/v51/gonzalez16b.pdf UR - https://proceedings.mlr.press/v51/gonzalez16b.html AB - We present GLASSES: Global optimisation with Look-Ahead through Stochastic Simulation and Expected-loss Search. The majority of global optimisation approaches in use are myopic, in only considering the impact of the next function value; the non-myopic approaches that do exist are able to consider only a handful of future evaluations. Our novel algorithm, GLASSES, permits the consideration of dozens of evaluations into the future. This is done by approximating the ideal look-ahead loss function, which is expensive to evaluate, by a cheaper alternative in which the future steps of the algorithm are simulated beforehand. An Expectation Propagation algorithm is used to compute the expected value of the loss. We show that the far-horizon planning thus enabled leads to substantive performance gains in empirical tests. ER -
APA
Gonzalez, J., Osborne, M. & Lawrence, N.. (2016). GLASSES: Relieving The Myopia Of Bayesian Optimisation. Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 51:790-799 Available from https://proceedings.mlr.press/v51/gonzalez16b.html.

Related Material