Obeying the Order: Introducing Ordered Transfer Hyperparameter Optimization

Sigrid Passano Hellan, Huibin Shen, Francois-Xavier Aubet, David Salinas, Aaron Klein
Proceedings of the Fourth International Conference on Automated Machine Learning, PMLR 293:14/1-29, 2025.

Abstract

In many deployed settings, hyperparameters are retuned as more data are collected; for instance tuning a sequence of movie recommendation systems as more movies and rating are added. Despite this, transfer hyperparameter optimisation (HPO) has not been thoroughly analysed in this setting. We introduce ordered transfer hyperparameter optimisation (OTHPO), a version of transfer learning for HPO where the tasks follow a sequential order. Unlike for state-of-the-art transfer HPO, the assumption is that each task is most correlated to those immediately before it. We propose a formal definition and illustrate the key difference with standard transfer HPO approaches. We show how simple methods taking the order into account can outperform more sophisticated transfer methods by better tracking smooth shifts of the hyperparameter landscape. The ten benchmarks are in the setting of gradually accumulating data, as well as a separate real-world motivated optimisation problem, and are open sourced to foster future research on ordered transfer HPO.

Cite this Paper

BibTeX
@InProceedings{pmlr-v293-hellan25a, title = {Obeying the Order: Introducing Ordered Transfer Hyperparameter Optimization}, author = {Hellan, Sigrid Passano and Shen, Huibin and Aubet, Francois-Xavier and Salinas, David and Klein, Aaron}, booktitle = {Proceedings of the Fourth International Conference on Automated Machine Learning}, pages = {14/1--29}, year = {2025}, editor = {Akoglu, Leman and Doerr, Carola and van Rijn, Jan N. and Garnett, Roman and Gardner, Jacob R.}, volume = {293}, series = {Proceedings of Machine Learning Research}, month = {08--11 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v293/main/assets/hellan25a/hellan25a.pdf}, url = {https://proceedings.mlr.press/v293/hellan25a.html}, abstract = {In many deployed settings, hyperparameters are retuned as more data are collected; for instance tuning a sequence of movie recommendation systems as more movies and rating are added. Despite this, transfer hyperparameter optimisation (HPO) has not been thoroughly analysed in this setting. We introduce ordered transfer hyperparameter optimisation (OTHPO), a version of transfer learning for HPO where the tasks follow a sequential order. Unlike for state-of-the-art transfer HPO, the assumption is that each task is most correlated to those immediately before it. We propose a formal definition and illustrate the key difference with standard transfer HPO approaches. We show how simple methods taking the order into account can outperform more sophisticated transfer methods by better tracking smooth shifts of the hyperparameter landscape. The ten benchmarks are in the setting of gradually accumulating data, as well as a separate real-world motivated optimisation problem, and are open sourced to foster future research on ordered transfer HPO.} }
Endnote
%0 Conference Paper %T Obeying the Order: Introducing Ordered Transfer Hyperparameter Optimization %A Sigrid Passano Hellan %A Huibin Shen %A Francois-Xavier Aubet %A David Salinas %A Aaron Klein %B Proceedings of the Fourth International Conference on Automated Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Leman Akoglu %E Carola Doerr %E Jan N. van Rijn %E Roman Garnett %E Jacob R. Gardner %F pmlr-v293-hellan25a %I PMLR %P 14/1--29 %U https://proceedings.mlr.press/v293/hellan25a.html %V 293 %X In many deployed settings, hyperparameters are retuned as more data are collected; for instance tuning a sequence of movie recommendation systems as more movies and rating are added. Despite this, transfer hyperparameter optimisation (HPO) has not been thoroughly analysed in this setting. We introduce ordered transfer hyperparameter optimisation (OTHPO), a version of transfer learning for HPO where the tasks follow a sequential order. Unlike for state-of-the-art transfer HPO, the assumption is that each task is most correlated to those immediately before it. We propose a formal definition and illustrate the key difference with standard transfer HPO approaches. We show how simple methods taking the order into account can outperform more sophisticated transfer methods by better tracking smooth shifts of the hyperparameter landscape. The ten benchmarks are in the setting of gradually accumulating data, as well as a separate real-world motivated optimisation problem, and are open sourced to foster future research on ordered transfer HPO.
APA
Hellan, S.P., Shen, H., Aubet, F., Salinas, D. & Klein, A.. (2025). Obeying the Order: Introducing Ordered Transfer Hyperparameter Optimization. Proceedings of the Fourth International Conference on Automated Machine Learning, in Proceedings of Machine Learning Research 293:14/1-29 Available from https://proceedings.mlr.press/v293/hellan25a.html.

Related Material