Distributionally Robust Active Learning for Gaussian Process Regression

Shion Takeno, Yoshito Okura, Yu Inatsu, Aoyama Tatsuya, Tomonari Tanaka, Akahane Satoshi, Hiroyuki Hanada, Noriaki Hashimoto, Taro Murayama, Hanju Lee, Shinya Kojima, Ichiro Takeuchi
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:58339-58358, 2025.

Abstract

Gaussian process regression (GPR) or kernel ridge regression is a widely used and powerful tool for nonlinear prediction. Therefore, active learning (AL) for GPR, which actively collects data labels to achieve an accurate prediction with fewer data labels, is an important problem. However, existing AL methods do not theoretically guarantee prediction accuracy for target distribution. Furthermore, as discussed in the distributionally robust learning literature, specifying the target distribution is often difficult. Thus, this paper proposes two AL methods that effectively reduce the worst-case expected error for GPR, which is the worst-case expectation in target distribution candidates. We show an upper bound of the worst-case expected squared error, which suggests that the error will be arbitrarily small by a finite number of data labels under mild conditions. Finally, we demonstrate the effectiveness of the proposed methods through synthetic and real-world datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-takeno25a, title = {Distributionally Robust Active Learning for {G}aussian Process Regression}, author = {Takeno, Shion and Okura, Yoshito and Inatsu, Yu and Tatsuya, Aoyama and Tanaka, Tomonari and Satoshi, Akahane and Hanada, Hiroyuki and Hashimoto, Noriaki and Murayama, Taro and Lee, Hanju and Kojima, Shinya and Takeuchi, Ichiro}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {58339--58358}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/takeno25a/takeno25a.pdf}, url = {https://proceedings.mlr.press/v267/takeno25a.html}, abstract = {Gaussian process regression (GPR) or kernel ridge regression is a widely used and powerful tool for nonlinear prediction. Therefore, active learning (AL) for GPR, which actively collects data labels to achieve an accurate prediction with fewer data labels, is an important problem. However, existing AL methods do not theoretically guarantee prediction accuracy for target distribution. Furthermore, as discussed in the distributionally robust learning literature, specifying the target distribution is often difficult. Thus, this paper proposes two AL methods that effectively reduce the worst-case expected error for GPR, which is the worst-case expectation in target distribution candidates. We show an upper bound of the worst-case expected squared error, which suggests that the error will be arbitrarily small by a finite number of data labels under mild conditions. Finally, we demonstrate the effectiveness of the proposed methods through synthetic and real-world datasets.} }
Endnote
%0 Conference Paper %T Distributionally Robust Active Learning for Gaussian Process Regression %A Shion Takeno %A Yoshito Okura %A Yu Inatsu %A Aoyama Tatsuya %A Tomonari Tanaka %A Akahane Satoshi %A Hiroyuki Hanada %A Noriaki Hashimoto %A Taro Murayama %A Hanju Lee %A Shinya Kojima %A Ichiro Takeuchi %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-takeno25a %I PMLR %P 58339--58358 %U https://proceedings.mlr.press/v267/takeno25a.html %V 267 %X Gaussian process regression (GPR) or kernel ridge regression is a widely used and powerful tool for nonlinear prediction. Therefore, active learning (AL) for GPR, which actively collects data labels to achieve an accurate prediction with fewer data labels, is an important problem. However, existing AL methods do not theoretically guarantee prediction accuracy for target distribution. Furthermore, as discussed in the distributionally robust learning literature, specifying the target distribution is often difficult. Thus, this paper proposes two AL methods that effectively reduce the worst-case expected error for GPR, which is the worst-case expectation in target distribution candidates. We show an upper bound of the worst-case expected squared error, which suggests that the error will be arbitrarily small by a finite number of data labels under mild conditions. Finally, we demonstrate the effectiveness of the proposed methods through synthetic and real-world datasets.
APA
Takeno, S., Okura, Y., Inatsu, Y., Tatsuya, A., Tanaka, T., Satoshi, A., Hanada, H., Hashimoto, N., Murayama, T., Lee, H., Kojima, S. & Takeuchi, I.. (2025). Distributionally Robust Active Learning for Gaussian Process Regression. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:58339-58358 Available from https://proceedings.mlr.press/v267/takeno25a.html.

Related Material