Anchor-Based Heteroscedastic Noise for Preferential Bayesian Optimization

Marshal Arijona Sinaga, Julien Martinelli, Samuel Kaski
Proceedings of The 1st Symposium on Probabilistic Machine Learning, PMLR 327:1-26, 2026.

Abstract

Preferential Bayesian optimization (PBO) learns latent utilities from pairwise comparisons, but most existing methods assume homoscedastic comparison noise. This is inadequate in human-in-the-loop settings, where a user may compare some designs reliably and others only hesitantly. We propose a heteroscedastic noise model for PBO: before optimization, the user provides a small set of reliable examples, called anchors, and a kernel density estimator (KDE) turns these anchors into an input-dependent map of user uncertainty. We incorporate this map into preferential GP surrogates and derive risk-averse acquisition functions that trade off utility and ease of comparison. We further show that a risk-adjusted variant of the popular expected utility of the best option (EUBO) preserves the one-step Bayes-optimality guarantee up to an additive constant, and that under an idealized i.i.d. anchor model the KDE estimator enjoys standard consistency and concentration rates. Experiments on synthetic problems and human-preference datasets show improved risk-adjusted performance and clarify how anchor placement affects the method.

Cite this Paper

BibTeX
@InProceedings{pmlr-v327-sinaga26a, title = {Anchor-Based Heteroscedastic Noise for Preferential Bayesian Optimization}, author = {Sinaga, Marshal Arijona and Martinelli, Julien and Kaski, Samuel}, booktitle = {Proceedings of The 1st Symposium on Probabilistic Machine Learning}, pages = {1--26}, year = {2026}, editor = {Swaroop, Siddharth and Rügamer, David and Kristiadi, Agustinus}, volume = {327}, series = {Proceedings of Machine Learning Research}, month = {05 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v327/main/assets/sinaga26a/sinaga26a.pdf}, url = {https://proceedings.mlr.press/v327/sinaga26a.html}, abstract = { Preferential Bayesian optimization (PBO) learns latent utilities from pairwise comparisons, but most existing methods assume homoscedastic comparison noise. This is inadequate in human-in-the-loop settings, where a user may compare some designs reliably and others only hesitantly. We propose a heteroscedastic noise model for PBO: before optimization, the user provides a small set of reliable examples, called anchors, and a kernel density estimator (KDE) turns these anchors into an input-dependent map of user uncertainty. We incorporate this map into preferential GP surrogates and derive risk-averse acquisition functions that trade off utility and ease of comparison. We further show that a risk-adjusted variant of the popular expected utility of the best option (EUBO) preserves the one-step Bayes-optimality guarantee up to an additive constant, and that under an idealized i.i.d. anchor model the KDE estimator enjoys standard consistency and concentration rates. Experiments on synthetic problems and human-preference datasets show improved risk-adjusted performance and clarify how anchor placement affects the method. } }
Endnote
%0 Conference Paper %T Anchor-Based Heteroscedastic Noise for Preferential Bayesian Optimization %A Marshal Arijona Sinaga %A Julien Martinelli %A Samuel Kaski %B Proceedings of The 1st Symposium on Probabilistic Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Siddharth Swaroop %E David Rügamer %E Agustinus Kristiadi %F pmlr-v327-sinaga26a %I PMLR %P 1--26 %U https://proceedings.mlr.press/v327/sinaga26a.html %V 327 %X Preferential Bayesian optimization (PBO) learns latent utilities from pairwise comparisons, but most existing methods assume homoscedastic comparison noise. This is inadequate in human-in-the-loop settings, where a user may compare some designs reliably and others only hesitantly. We propose a heteroscedastic noise model for PBO: before optimization, the user provides a small set of reliable examples, called anchors, and a kernel density estimator (KDE) turns these anchors into an input-dependent map of user uncertainty. We incorporate this map into preferential GP surrogates and derive risk-averse acquisition functions that trade off utility and ease of comparison. We further show that a risk-adjusted variant of the popular expected utility of the best option (EUBO) preserves the one-step Bayes-optimality guarantee up to an additive constant, and that under an idealized i.i.d. anchor model the KDE estimator enjoys standard consistency and concentration rates. Experiments on synthetic problems and human-preference datasets show improved risk-adjusted performance and clarify how anchor placement affects the method.
APA
Sinaga, M.A., Martinelli, J. & Kaski, S.. (2026). Anchor-Based Heteroscedastic Noise for Preferential Bayesian Optimization. Proceedings of The 1st Symposium on Probabilistic Machine Learning, in Proceedings of Machine Learning Research 327:1-26 Available from https://proceedings.mlr.press/v327/sinaga26a.html.

Related Material