Cold-Start Personalization via Bayesian Adaptive Questioning

Avinandan Bose, Shuyue Stella Li, Faeze Brahman, Pang Wei Koh, Simon Shaolei Du, Yulia Tsvetkov, Maryam Fazel, Lin Xiao, Asli Celikyilmaz
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:9295-9321, 2026.

Abstract

Cold-start personalization requires inferring preferences from minimal interaction when no user-specific historical data is available. The space of possible preferences is vast, yet users care about only a sparse subset and rarely articulate them upfront; combined with limited interaction budgets, this makes preference elicitation challenging. Our key insight is that preferences exhibit predictable structure across populations; e.g., users who want detailed explanations often also value worked examples. We propose CAPE (Cold start Adaptive Preference Elicitation with Priors), a principled system decomposition framework for cold-start personalization: learning a structured world model of preference correlations offline using latent variables, then performing Bayesian inference online without retraining. Even simple belief model instantiations (e.g., linear regression) substantially outperform end-to-end RL. Across medical, mathematical, social, and commonsense reasoning, CAPE achieves 80.8% alignment with ground-truth user preferences versus 68.5% for RL, requires 3-5$\times$ fewer interactions, and adapts twice as often. Our contribution is a principled decomposition of cold-start personalization that makes Bayesian preference elicitation practical at scale for LLM systems.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-bose26a, title = {Cold-Start Personalization via {B}ayesian Adaptive Questioning}, author = {Bose, Avinandan and Li, Shuyue Stella and Brahman, Faeze and Koh, Pang Wei and Du, Simon Shaolei and Tsvetkov, Yulia and Fazel, Maryam and Xiao, Lin and Celikyilmaz, Asli}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {9295--9321}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/bose26a/bose26a.pdf}, url = {https://proceedings.mlr.press/v306/bose26a.html}, abstract = {Cold-start personalization requires inferring preferences from minimal interaction when no user-specific historical data is available. The space of possible preferences is vast, yet users care about only a sparse subset and rarely articulate them upfront; combined with limited interaction budgets, this makes preference elicitation challenging. Our key insight is that preferences exhibit predictable structure across populations; e.g., users who want detailed explanations often also value worked examples. We propose CAPE (Cold start Adaptive Preference Elicitation with Priors), a principled system decomposition framework for cold-start personalization: learning a structured world model of preference correlations offline using latent variables, then performing Bayesian inference online without retraining. Even simple belief model instantiations (e.g., linear regression) substantially outperform end-to-end RL. Across medical, mathematical, social, and commonsense reasoning, CAPE achieves 80.8% alignment with ground-truth user preferences versus 68.5% for RL, requires 3-5$\times$ fewer interactions, and adapts twice as often. Our contribution is a principled decomposition of cold-start personalization that makes Bayesian preference elicitation practical at scale for LLM systems.} }
Endnote
%0 Conference Paper %T Cold-Start Personalization via Bayesian Adaptive Questioning %A Avinandan Bose %A Shuyue Stella Li %A Faeze Brahman %A Pang Wei Koh %A Simon Shaolei Du %A Yulia Tsvetkov %A Maryam Fazel %A Lin Xiao %A Asli Celikyilmaz %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-bose26a %I PMLR %P 9295--9321 %U https://proceedings.mlr.press/v306/bose26a.html %V 306 %X Cold-start personalization requires inferring preferences from minimal interaction when no user-specific historical data is available. The space of possible preferences is vast, yet users care about only a sparse subset and rarely articulate them upfront; combined with limited interaction budgets, this makes preference elicitation challenging. Our key insight is that preferences exhibit predictable structure across populations; e.g., users who want detailed explanations often also value worked examples. We propose CAPE (Cold start Adaptive Preference Elicitation with Priors), a principled system decomposition framework for cold-start personalization: learning a structured world model of preference correlations offline using latent variables, then performing Bayesian inference online without retraining. Even simple belief model instantiations (e.g., linear regression) substantially outperform end-to-end RL. Across medical, mathematical, social, and commonsense reasoning, CAPE achieves 80.8% alignment with ground-truth user preferences versus 68.5% for RL, requires 3-5$\times$ fewer interactions, and adapts twice as often. Our contribution is a principled decomposition of cold-start personalization that makes Bayesian preference elicitation practical at scale for LLM systems.
APA
Bose, A., Li, S.S., Brahman, F., Koh, P.W., Du, S.S., Tsvetkov, Y., Fazel, M., Xiao, L. & Celikyilmaz, A.. (2026). Cold-Start Personalization via Bayesian Adaptive Questioning. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:9295-9321 Available from https://proceedings.mlr.press/v306/bose26a.html.

Related Material