Contextual Linear Bandits under Noisy Features: Towards Bayesian Oracles

Jung-Hun Kim, Se-Young Yun, Minchan Jeong, Junhyun Nam, Jinwoo Shin, Richard Combes
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:1624-1645, 2023.

Abstract

We study contextual linear bandit problems under feature uncertainty; they are noisy with missing entries. To address the challenges of the noise, we analyze Bayesian oracles given observed noisy features. Our Bayesian analysis finds that the optimal hypothesis can be far from the underlying realizability function, depending on the noise characteristics, which are highly non-intuitive and do not occur for classical noiseless setups. This implies that classical approaches cannot guarantee a non-trivial regret bound. Therefore, we propose an algorithm that aims at the Bayesian oracle from observed information under this model, achieving $\tilde{O}(d\sqrt{T})$ regret bound when there is a large number of arms. We demonstrate the proposed algorithm using synthetic and real-world datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v206-kim23b, title = {Contextual Linear Bandits under Noisy Features: Towards Bayesian Oracles}, author = {Kim, Jung-Hun and Yun, Se-Young and Jeong, Minchan and Nam, Junhyun and Shin, Jinwoo and Combes, Richard}, booktitle = {Proceedings of The 26th International Conference on Artificial Intelligence and Statistics}, pages = {1624--1645}, year = {2023}, editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}, volume = {206}, series = {Proceedings of Machine Learning Research}, month = {25--27 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v206/kim23b/kim23b.pdf}, url = {https://proceedings.mlr.press/v206/kim23b.html}, abstract = {We study contextual linear bandit problems under feature uncertainty; they are noisy with missing entries. To address the challenges of the noise, we analyze Bayesian oracles given observed noisy features. Our Bayesian analysis finds that the optimal hypothesis can be far from the underlying realizability function, depending on the noise characteristics, which are highly non-intuitive and do not occur for classical noiseless setups. This implies that classical approaches cannot guarantee a non-trivial regret bound. Therefore, we propose an algorithm that aims at the Bayesian oracle from observed information under this model, achieving $\tilde{O}(d\sqrt{T})$ regret bound when there is a large number of arms. We demonstrate the proposed algorithm using synthetic and real-world datasets.} }
Endnote
%0 Conference Paper %T Contextual Linear Bandits under Noisy Features: Towards Bayesian Oracles %A Jung-Hun Kim %A Se-Young Yun %A Minchan Jeong %A Junhyun Nam %A Jinwoo Shin %A Richard Combes %B Proceedings of The 26th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2023 %E Francisco Ruiz %E Jennifer Dy %E Jan-Willem van de Meent %F pmlr-v206-kim23b %I PMLR %P 1624--1645 %U https://proceedings.mlr.press/v206/kim23b.html %V 206 %X We study contextual linear bandit problems under feature uncertainty; they are noisy with missing entries. To address the challenges of the noise, we analyze Bayesian oracles given observed noisy features. Our Bayesian analysis finds that the optimal hypothesis can be far from the underlying realizability function, depending on the noise characteristics, which are highly non-intuitive and do not occur for classical noiseless setups. This implies that classical approaches cannot guarantee a non-trivial regret bound. Therefore, we propose an algorithm that aims at the Bayesian oracle from observed information under this model, achieving $\tilde{O}(d\sqrt{T})$ regret bound when there is a large number of arms. We demonstrate the proposed algorithm using synthetic and real-world datasets.
APA
Kim, J., Yun, S., Jeong, M., Nam, J., Shin, J. & Combes, R.. (2023). Contextual Linear Bandits under Noisy Features: Towards Bayesian Oracles. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 206:1624-1645 Available from https://proceedings.mlr.press/v206/kim23b.html.

Related Material