Unifying Clustered and Non-stationary Bandits

Chuanhao Li, Qingyun Wu, Hongning Wang
Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR 130:1063-1071, 2021.

Abstract

Non-stationary bandits and clustered bandits lift the restrictive assumptions in contextual bandits and provide solutions to many important real-world scenarios. Though they have been studied independently so far, we point out the essence in solving these two problems overlaps considerably. In this work, we connect these two strands of bandit research under the notion of test of homogeneity, which seamlessly addresses change detection for non-stationary bandit and cluster identification for clustered bandit in a unified solution framework. Rigorous regret analysis and extensive empirical evaluations demonstrate the value of our proposed solution, especially its flexibility in handling various environment assumptions, e.g., a clustered non-stationary environment.

Cite this Paper

BibTeX
@InProceedings{pmlr-v130-li21c, title = { Unifying Clustered and Non-stationary Bandits }, author = {Li, Chuanhao and Wu, Qingyun and Wang, Hongning}, booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics}, pages = {1063--1071}, year = {2021}, editor = {Banerjee, Arindam and Fukumizu, Kenji}, volume = {130}, series = {Proceedings of Machine Learning Research}, month = {13--15 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v130/li21c/li21c.pdf}, url = {https://proceedings.mlr.press/v130/li21c.html}, abstract = { Non-stationary bandits and clustered bandits lift the restrictive assumptions in contextual bandits and provide solutions to many important real-world scenarios. Though they have been studied independently so far, we point out the essence in solving these two problems overlaps considerably. In this work, we connect these two strands of bandit research under the notion of test of homogeneity, which seamlessly addresses change detection for non-stationary bandit and cluster identification for clustered bandit in a unified solution framework. Rigorous regret analysis and extensive empirical evaluations demonstrate the value of our proposed solution, especially its flexibility in handling various environment assumptions, e.g., a clustered non-stationary environment. } }
Endnote
%0 Conference Paper %T Unifying Clustered and Non-stationary Bandits %A Chuanhao Li %A Qingyun Wu %A Hongning Wang %B Proceedings of The 24th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2021 %E Arindam Banerjee %E Kenji Fukumizu %F pmlr-v130-li21c %I PMLR %P 1063--1071 %U https://proceedings.mlr.press/v130/li21c.html %V 130 %X Non-stationary bandits and clustered bandits lift the restrictive assumptions in contextual bandits and provide solutions to many important real-world scenarios. Though they have been studied independently so far, we point out the essence in solving these two problems overlaps considerably. In this work, we connect these two strands of bandit research under the notion of test of homogeneity, which seamlessly addresses change detection for non-stationary bandit and cluster identification for clustered bandit in a unified solution framework. Rigorous regret analysis and extensive empirical evaluations demonstrate the value of our proposed solution, especially its flexibility in handling various environment assumptions, e.g., a clustered non-stationary environment.
APA
Li, C., Wu, Q. & Wang, H.. (2021). Unifying Clustered and Non-stationary Bandits . Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 130:1063-1071 Available from https://proceedings.mlr.press/v130/li21c.html.

Related Material