Max-Utility Based Arm Selection Strategy For Sequential Query Recommendations

Shameem Puthiya Parambath, Christos Anagnostopoulos, Roderick Murray-Smith, Sean MacAvaney, Evangelos and Zervas
Proceedings of The 13th Asian Conference on Machine Learning, PMLR 157:564-579, 2021.

Abstract

We consider the query recommendation problem in closed loop interactive learning settings like online information gathering and exploratory analytics. The problem can be naturally modelled using the Multi-Armed Bandits (MAB) framework with countably many arms. The standard MAB algorithms for countably many arms begin with selecting a random set of candidate arms and then applying standard MAB algorithms, e.g., UCB, on this candidate set downstream. We show that such a selection strategy often results in higher cumulative regret and to this end, we propose a selection strategy based on the maximum utility of the arms. We show that in tasks like online information gathering, where sequential query recommendations are employed, the sequences of queries are correlated and the number of potentially optimal queries can be reduced to a manageable size by selecting queries with maximum utility with respect to the currently executing query. Our experimental results using a recent real online literature discovery service log file demonstrate that the proposed arm selection strategy improves the cumulative regret substantially with respect to the state-of-the-art baseline algorithms. Our data model and source code are available at  \url{https://anonymous.4open.science/r/0e5ad6b7-ac02-4577-9212-c9d505d3dbdb/}

Cite this Paper


BibTeX
@InProceedings{pmlr-v157-puthiya-parambath21a, title = {Max-Utility Based Arm Selection Strategy For Sequential Query Recommendations}, author = {Puthiya Parambath, Shameem and Anagnostopoulos, Christos and Murray-Smith, Roderick and MacAvaney, Sean and and Zervas, Evangelos}, booktitle = {Proceedings of The 13th Asian Conference on Machine Learning}, pages = {564--579}, year = {2021}, editor = {Balasubramanian, Vineeth N. and Tsang, Ivor}, volume = {157}, series = {Proceedings of Machine Learning Research}, month = {17--19 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v157/puthiya-parambath21a/puthiya-parambath21a.pdf}, url = {https://proceedings.mlr.press/v157/puthiya-parambath21a.html}, abstract = {We consider the query recommendation problem in closed loop interactive learning settings like online information gathering and exploratory analytics. The problem can be naturally modelled using the Multi-Armed Bandits (MAB) framework with countably many arms. The standard MAB algorithms for countably many arms begin with selecting a random set of candidate arms and then applying standard MAB algorithms, e.g., UCB, on this candidate set downstream. We show that such a selection strategy often results in higher cumulative regret and to this end, we propose a selection strategy based on the maximum utility of the arms. We show that in tasks like online information gathering, where sequential query recommendations are employed, the sequences of queries are correlated and the number of potentially optimal queries can be reduced to a manageable size by selecting queries with maximum utility with respect to the currently executing query. Our experimental results using a recent real online literature discovery service log file demonstrate that the proposed arm selection strategy improves the cumulative regret substantially with respect to the state-of-the-art baseline algorithms. Our data model and source code are available at  \url{https://anonymous.4open.science/r/0e5ad6b7-ac02-4577-9212-c9d505d3dbdb/}} }
Endnote
%0 Conference Paper %T Max-Utility Based Arm Selection Strategy For Sequential Query Recommendations %A Shameem Puthiya Parambath %A Christos Anagnostopoulos %A Roderick Murray-Smith %A Sean MacAvaney %A Evangelos and Zervas %B Proceedings of The 13th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Vineeth N. Balasubramanian %E Ivor Tsang %F pmlr-v157-puthiya-parambath21a %I PMLR %P 564--579 %U https://proceedings.mlr.press/v157/puthiya-parambath21a.html %V 157 %X We consider the query recommendation problem in closed loop interactive learning settings like online information gathering and exploratory analytics. The problem can be naturally modelled using the Multi-Armed Bandits (MAB) framework with countably many arms. The standard MAB algorithms for countably many arms begin with selecting a random set of candidate arms and then applying standard MAB algorithms, e.g., UCB, on this candidate set downstream. We show that such a selection strategy often results in higher cumulative regret and to this end, we propose a selection strategy based on the maximum utility of the arms. We show that in tasks like online information gathering, where sequential query recommendations are employed, the sequences of queries are correlated and the number of potentially optimal queries can be reduced to a manageable size by selecting queries with maximum utility with respect to the currently executing query. Our experimental results using a recent real online literature discovery service log file demonstrate that the proposed arm selection strategy improves the cumulative regret substantially with respect to the state-of-the-art baseline algorithms. Our data model and source code are available at  \url{https://anonymous.4open.science/r/0e5ad6b7-ac02-4577-9212-c9d505d3dbdb/}
APA
Puthiya Parambath, S., Anagnostopoulos, C., Murray-Smith, R., MacAvaney, S. & and Zervas, E.. (2021). Max-Utility Based Arm Selection Strategy For Sequential Query Recommendations. Proceedings of The 13th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 157:564-579 Available from https://proceedings.mlr.press/v157/puthiya-parambath21a.html.

Related Material