Quantifying Availability and Discovery in Recommender Systems via Stochastic Reachability

Mihaela Curmei, Sarah Dean, Benjamin Recht
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:2265-2275, 2021.

Abstract

In this work, we consider how preference models in interactive recommendation systems determine the availability of content and users’ opportunities for discovery. We propose an evaluation procedure based on stochastic reachability to quantify the maximum probability of recommending a target piece of content to an user for a set of allowable strategic modifications. This framework allows us to compute an upper bound on the likelihood of recommendation with minimal assumptions about user behavior. Stochastic reachability can be used to detect biases in the availability of content and diagnose limitations in the opportunities for discovery granted to users. We show that this metric can be computed efficiently as a convex program for a variety of practical settings, and further argue that reachability is not inherently at odds with accuracy. We demonstrate evaluations of recommendation algorithms trained on large datasets of explicit and implicit ratings. Our results illustrate how preference models, selection rules, and user interventions impact reachability and how these effects can be distributed unevenly.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-curmei21a, title = {Quantifying Availability and Discovery in Recommender Systems via Stochastic Reachability}, author = {Curmei, Mihaela and Dean, Sarah and Recht, Benjamin}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {2265--2275}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/curmei21a/curmei21a.pdf}, url = {https://proceedings.mlr.press/v139/curmei21a.html}, abstract = {In this work, we consider how preference models in interactive recommendation systems determine the availability of content and users’ opportunities for discovery. We propose an evaluation procedure based on stochastic reachability to quantify the maximum probability of recommending a target piece of content to an user for a set of allowable strategic modifications. This framework allows us to compute an upper bound on the likelihood of recommendation with minimal assumptions about user behavior. Stochastic reachability can be used to detect biases in the availability of content and diagnose limitations in the opportunities for discovery granted to users. We show that this metric can be computed efficiently as a convex program for a variety of practical settings, and further argue that reachability is not inherently at odds with accuracy. We demonstrate evaluations of recommendation algorithms trained on large datasets of explicit and implicit ratings. Our results illustrate how preference models, selection rules, and user interventions impact reachability and how these effects can be distributed unevenly.} }
Endnote
%0 Conference Paper %T Quantifying Availability and Discovery in Recommender Systems via Stochastic Reachability %A Mihaela Curmei %A Sarah Dean %A Benjamin Recht %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-curmei21a %I PMLR %P 2265--2275 %U https://proceedings.mlr.press/v139/curmei21a.html %V 139 %X In this work, we consider how preference models in interactive recommendation systems determine the availability of content and users’ opportunities for discovery. We propose an evaluation procedure based on stochastic reachability to quantify the maximum probability of recommending a target piece of content to an user for a set of allowable strategic modifications. This framework allows us to compute an upper bound on the likelihood of recommendation with minimal assumptions about user behavior. Stochastic reachability can be used to detect biases in the availability of content and diagnose limitations in the opportunities for discovery granted to users. We show that this metric can be computed efficiently as a convex program for a variety of practical settings, and further argue that reachability is not inherently at odds with accuracy. We demonstrate evaluations of recommendation algorithms trained on large datasets of explicit and implicit ratings. Our results illustrate how preference models, selection rules, and user interventions impact reachability and how these effects can be distributed unevenly.
APA
Curmei, M., Dean, S. & Recht, B.. (2021). Quantifying Availability and Discovery in Recommender Systems via Stochastic Reachability. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:2265-2275 Available from https://proceedings.mlr.press/v139/curmei21a.html.

Related Material