Propensity Matters: Measuring and Enhancing Balancing for Recommendation

Haoxuan Li, Yanghao Xiao, Chunyuan Zheng, Peng Wu, Peng Cui
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:20182-20194, 2023.

Abstract

Propensity-based weighting methods have been widely studied and demonstrated competitive performance in debiased recommendations. Nevertheless, there are still many questions to be addressed. How to estimate the propensity more conducive to debiasing performance? Which metric is more reasonable to measure the quality of the learned propensities? Is it better to make the cross-entropy loss as small as possible when learning propensities? In this paper, we first discuss the potential problems of the previously widely adopted metrics for learned propensities, and propose balanced-mean-squared-error (BMSE) metric for debiased recommendations. Based on BMSE, we propose IPS-V2 and DR-V2 as the estimators of unbiased loss, and theoretically show that IPS-V2 and DR-V2 have greater propensity balancing and smaller variance without sacrificing additional bias. We further propose a co-training method for learning balanced representation and unbiased prediction. Extensive experiments are conducted on three real-world datasets including a large industrial dataset, and the results show that our approach boosts the balancing property and results in enhanced debiasing performance.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-li23ah, title = {Propensity Matters: Measuring and Enhancing Balancing for Recommendation}, author = {Li, Haoxuan and Xiao, Yanghao and Zheng, Chunyuan and Wu, Peng and Cui, Peng}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {20182--20194}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/li23ah/li23ah.pdf}, url = {https://proceedings.mlr.press/v202/li23ah.html}, abstract = {Propensity-based weighting methods have been widely studied and demonstrated competitive performance in debiased recommendations. Nevertheless, there are still many questions to be addressed. How to estimate the propensity more conducive to debiasing performance? Which metric is more reasonable to measure the quality of the learned propensities? Is it better to make the cross-entropy loss as small as possible when learning propensities? In this paper, we first discuss the potential problems of the previously widely adopted metrics for learned propensities, and propose balanced-mean-squared-error (BMSE) metric for debiased recommendations. Based on BMSE, we propose IPS-V2 and DR-V2 as the estimators of unbiased loss, and theoretically show that IPS-V2 and DR-V2 have greater propensity balancing and smaller variance without sacrificing additional bias. We further propose a co-training method for learning balanced representation and unbiased prediction. Extensive experiments are conducted on three real-world datasets including a large industrial dataset, and the results show that our approach boosts the balancing property and results in enhanced debiasing performance.} }
Endnote
%0 Conference Paper %T Propensity Matters: Measuring and Enhancing Balancing for Recommendation %A Haoxuan Li %A Yanghao Xiao %A Chunyuan Zheng %A Peng Wu %A Peng Cui %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-li23ah %I PMLR %P 20182--20194 %U https://proceedings.mlr.press/v202/li23ah.html %V 202 %X Propensity-based weighting methods have been widely studied and demonstrated competitive performance in debiased recommendations. Nevertheless, there are still many questions to be addressed. How to estimate the propensity more conducive to debiasing performance? Which metric is more reasonable to measure the quality of the learned propensities? Is it better to make the cross-entropy loss as small as possible when learning propensities? In this paper, we first discuss the potential problems of the previously widely adopted metrics for learned propensities, and propose balanced-mean-squared-error (BMSE) metric for debiased recommendations. Based on BMSE, we propose IPS-V2 and DR-V2 as the estimators of unbiased loss, and theoretically show that IPS-V2 and DR-V2 have greater propensity balancing and smaller variance without sacrificing additional bias. We further propose a co-training method for learning balanced representation and unbiased prediction. Extensive experiments are conducted on three real-world datasets including a large industrial dataset, and the results show that our approach boosts the balancing property and results in enhanced debiasing performance.
APA
Li, H., Xiao, Y., Zheng, C., Wu, P. & Cui, P.. (2023). Propensity Matters: Measuring and Enhancing Balancing for Recommendation. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:20182-20194 Available from https://proceedings.mlr.press/v202/li23ah.html.

Related Material