Online non-parametric likelihood-ratio estimation by Pearson-divergence functional minimization

Alejandro D. de la Concha Duarte, Nicolas Vayatis, Argyris Kalogeratos
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, PMLR 238:1189-1197, 2024.

Abstract

Quantifying the difference between two probability density functions, $p$ and $q$, using available data, is a fundamental problem in Statistics and Machine Learning. A usual approach for addressing this problem is the likelihood-ratio estimation (LRE) between $p$ and $q$, which -to our best knowledge- has been investigated mainly for the offline case. This paper contributes by introducing a new framework for online non-parametric LRE (OLRE) for the setting where pairs of iid observations $(x_t \sim p, x’_t \sim q)$ are observed over time. The non-parametric nature of our approach has the advantage of being agnostic to the forms of $p$ and $q$. Moreover, we capitalize on the recent advances in Kernel Methods and functional minimization to develop an estimator that can be efficiently updated at every iteration. We provide theoretical guarantees for the performance of the OLRE method along with empirical validation in synthetic experiments.

Cite this Paper

BibTeX
@InProceedings{pmlr-v238-concha-duarte24a, title = {Online non-parametric likelihood-ratio estimation by {P}earson-divergence functional minimization}, author = {de la Concha Duarte, Alejandro D. and Vayatis, Nicolas and Kalogeratos, Argyris}, booktitle = {Proceedings of The 27th International Conference on Artificial Intelligence and Statistics}, pages = {1189--1197}, year = {2024}, editor = {Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen}, volume = {238}, series = {Proceedings of Machine Learning Research}, month = {02--04 May}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v238/concha-duarte24a/concha-duarte24a.pdf}, url = {https://proceedings.mlr.press/v238/concha-duarte24a.html}, abstract = {Quantifying the difference between two probability density functions, $p$ and $q$, using available data, is a fundamental problem in Statistics and Machine Learning. A usual approach for addressing this problem is the likelihood-ratio estimation (LRE) between $p$ and $q$, which -to our best knowledge- has been investigated mainly for the offline case. This paper contributes by introducing a new framework for online non-parametric LRE (OLRE) for the setting where pairs of iid observations $(x_t \sim p, x’_t \sim q)$ are observed over time. The non-parametric nature of our approach has the advantage of being agnostic to the forms of $p$ and $q$. Moreover, we capitalize on the recent advances in Kernel Methods and functional minimization to develop an estimator that can be efficiently updated at every iteration. We provide theoretical guarantees for the performance of the OLRE method along with empirical validation in synthetic experiments.} }
Endnote
%0 Conference Paper %T Online non-parametric likelihood-ratio estimation by Pearson-divergence functional minimization %A Alejandro D. de la Concha Duarte %A Nicolas Vayatis %A Argyris Kalogeratos %B Proceedings of The 27th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2024 %E Sanjoy Dasgupta %E Stephan Mandt %E Yingzhen Li %F pmlr-v238-concha-duarte24a %I PMLR %P 1189--1197 %U https://proceedings.mlr.press/v238/concha-duarte24a.html %V 238 %X Quantifying the difference between two probability density functions, $p$ and $q$, using available data, is a fundamental problem in Statistics and Machine Learning. A usual approach for addressing this problem is the likelihood-ratio estimation (LRE) between $p$ and $q$, which -to our best knowledge- has been investigated mainly for the offline case. This paper contributes by introducing a new framework for online non-parametric LRE (OLRE) for the setting where pairs of iid observations $(x_t \sim p, x’_t \sim q)$ are observed over time. The non-parametric nature of our approach has the advantage of being agnostic to the forms of $p$ and $q$. Moreover, we capitalize on the recent advances in Kernel Methods and functional minimization to develop an estimator that can be efficiently updated at every iteration. We provide theoretical guarantees for the performance of the OLRE method along with empirical validation in synthetic experiments.
APA
de la Concha Duarte, A.D., Vayatis, N. & Kalogeratos, A.. (2024). Online non-parametric likelihood-ratio estimation by Pearson-divergence functional minimization. Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 238:1189-1197 Available from https://proceedings.mlr.press/v238/concha-duarte24a.html.

Related Material