Conditional Density Estimation via Least-Squares Density Ratio Estimation

Masashi Sugiyama, Ichiro Takeuchi, Taiji Suzuki, Takafumi Kanamori, Hirotaka Hachiya, Daisuke Okanohara
Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, PMLR 9:781-788, 2010.

Abstract

Estimating the conditional mean of an input-output relation is the goal of regression. However, regression analysis is not sufficiently informative if the conditional distribution has multi-modality, is highly asymmetric, or contains heteroscedastic noise. In such scenarios, estimating the conditional distribution itself would be more useful. In this paper, we propose a novel method of conditional density estimation that is suitable for multi-dimensional continuous variables. The basic idea of the proposed method is to express the conditional density in terms of the density ratio and the ratio is directly estimated without going through density estimation. Experiments using benchmark and robot transition datasets illustrate the usefulness of the proposed approach.

Cite this Paper

BibTeX
@InProceedings{pmlr-v9-sugiyama10a, title = {Conditional Density Estimation via Least-Squares Density Ratio Estimation}, author = {Sugiyama, Masashi and Takeuchi, Ichiro and Suzuki, Taiji and Kanamori, Takafumi and Hachiya, Hirotaka and Okanohara, Daisuke}, booktitle = {Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics}, pages = {781--788}, year = {2010}, editor = {Teh, Yee Whye and Titterington, Mike}, volume = {9}, series = {Proceedings of Machine Learning Research}, address = {Chia Laguna Resort, Sardinia, Italy}, month = {13--15 May}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v9/sugiyama10a/sugiyama10a.pdf}, url = {https://proceedings.mlr.press/v9/sugiyama10a.html}, abstract = {Estimating the conditional mean of an input-output relation is the goal of regression. However, regression analysis is not sufficiently informative if the conditional distribution has multi-modality, is highly asymmetric, or contains heteroscedastic noise. In such scenarios, estimating the conditional distribution itself would be more useful. In this paper, we propose a novel method of conditional density estimation that is suitable for multi-dimensional continuous variables. The basic idea of the proposed method is to express the conditional density in terms of the density ratio and the ratio is directly estimated without going through density estimation. Experiments using benchmark and robot transition datasets illustrate the usefulness of the proposed approach.} }
Endnote
%0 Conference Paper %T Conditional Density Estimation via Least-Squares Density Ratio Estimation %A Masashi Sugiyama %A Ichiro Takeuchi %A Taiji Suzuki %A Takafumi Kanamori %A Hirotaka Hachiya %A Daisuke Okanohara %B Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2010 %E Yee Whye Teh %E Mike Titterington %F pmlr-v9-sugiyama10a %I PMLR %P 781--788 %U https://proceedings.mlr.press/v9/sugiyama10a.html %V 9 %X Estimating the conditional mean of an input-output relation is the goal of regression. However, regression analysis is not sufficiently informative if the conditional distribution has multi-modality, is highly asymmetric, or contains heteroscedastic noise. In such scenarios, estimating the conditional distribution itself would be more useful. In this paper, we propose a novel method of conditional density estimation that is suitable for multi-dimensional continuous variables. The basic idea of the proposed method is to express the conditional density in terms of the density ratio and the ratio is directly estimated without going through density estimation. Experiments using benchmark and robot transition datasets illustrate the usefulness of the proposed approach.
RIS
TY - CPAPER TI - Conditional Density Estimation via Least-Squares Density Ratio Estimation AU - Masashi Sugiyama AU - Ichiro Takeuchi AU - Taiji Suzuki AU - Takafumi Kanamori AU - Hirotaka Hachiya AU - Daisuke Okanohara BT - Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics DA - 2010/03/31 ED - Yee Whye Teh ED - Mike Titterington ID - pmlr-v9-sugiyama10a PB - PMLR DP - Proceedings of Machine Learning Research VL - 9 SP - 781 EP - 788 L1 - http://proceedings.mlr.press/v9/sugiyama10a/sugiyama10a.pdf UR - https://proceedings.mlr.press/v9/sugiyama10a.html AB - Estimating the conditional mean of an input-output relation is the goal of regression. However, regression analysis is not sufficiently informative if the conditional distribution has multi-modality, is highly asymmetric, or contains heteroscedastic noise. In such scenarios, estimating the conditional distribution itself would be more useful. In this paper, we propose a novel method of conditional density estimation that is suitable for multi-dimensional continuous variables. The basic idea of the proposed method is to express the conditional density in terms of the density ratio and the ratio is directly estimated without going through density estimation. Experiments using benchmark and robot transition datasets illustrate the usefulness of the proposed approach. ER -
APA
Sugiyama, M., Takeuchi, I., Suzuki, T., Kanamori, T., Hachiya, H. & Okanohara, D.. (2010). Conditional Density Estimation via Least-Squares Density Ratio Estimation. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 9:781-788 Available from https://proceedings.mlr.press/v9/sugiyama10a.html.

Related Material