Latent Feature Lasso

Ian En-Hsu Yen, Wei-Cheng Lee, Sung-En Chang, Arun Sai Suggala, Shou-De Lin, Pradeep Ravikumar
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:3949-3957, 2017.

Abstract

The latent feature model (LFM), proposed in (Griffiths \& Ghahramani, 2005), but possibly with earlier origins, is a generalization of a mixture model, where each instance is generated not from a single latent class but from a combination of latent features. Thus, each instance has an associated latent binary feature incidence vector indicating the presence or absence of a feature. Due to its combinatorial nature, inference of LFMs is considerably intractable, and accordingly, most of the attention has focused on nonparametric LFMs, with priors such as the Indian Buffet Process (IBP) on infinite binary matrices. Recent efforts to tackle this complexity either still have computational complexity that is exponential, or sample complexity that is high-order polynomial w.r.t. the number of latent features. In this paper, we address this outstanding problem of tractable estimation of LFMs via a novel atomic-norm regularization, which gives an algorithm with polynomial run-time and sample complexity without impractical assumptions on the data distribution.

Cite this Paper

BibTeX
@InProceedings{pmlr-v70-yen17a, title = {Latent Feature Lasso}, author = {Ian En-Hsu Yen and Wei-Cheng Lee and Sung-En Chang and Arun Sai Suggala and Shou-De Lin and Pradeep Ravikumar}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {3949--3957}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, month = {06--11 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/yen17a/yen17a.pdf}, url = {https://proceedings.mlr.press/v70/yen17a.html}, abstract = {The latent feature model (LFM), proposed in (Griffiths \& Ghahramani, 2005), but possibly with earlier origins, is a generalization of a mixture model, where each instance is generated not from a single latent class but from a combination of latent features. Thus, each instance has an associated latent binary feature incidence vector indicating the presence or absence of a feature. Due to its combinatorial nature, inference of LFMs is considerably intractable, and accordingly, most of the attention has focused on nonparametric LFMs, with priors such as the Indian Buffet Process (IBP) on infinite binary matrices. Recent efforts to tackle this complexity either still have computational complexity that is exponential, or sample complexity that is high-order polynomial w.r.t. the number of latent features. In this paper, we address this outstanding problem of tractable estimation of LFMs via a novel atomic-norm regularization, which gives an algorithm with polynomial run-time and sample complexity without impractical assumptions on the data distribution.} }
Endnote
%0 Conference Paper %T Latent Feature Lasso %A Ian En-Hsu Yen %A Wei-Cheng Lee %A Sung-En Chang %A Arun Sai Suggala %A Shou-De Lin %A Pradeep Ravikumar %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F pmlr-v70-yen17a %I PMLR %P 3949--3957 %U https://proceedings.mlr.press/v70/yen17a.html %V 70 %X The latent feature model (LFM), proposed in (Griffiths \& Ghahramani, 2005), but possibly with earlier origins, is a generalization of a mixture model, where each instance is generated not from a single latent class but from a combination of latent features. Thus, each instance has an associated latent binary feature incidence vector indicating the presence or absence of a feature. Due to its combinatorial nature, inference of LFMs is considerably intractable, and accordingly, most of the attention has focused on nonparametric LFMs, with priors such as the Indian Buffet Process (IBP) on infinite binary matrices. Recent efforts to tackle this complexity either still have computational complexity that is exponential, or sample complexity that is high-order polynomial w.r.t. the number of latent features. In this paper, we address this outstanding problem of tractable estimation of LFMs via a novel atomic-norm regularization, which gives an algorithm with polynomial run-time and sample complexity without impractical assumptions on the data distribution.
APA
Yen, I.E., Lee, W., Chang, S., Suggala, A.S., Lin, S. & Ravikumar, P.. (2017). Latent Feature Lasso. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:3949-3957 Available from https://proceedings.mlr.press/v70/yen17a.html.

Related Material