Learning Exponential Families in High-Dimensions: Strong Convexity and Sparsity

Sham Kakade, Ohad Shamir, Karthik Sindharan, Ambuj Tewari
Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, PMLR 9:381-388, 2010.

Abstract

The versatility of exponential families, along with their attendant convexity properties, make them a popular and effective statistical model. A central issue is learning these models in high-dimensions when the optimal parameter vector is sparse. This work characterizes a certain strong convexity property of general exponential families, which allows their generalization ability to be quantified. In particular, we show how this property can be used to analyze generic exponential families under L1 regularization.

Cite this Paper

BibTeX
@InProceedings{pmlr-v9-kakade10a, title = {Learning Exponential Families in High-Dimensions: Strong Convexity and Sparsity}, author = {Kakade, Sham and Shamir, Ohad and Sindharan, Karthik and Tewari, Ambuj}, booktitle = {Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics}, pages = {381--388}, year = {2010}, editor = {Teh, Yee Whye and Titterington, Mike}, volume = {9}, series = {Proceedings of Machine Learning Research}, address = {Chia Laguna Resort, Sardinia, Italy}, month = {13--15 May}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v9/kakade10a/kakade10a.pdf}, url = {https://proceedings.mlr.press/v9/kakade10a.html}, abstract = {The versatility of exponential families, along with their attendant convexity properties, make them a popular and effective statistical model. A central issue is learning these models in high-dimensions when the optimal parameter vector is sparse. This work characterizes a certain strong convexity property of general exponential families, which allows their generalization ability to be quantified. In particular, we show how this property can be used to analyze generic exponential families under L1 regularization.} }
Endnote
%0 Conference Paper %T Learning Exponential Families in High-Dimensions: Strong Convexity and Sparsity %A Sham Kakade %A Ohad Shamir %A Karthik Sindharan %A Ambuj Tewari %B Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2010 %E Yee Whye Teh %E Mike Titterington %F pmlr-v9-kakade10a %I PMLR %P 381--388 %U https://proceedings.mlr.press/v9/kakade10a.html %V 9 %X The versatility of exponential families, along with their attendant convexity properties, make them a popular and effective statistical model. A central issue is learning these models in high-dimensions when the optimal parameter vector is sparse. This work characterizes a certain strong convexity property of general exponential families, which allows their generalization ability to be quantified. In particular, we show how this property can be used to analyze generic exponential families under L1 regularization.
RIS
TY - CPAPER TI - Learning Exponential Families in High-Dimensions: Strong Convexity and Sparsity AU - Sham Kakade AU - Ohad Shamir AU - Karthik Sindharan AU - Ambuj Tewari BT - Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics DA - 2010/03/31 ED - Yee Whye Teh ED - Mike Titterington ID - pmlr-v9-kakade10a PB - PMLR DP - Proceedings of Machine Learning Research VL - 9 SP - 381 EP - 388 L1 - http://proceedings.mlr.press/v9/kakade10a/kakade10a.pdf UR - https://proceedings.mlr.press/v9/kakade10a.html AB - The versatility of exponential families, along with their attendant convexity properties, make them a popular and effective statistical model. A central issue is learning these models in high-dimensions when the optimal parameter vector is sparse. This work characterizes a certain strong convexity property of general exponential families, which allows their generalization ability to be quantified. In particular, we show how this property can be used to analyze generic exponential families under L1 regularization. ER -
APA
Kakade, S., Shamir, O., Sindharan, K. & Tewari, A.. (2010). Learning Exponential Families in High-Dimensions: Strong Convexity and Sparsity. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 9:381-388 Available from https://proceedings.mlr.press/v9/kakade10a.html.

Related Material