Learning Exponential Families in High-Dimensions: Strong Convexity and Sparsity

Sham Kakade, Ohad Shamir, Karthik Sindharan, Ambuj Tewari
; Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, JMLR Workshop and Conference Proceedings 9:381-388, 2010.

Abstract

The versatility of exponential families, along with their attendant convexity properties, make them a popular and effective statistical model. A central issue is learning these models in high-dimensions when the optimal parameter vector is sparse. This work characterizes a certain strong convexity property of general exponential families, which allows their generalization ability to be quantified. In particular, we show how this property can be used to analyze generic exponential families under L1 regularization.

Cite this Paper


BibTeX
@InProceedings{pmlr-v9-kakade10a, title = {Learning Exponential Families in High-Dimensions: Strong Convexity and Sparsity}, author = {Sham Kakade and Ohad Shamir and Karthik Sindharan and Ambuj Tewari}, booktitle = {Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics}, pages = {381--388}, year = {2010}, editor = {Yee Whye Teh and Mike Titterington}, volume = {9}, series = {Proceedings of Machine Learning Research}, address = {Chia Laguna Resort, Sardinia, Italy}, month = {13--15 May}, publisher = {JMLR Workshop and Conference Proceedings}, pdf = {http://proceedings.mlr.press/v9/kakade10a/kakade10a.pdf}, url = {http://proceedings.mlr.press/v9/kakade10a.html}, abstract = {The versatility of exponential families, along with their attendant convexity properties, make them a popular and effective statistical model. A central issue is learning these models in high-dimensions when the optimal parameter vector is sparse. This work characterizes a certain strong convexity property of general exponential families, which allows their generalization ability to be quantified. In particular, we show how this property can be used to analyze generic exponential families under L1 regularization.} }
Endnote
%0 Conference Paper %T Learning Exponential Families in High-Dimensions: Strong Convexity and Sparsity %A Sham Kakade %A Ohad Shamir %A Karthik Sindharan %A Ambuj Tewari %B Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2010 %E Yee Whye Teh %E Mike Titterington %F pmlr-v9-kakade10a %I PMLR %J Proceedings of Machine Learning Research %P 381--388 %U http://proceedings.mlr.press %V 9 %W PMLR %X The versatility of exponential families, along with their attendant convexity properties, make them a popular and effective statistical model. A central issue is learning these models in high-dimensions when the optimal parameter vector is sparse. This work characterizes a certain strong convexity property of general exponential families, which allows their generalization ability to be quantified. In particular, we show how this property can be used to analyze generic exponential families under L1 regularization.
RIS
TY - CPAPER TI - Learning Exponential Families in High-Dimensions: Strong Convexity and Sparsity AU - Sham Kakade AU - Ohad Shamir AU - Karthik Sindharan AU - Ambuj Tewari BT - Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics PY - 2010/03/31 DA - 2010/03/31 ED - Yee Whye Teh ED - Mike Titterington ID - pmlr-v9-kakade10a PB - PMLR SP - 381 DP - PMLR EP - 388 L1 - http://proceedings.mlr.press/v9/kakade10a/kakade10a.pdf UR - http://proceedings.mlr.press/v9/kakade10a.html AB - The versatility of exponential families, along with their attendant convexity properties, make them a popular and effective statistical model. A central issue is learning these models in high-dimensions when the optimal parameter vector is sparse. This work characterizes a certain strong convexity property of general exponential families, which allows their generalization ability to be quantified. In particular, we show how this property can be used to analyze generic exponential families under L1 regularization. ER -
APA
Kakade, S., Shamir, O., Sindharan, K. & Tewari, A.. (2010). Learning Exponential Families in High-Dimensions: Strong Convexity and Sparsity. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, in PMLR 9:381-388

Related Material