Smooth Sparse Coding via Marginal Regression for Learning Sparse Representations

Krishnakumar Balasubramanian, Kai Yu, Guy Lebanon
Proceedings of the 30th International Conference on Machine Learning, PMLR 28(3):289-297, 2013.

Abstract

We propose and analyze a novel framework for learning sparse representations, based on two statistical techniques: kernel smoothing and marginal regression. The proposed approach provides a flexible framework for incorporating feature similarity or temporal information present in data sets, via nonparametric kernel smoothing. We provide generalization bounds for dictionary learning using smooth sparse coding and show how the sample complexity depends on the L1 norm of kernel function used. Furthermore, we propose using marginal regression for obtaining sparse codes, which significantly improves the speed and allows one to scale to large dictionary sizes easily. We demonstrate the advantages of the proposed approach, both in terms of accuracy and speed by extensive experimentation on several real data sets. In addition, we demonstrate how the proposed approach could be used for improving semisupervised sparse coding.

Cite this Paper

BibTeX
@InProceedings{pmlr-v28-balasubramanian13, title = {Smooth Sparse Coding via Marginal Regression for Learning Sparse Representations}, author = {Balasubramanian, Krishnakumar and Yu, Kai and Lebanon, Guy}, booktitle = {Proceedings of the 30th International Conference on Machine Learning}, pages = {289--297}, year = {2013}, editor = {Dasgupta, Sanjoy and McAllester, David}, volume = {28}, number = {3}, series = {Proceedings of Machine Learning Research}, address = {Atlanta, Georgia, USA}, month = {17--19 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v28/balasubramanian13.pdf}, url = {https://proceedings.mlr.press/v28/balasubramanian13.html}, abstract = {We propose and analyze a novel framework for learning sparse representations, based on two statistical techniques: kernel smoothing and marginal regression. The proposed approach provides a flexible framework for incorporating feature similarity or temporal information present in data sets, via nonparametric kernel smoothing. We provide generalization bounds for dictionary learning using smooth sparse coding and show how the sample complexity depends on the L1 norm of kernel function used. Furthermore, we propose using marginal regression for obtaining sparse codes, which significantly improves the speed and allows one to scale to large dictionary sizes easily. We demonstrate the advantages of the proposed approach, both in terms of accuracy and speed by extensive experimentation on several real data sets. In addition, we demonstrate how the proposed approach could be used for improving semisupervised sparse coding.} }
Endnote
%0 Conference Paper %T Smooth Sparse Coding via Marginal Regression for Learning Sparse Representations %A Krishnakumar Balasubramanian %A Kai Yu %A Guy Lebanon %B Proceedings of the 30th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2013 %E Sanjoy Dasgupta %E David McAllester %F pmlr-v28-balasubramanian13 %I PMLR %P 289--297 %U https://proceedings.mlr.press/v28/balasubramanian13.html %V 28 %N 3 %X We propose and analyze a novel framework for learning sparse representations, based on two statistical techniques: kernel smoothing and marginal regression. The proposed approach provides a flexible framework for incorporating feature similarity or temporal information present in data sets, via nonparametric kernel smoothing. We provide generalization bounds for dictionary learning using smooth sparse coding and show how the sample complexity depends on the L1 norm of kernel function used. Furthermore, we propose using marginal regression for obtaining sparse codes, which significantly improves the speed and allows one to scale to large dictionary sizes easily. We demonstrate the advantages of the proposed approach, both in terms of accuracy and speed by extensive experimentation on several real data sets. In addition, we demonstrate how the proposed approach could be used for improving semisupervised sparse coding.
RIS
TY - CPAPER TI - Smooth Sparse Coding via Marginal Regression for Learning Sparse Representations AU - Krishnakumar Balasubramanian AU - Kai Yu AU - Guy Lebanon BT - Proceedings of the 30th International Conference on Machine Learning DA - 2013/05/26 ED - Sanjoy Dasgupta ED - David McAllester ID - pmlr-v28-balasubramanian13 PB - PMLR DP - Proceedings of Machine Learning Research VL - 28 IS - 3 SP - 289 EP - 297 L1 - http://proceedings.mlr.press/v28/balasubramanian13.pdf UR - https://proceedings.mlr.press/v28/balasubramanian13.html AB - We propose and analyze a novel framework for learning sparse representations, based on two statistical techniques: kernel smoothing and marginal regression. The proposed approach provides a flexible framework for incorporating feature similarity or temporal information present in data sets, via nonparametric kernel smoothing. We provide generalization bounds for dictionary learning using smooth sparse coding and show how the sample complexity depends on the L1 norm of kernel function used. Furthermore, we propose using marginal regression for obtaining sparse codes, which significantly improves the speed and allows one to scale to large dictionary sizes easily. We demonstrate the advantages of the proposed approach, both in terms of accuracy and speed by extensive experimentation on several real data sets. In addition, we demonstrate how the proposed approach could be used for improving semisupervised sparse coding. ER -
APA
Balasubramanian, K., Yu, K. & Lebanon, G.. (2013). Smooth Sparse Coding via Marginal Regression for Learning Sparse Representations. Proceedings of the 30th International Conference on Machine Learning, in Proceedings of Machine Learning Research 28(3):289-297 Available from https://proceedings.mlr.press/v28/balasubramanian13.html.

Related Material