A Maximum Matching Algorithm for Basis Selection in Spectral Learning

Ariadna Quattoni, Xavier Carreras, Matthias Gallé
Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, PMLR 54:1477-1485, 2017.

Abstract

We present a solution to scale spectral algorithms for learning sequence functions. We are interested in the case where these functions are sparse (that is, for most sequences they return 0). Spectral algorithms reduce the learning problem to the task of computing an SVD decomposition over a special type of matrix called the Hankel matrix. This matrix is designed to capture the relevant statistics of the training sequences. What is crucial is that to capture long range dependencies we must consider very large Hankel matrices. Thus the computation of the SVD becomes a critical bottleneck. Our solution finds a subset of rows and columns of the Hankel that realizes a compact and informative Hankel submatrix. The novelty lies in the way that this subset is selected: we exploit a maximal bipartite matching combinatorial algorithm to look for a sub-block with full structural rank, and show how computation of this sub-block can be further improved by exploiting the specific structure of Hankel matrices.

Cite this Paper

BibTeX
@InProceedings{pmlr-v54-quattoni17a, title = {{A Maximum Matching Algorithm for Basis Selection in Spectral Learning}}, author = {Quattoni, Ariadna and Carreras, Xavier and Gallé, Matthias}, booktitle = {Proceedings of the 20th International Conference on Artificial Intelligence and Statistics}, pages = {1477--1485}, year = {2017}, editor = {Singh, Aarti and Zhu, Jerry}, volume = {54}, series = {Proceedings of Machine Learning Research}, month = {20--22 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v54/quattoni17a/quattoni17a.pdf}, url = {https://proceedings.mlr.press/v54/quattoni17a.html}, abstract = {We present a solution to scale spectral algorithms for learning sequence functions. We are interested in the case where these functions are sparse (that is, for most sequences they return 0). Spectral algorithms reduce the learning problem to the task of computing an SVD decomposition over a special type of matrix called the Hankel matrix. This matrix is designed to capture the relevant statistics of the training sequences. What is crucial is that to capture long range dependencies we must consider very large Hankel matrices. Thus the computation of the SVD becomes a critical bottleneck. Our solution finds a subset of rows and columns of the Hankel that realizes a compact and informative Hankel submatrix. The novelty lies in the way that this subset is selected: we exploit a maximal bipartite matching combinatorial algorithm to look for a sub-block with full structural rank, and show how computation of this sub-block can be further improved by exploiting the specific structure of Hankel matrices.} }
Endnote
%0 Conference Paper %T A Maximum Matching Algorithm for Basis Selection in Spectral Learning %A Ariadna Quattoni %A Xavier Carreras %A Matthias Gallé %B Proceedings of the 20th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2017 %E Aarti Singh %E Jerry Zhu %F pmlr-v54-quattoni17a %I PMLR %P 1477--1485 %U https://proceedings.mlr.press/v54/quattoni17a.html %V 54 %X We present a solution to scale spectral algorithms for learning sequence functions. We are interested in the case where these functions are sparse (that is, for most sequences they return 0). Spectral algorithms reduce the learning problem to the task of computing an SVD decomposition over a special type of matrix called the Hankel matrix. This matrix is designed to capture the relevant statistics of the training sequences. What is crucial is that to capture long range dependencies we must consider very large Hankel matrices. Thus the computation of the SVD becomes a critical bottleneck. Our solution finds a subset of rows and columns of the Hankel that realizes a compact and informative Hankel submatrix. The novelty lies in the way that this subset is selected: we exploit a maximal bipartite matching combinatorial algorithm to look for a sub-block with full structural rank, and show how computation of this sub-block can be further improved by exploiting the specific structure of Hankel matrices.
APA
Quattoni, A., Carreras, X. & Gallé, M.. (2017). A Maximum Matching Algorithm for Basis Selection in Spectral Learning. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 54:1477-1485 Available from https://proceedings.mlr.press/v54/quattoni17a.html.

Related Material