Beyond CCA: Moment Matching for Multi-View Models

Anastasia Podosinnikova, Francis Bach, Simon Lacoste-Julien
Proceedings of The 33rd International Conference on Machine Learning, PMLR 48:458-467, 2016.

Abstract

We introduce three novel semi-parametric extensions of probabilistic canonical correlation analysis with identifiability guarantees. We consider moment matching techniques for estimation in these models. For that, by drawing explicit links between the new models and a discrete version of independent component analysis (DICA), we first extend the DICA cumulant tensors to the new discrete version of CCA. By further using a close connection with independent component analysis, we introduce generalized covariance matrices, which can replace the cumulant tensors in the moment matching framework, and, therefore, improve sample complexity and simplify derivations and algorithms significantly. As the tensor power method or orthogonal joint diagonalization are not applicable in the new setting, we use non-orthogonal joint diagonalization techniques for matching the cumulants. We demonstrate performance of the proposed models and estimation techniques on experiments with both synthetic and real datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v48-podosinnikova16, title = {Beyond CCA: Moment Matching for Multi-View Models}, author = {Podosinnikova, Anastasia and Bach, Francis and Lacoste-Julien, Simon}, booktitle = {Proceedings of The 33rd International Conference on Machine Learning}, pages = {458--467}, year = {2016}, editor = {Balcan, Maria Florina and Weinberger, Kilian Q.}, volume = {48}, series = {Proceedings of Machine Learning Research}, address = {New York, New York, USA}, month = {20--22 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v48/podosinnikova16.pdf}, url = {https://proceedings.mlr.press/v48/podosinnikova16.html}, abstract = {We introduce three novel semi-parametric extensions of probabilistic canonical correlation analysis with identifiability guarantees. We consider moment matching techniques for estimation in these models. For that, by drawing explicit links between the new models and a discrete version of independent component analysis (DICA), we first extend the DICA cumulant tensors to the new discrete version of CCA. By further using a close connection with independent component analysis, we introduce generalized covariance matrices, which can replace the cumulant tensors in the moment matching framework, and, therefore, improve sample complexity and simplify derivations and algorithms significantly. As the tensor power method or orthogonal joint diagonalization are not applicable in the new setting, we use non-orthogonal joint diagonalization techniques for matching the cumulants. We demonstrate performance of the proposed models and estimation techniques on experiments with both synthetic and real datasets.} }
Endnote
%0 Conference Paper %T Beyond CCA: Moment Matching for Multi-View Models %A Anastasia Podosinnikova %A Francis Bach %A Simon Lacoste-Julien %B Proceedings of The 33rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2016 %E Maria Florina Balcan %E Kilian Q. Weinberger %F pmlr-v48-podosinnikova16 %I PMLR %P 458--467 %U https://proceedings.mlr.press/v48/podosinnikova16.html %V 48 %X We introduce three novel semi-parametric extensions of probabilistic canonical correlation analysis with identifiability guarantees. We consider moment matching techniques for estimation in these models. For that, by drawing explicit links between the new models and a discrete version of independent component analysis (DICA), we first extend the DICA cumulant tensors to the new discrete version of CCA. By further using a close connection with independent component analysis, we introduce generalized covariance matrices, which can replace the cumulant tensors in the moment matching framework, and, therefore, improve sample complexity and simplify derivations and algorithms significantly. As the tensor power method or orthogonal joint diagonalization are not applicable in the new setting, we use non-orthogonal joint diagonalization techniques for matching the cumulants. We demonstrate performance of the proposed models and estimation techniques on experiments with both synthetic and real datasets.
RIS
TY - CPAPER TI - Beyond CCA: Moment Matching for Multi-View Models AU - Anastasia Podosinnikova AU - Francis Bach AU - Simon Lacoste-Julien BT - Proceedings of The 33rd International Conference on Machine Learning DA - 2016/06/11 ED - Maria Florina Balcan ED - Kilian Q. Weinberger ID - pmlr-v48-podosinnikova16 PB - PMLR DP - Proceedings of Machine Learning Research VL - 48 SP - 458 EP - 467 L1 - http://proceedings.mlr.press/v48/podosinnikova16.pdf UR - https://proceedings.mlr.press/v48/podosinnikova16.html AB - We introduce three novel semi-parametric extensions of probabilistic canonical correlation analysis with identifiability guarantees. We consider moment matching techniques for estimation in these models. For that, by drawing explicit links between the new models and a discrete version of independent component analysis (DICA), we first extend the DICA cumulant tensors to the new discrete version of CCA. By further using a close connection with independent component analysis, we introduce generalized covariance matrices, which can replace the cumulant tensors in the moment matching framework, and, therefore, improve sample complexity and simplify derivations and algorithms significantly. As the tensor power method or orthogonal joint diagonalization are not applicable in the new setting, we use non-orthogonal joint diagonalization techniques for matching the cumulants. We demonstrate performance of the proposed models and estimation techniques on experiments with both synthetic and real datasets. ER -
APA
Podosinnikova, A., Bach, F. & Lacoste-Julien, S.. (2016). Beyond CCA: Moment Matching for Multi-View Models. Proceedings of The 33rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 48:458-467 Available from https://proceedings.mlr.press/v48/podosinnikova16.html.

Related Material