Towards a Theoretical Understanding of Negative Transfer in Collective Matrix Factorization

Chao Lan, Jianxin Wang, Jun Huan
Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence, PMLR R14:622-631, 2016.

Abstract

Collective matrix factorization (CMF) is a popular technique to improve the overall factorization quality of multiple matrices presuming they share the same latent factor. However, it suffers from performance degeneration when this assumption fails, an effect called negative transfer (n.t.). Although the effect is widely admitted, its theoretical nature remains a mystery to date. This paper presents a first theoretical understanding of n.t. in theory. Under the statistical mini-max framework, we derive lower bounds for the CMF estimator and gain two insights. First, the n.t. effect can be explained as the rise of a bias term in the standard lower bound, which depends only on the structure of factor space but neither the estimator nor samples. Second, the n.t. effect can be explained as the rise of an d_{th}-root function on the learning rate, where d is the dimension of a Grassmannian containing the subspaces spanned by latent factors. These discoveries are also supported in simulation, and suggest n.t. may be more effectively addressed via model construction other than model selection.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR14-lan16a, title = {Towards a Theoretical Understanding of Negative Transfer in Collective Matrix Factorization}, author = {Lan, Chao and Wang, Jianxin and Huan, Jun}, booktitle = {Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence}, pages = {622--631}, year = {2016}, editor = {Ihler, Alexander and Janzing, Dominik}, volume = {R14}, series = {Proceedings of Machine Learning Research}, month = {25--29 Jun}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r14/main/assets/lan16a/lan16a.pdf}, url = {https://proceedings.mlr.press/r14/lan16a.html}, abstract = {Collective matrix factorization (CMF) is a popular technique to improve the overall factorization quality of multiple matrices presuming they share the same latent factor. However, it suffers from performance degeneration when this assumption fails, an effect called negative transfer (n.t.). Although the effect is widely admitted, its theoretical nature remains a mystery to date. This paper presents a first theoretical understanding of n.t. in theory. Under the statistical mini-max framework, we derive lower bounds for the CMF estimator and gain two insights. First, the n.t. effect can be explained as the rise of a bias term in the standard lower bound, which depends only on the structure of factor space but neither the estimator nor samples. Second, the n.t. effect can be explained as the rise of an d_{th}-root function on the learning rate, where d is the dimension of a Grassmannian containing the subspaces spanned by latent factors. These discoveries are also supported in simulation, and suggest n.t. may be more effectively addressed via model construction other than model selection.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Towards a Theoretical Understanding of Negative Transfer in Collective Matrix Factorization %A Chao Lan %A Jianxin Wang %A Jun Huan %B Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2016 %E Alexander Ihler %E Dominik Janzing %F pmlr-vR14-lan16a %I PMLR %P 622--631 %U https://proceedings.mlr.press/r14/lan16a.html %V R14 %X Collective matrix factorization (CMF) is a popular technique to improve the overall factorization quality of multiple matrices presuming they share the same latent factor. However, it suffers from performance degeneration when this assumption fails, an effect called negative transfer (n.t.). Although the effect is widely admitted, its theoretical nature remains a mystery to date. This paper presents a first theoretical understanding of n.t. in theory. Under the statistical mini-max framework, we derive lower bounds for the CMF estimator and gain two insights. First, the n.t. effect can be explained as the rise of a bias term in the standard lower bound, which depends only on the structure of factor space but neither the estimator nor samples. Second, the n.t. effect can be explained as the rise of an d_{th}-root function on the learning rate, where d is the dimension of a Grassmannian containing the subspaces spanned by latent factors. These discoveries are also supported in simulation, and suggest n.t. may be more effectively addressed via model construction other than model selection. %Z Reissued by PMLR on 04 October 2026.
APA
Lan, C., Wang, J. & Huan, J.. (2016). Towards a Theoretical Understanding of Negative Transfer in Collective Matrix Factorization. Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R14:622-631 Available from https://proceedings.mlr.press/r14/lan16a.html. Reissued by PMLR on 04 October 2026.

Related Material