A Convex Formulation for Learning Task Relationships in Multi-Task Learning

Yu Zhang, Dit-Yan Yeung
Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, PMLR R8:700-709, 2010.

Abstract

Multi-task learning is a learning paradigm which seeks to improve the generalization performance of a learning task with the help of some other re- lated tasks. In this paper, we propose a regular- ization formulation for learning the relationships between tasks in multi-task learning. This for- mulation can be viewed as a novel generalization of the regularization framework for single-task learning. Besides modeling positive task cor- relation, our method, called multi-task relation- ship learning (MTRL), can also describe neg- ative task correlation and identify outlier tasks based on the same underlying principle. Un- der this regularization framework, the objective function of MTRL is convex. For efficiency, we use an alternating method to learn the op- timal model parameters for each task as well as the relationships between tasks. We study MTRL in the symmetric multi-task learning set- ting and then generalize it to the asymmetric set- ting as well. We also study the relationships be- tween MTRL and some existing multi-task learn- ing methods. Experiments conducted on a toy problem as well as several benchmark data sets demonstrate the effectiveness of MTRL.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR8-zhang10a, title = {A Convex Formulation for Learning Task Relationships in Multi-Task Learning}, author = {Zhang, Yu and Yeung, Dit-Yan}, booktitle = {Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence}, pages = {700--709}, year = {2010}, editor = {Grünwald, Peter and Spirtes, Peter}, volume = {R8}, series = {Proceedings of Machine Learning Research}, month = {08--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r8/main/assets/zhang10a/zhang10a.pdf}, url = {https://proceedings.mlr.press/r8/zhang10a.html}, abstract = {Multi-task learning is a learning paradigm which seeks to improve the generalization performance of a learning task with the help of some other re- lated tasks. In this paper, we propose a regular- ization formulation for learning the relationships between tasks in multi-task learning. This for- mulation can be viewed as a novel generalization of the regularization framework for single-task learning. Besides modeling positive task cor- relation, our method, called multi-task relation- ship learning (MTRL), can also describe neg- ative task correlation and identify outlier tasks based on the same underlying principle. Un- der this regularization framework, the objective function of MTRL is convex. For efficiency, we use an alternating method to learn the op- timal model parameters for each task as well as the relationships between tasks. We study MTRL in the symmetric multi-task learning set- ting and then generalize it to the asymmetric set- ting as well. We also study the relationships be- tween MTRL and some existing multi-task learn- ing methods. Experiments conducted on a toy problem as well as several benchmark data sets demonstrate the effectiveness of MTRL.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T A Convex Formulation for Learning Task Relationships in Multi-Task Learning %A Yu Zhang %A Dit-Yan Yeung %B Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2010 %E Peter Grünwald %E Peter Spirtes %F pmlr-vR8-zhang10a %I PMLR %P 700--709 %U https://proceedings.mlr.press/r8/zhang10a.html %V R8 %X Multi-task learning is a learning paradigm which seeks to improve the generalization performance of a learning task with the help of some other re- lated tasks. In this paper, we propose a regular- ization formulation for learning the relationships between tasks in multi-task learning. This for- mulation can be viewed as a novel generalization of the regularization framework for single-task learning. Besides modeling positive task cor- relation, our method, called multi-task relation- ship learning (MTRL), can also describe neg- ative task correlation and identify outlier tasks based on the same underlying principle. Un- der this regularization framework, the objective function of MTRL is convex. For efficiency, we use an alternating method to learn the op- timal model parameters for each task as well as the relationships between tasks. We study MTRL in the symmetric multi-task learning set- ting and then generalize it to the asymmetric set- ting as well. We also study the relationships be- tween MTRL and some existing multi-task learn- ing methods. Experiments conducted on a toy problem as well as several benchmark data sets demonstrate the effectiveness of MTRL. %Z Reissued by PMLR on 04 October 2026.
APA
Zhang, Y. & Yeung, D.. (2010). A Convex Formulation for Learning Task Relationships in Multi-Task Learning. Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R8:700-709 Available from https://proceedings.mlr.press/r8/zhang10a.html. Reissued by PMLR on 04 October 2026.

Related Material