Multi-task Learning with Labeled and Unlabeled Tasks

Anastasia Pentina, Christoph H. Lampert
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:2807-2816, 2017.

Abstract

In multi-task learning, a learner is given a collection of prediction tasks and needs to solve all of them. In contrast to previous work, which required that annotated training data must be available for all tasks, we consider a new setting, in which for some tasks, potentially most of them, only unlabeled training data is provided. Consequently, to solve all tasks, information must be transferred between tasks with labels and tasks without labels. Focusing on an instance-based transfer method we analyze two variants of this setting: when the set of labeled tasks is fixed, and when it can be actively selected by the learner. We state and prove a generalization bound that covers both scenarios and derive from it an algorithm for making the choice of labeled tasks (in the active case) and for transferring information between the tasks in a principled way. We also illustrate the effectiveness of the algorithm on synthetic and real data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v70-pentina17a, title = {Multi-task Learning with Labeled and Unlabeled Tasks}, author = {Anastasia Pentina and Christoph H. Lampert}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {2807--2816}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, month = {06--11 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/pentina17a/pentina17a.pdf}, url = {https://proceedings.mlr.press/v70/pentina17a.html}, abstract = {In multi-task learning, a learner is given a collection of prediction tasks and needs to solve all of them. In contrast to previous work, which required that annotated training data must be available for all tasks, we consider a new setting, in which for some tasks, potentially most of them, only unlabeled training data is provided. Consequently, to solve all tasks, information must be transferred between tasks with labels and tasks without labels. Focusing on an instance-based transfer method we analyze two variants of this setting: when the set of labeled tasks is fixed, and when it can be actively selected by the learner. We state and prove a generalization bound that covers both scenarios and derive from it an algorithm for making the choice of labeled tasks (in the active case) and for transferring information between the tasks in a principled way. We also illustrate the effectiveness of the algorithm on synthetic and real data.} }
Endnote
%0 Conference Paper %T Multi-task Learning with Labeled and Unlabeled Tasks %A Anastasia Pentina %A Christoph H. Lampert %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F pmlr-v70-pentina17a %I PMLR %P 2807--2816 %U https://proceedings.mlr.press/v70/pentina17a.html %V 70 %X In multi-task learning, a learner is given a collection of prediction tasks and needs to solve all of them. In contrast to previous work, which required that annotated training data must be available for all tasks, we consider a new setting, in which for some tasks, potentially most of them, only unlabeled training data is provided. Consequently, to solve all tasks, information must be transferred between tasks with labels and tasks without labels. Focusing on an instance-based transfer method we analyze two variants of this setting: when the set of labeled tasks is fixed, and when it can be actively selected by the learner. We state and prove a generalization bound that covers both scenarios and derive from it an algorithm for making the choice of labeled tasks (in the active case) and for transferring information between the tasks in a principled way. We also illustrate the effectiveness of the algorithm on synthetic and real data.
APA
Pentina, A. & Lampert, C.H.. (2017). Multi-task Learning with Labeled and Unlabeled Tasks. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:2807-2816 Available from https://proceedings.mlr.press/v70/pentina17a.html.

Related Material