Pseudo-task Augmentation: From Deep Multitask Learning to Intratask Sharing—and Back

Elliot Meyerson, Risto Miikkulainen
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:3511-3520, 2018.

Abstract

Deep multitask learning boosts performance by sharing learned structure across related tasks. This paper adapts ideas from deep multitask learning to the setting where only a single task is available. The method is formalized as pseudo-task augmentation, in which models are trained with multiple decoders for each task. Pseudo-tasks simulate the effect of training towards closely-related tasks drawn from the same universe. In a suite of experiments, pseudo-task augmentation is shown to improve performance on single-task learning problems. When combined with multitask learning, further improvements are achieved, including state-of-the-art performance on the CelebA dataset, showing that pseudo-task augmentation and multitask learning have complementary value. All in all, pseudo-task augmentation is a broadly applicable and efficient way to boost performance in deep learning systems.

Cite this Paper

BibTeX
@InProceedings{pmlr-v80-meyerson18a, title = {Pseudo-task Augmentation: From Deep Multitask Learning to Intratask Sharing{—}and Back}, author = {Meyerson, Elliot and Miikkulainen, Risto}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {3511--3520}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/meyerson18a/meyerson18a.pdf}, url = {https://proceedings.mlr.press/v80/meyerson18a.html}, abstract = {Deep multitask learning boosts performance by sharing learned structure across related tasks. This paper adapts ideas from deep multitask learning to the setting where only a single task is available. The method is formalized as pseudo-task augmentation, in which models are trained with multiple decoders for each task. Pseudo-tasks simulate the effect of training towards closely-related tasks drawn from the same universe. In a suite of experiments, pseudo-task augmentation is shown to improve performance on single-task learning problems. When combined with multitask learning, further improvements are achieved, including state-of-the-art performance on the CelebA dataset, showing that pseudo-task augmentation and multitask learning have complementary value. All in all, pseudo-task augmentation is a broadly applicable and efficient way to boost performance in deep learning systems.} }
Endnote
%0 Conference Paper %T Pseudo-task Augmentation: From Deep Multitask Learning to Intratask Sharing—and Back %A Elliot Meyerson %A Risto Miikkulainen %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-meyerson18a %I PMLR %P 3511--3520 %U https://proceedings.mlr.press/v80/meyerson18a.html %V 80 %X Deep multitask learning boosts performance by sharing learned structure across related tasks. This paper adapts ideas from deep multitask learning to the setting where only a single task is available. The method is formalized as pseudo-task augmentation, in which models are trained with multiple decoders for each task. Pseudo-tasks simulate the effect of training towards closely-related tasks drawn from the same universe. In a suite of experiments, pseudo-task augmentation is shown to improve performance on single-task learning problems. When combined with multitask learning, further improvements are achieved, including state-of-the-art performance on the CelebA dataset, showing that pseudo-task augmentation and multitask learning have complementary value. All in all, pseudo-task augmentation is a broadly applicable and efficient way to boost performance in deep learning systems.
APA
Meyerson, E. & Miikkulainen, R.. (2018). Pseudo-task Augmentation: From Deep Multitask Learning to Intratask Sharing—and Back. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:3511-3520 Available from https://proceedings.mlr.press/v80/meyerson18a.html.

Related Material