Curbing Task Interference using Representation Similarity-Guided Multi-Task Feature Sharing

Naresh Kumar Gurulingan, Elahe Arani, Bahram Zonooz
Proceedings of The 1st Conference on Lifelong Learning Agents, PMLR 199:937-951, 2022.

Abstract

Multi-task learning of dense prediction tasks, by sharing both the encoder and decoder, as opposed to sharing only the encoder, provides an attractive front to increase both accuracy and computational efficiency. When the tasks are similar, sharing the decoder serves as an additional inductive bias providing more room for tasks to share complementary information among themselves. However, increased sharing exposes more parameters to task interference which likely hinders both generalization and robustness. Effective ways to curb this interference while exploiting the inductive bias of sharing the decoder remains an open challenge. To address this challenge, we propose Progressive Decoder Fusion (PDF) to progressively combine task decoders based on inter-task representation similarity. We show that this procedure leads to a multi-task network with better generalization to in-distribution and out-of-distribution data and improved robustness to adversarial attacks. Additionally, we observe that the predictions of different tasks of this multi-task network are more consistent among each other.

Cite this Paper

BibTeX
@InProceedings{pmlr-v199-gurulingan22a, title = {Curbing Task Interference using Representation Similarity-Guided Multi-Task Feature Sharing}, author = {Gurulingan, Naresh Kumar and Arani, Elahe and Zonooz, Bahram}, booktitle = {Proceedings of The 1st Conference on Lifelong Learning Agents}, pages = {937--951}, year = {2022}, editor = {Chandar, Sarath and Pascanu, Razvan and Precup, Doina}, volume = {199}, series = {Proceedings of Machine Learning Research}, month = {22--24 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v199/gurulingan22a/gurulingan22a.pdf}, url = {https://proceedings.mlr.press/v199/gurulingan22a.html}, abstract = {Multi-task learning of dense prediction tasks, by sharing both the encoder and decoder, as opposed to sharing only the encoder, provides an attractive front to increase both accuracy and computational efficiency. When the tasks are similar, sharing the decoder serves as an additional inductive bias providing more room for tasks to share complementary information among themselves. However, increased sharing exposes more parameters to task interference which likely hinders both generalization and robustness. Effective ways to curb this interference while exploiting the inductive bias of sharing the decoder remains an open challenge. To address this challenge, we propose Progressive Decoder Fusion (PDF) to progressively combine task decoders based on inter-task representation similarity. We show that this procedure leads to a multi-task network with better generalization to in-distribution and out-of-distribution data and improved robustness to adversarial attacks. Additionally, we observe that the predictions of different tasks of this multi-task network are more consistent among each other.} }
Endnote
%0 Conference Paper %T Curbing Task Interference using Representation Similarity-Guided Multi-Task Feature Sharing %A Naresh Kumar Gurulingan %A Elahe Arani %A Bahram Zonooz %B Proceedings of The 1st Conference on Lifelong Learning Agents %C Proceedings of Machine Learning Research %D 2022 %E Sarath Chandar %E Razvan Pascanu %E Doina Precup %F pmlr-v199-gurulingan22a %I PMLR %P 937--951 %U https://proceedings.mlr.press/v199/gurulingan22a.html %V 199 %X Multi-task learning of dense prediction tasks, by sharing both the encoder and decoder, as opposed to sharing only the encoder, provides an attractive front to increase both accuracy and computational efficiency. When the tasks are similar, sharing the decoder serves as an additional inductive bias providing more room for tasks to share complementary information among themselves. However, increased sharing exposes more parameters to task interference which likely hinders both generalization and robustness. Effective ways to curb this interference while exploiting the inductive bias of sharing the decoder remains an open challenge. To address this challenge, we propose Progressive Decoder Fusion (PDF) to progressively combine task decoders based on inter-task representation similarity. We show that this procedure leads to a multi-task network with better generalization to in-distribution and out-of-distribution data and improved robustness to adversarial attacks. Additionally, we observe that the predictions of different tasks of this multi-task network are more consistent among each other.
APA
Gurulingan, N.K., Arani, E. & Zonooz, B.. (2022). Curbing Task Interference using Representation Similarity-Guided Multi-Task Feature Sharing. Proceedings of The 1st Conference on Lifelong Learning Agents, in Proceedings of Machine Learning Research 199:937-951 Available from https://proceedings.mlr.press/v199/gurulingan22a.html.

Related Material