Auxiliary Learning as an Asymmetric Bargaining Game

Aviv Shamsian, Aviv Navon, Neta Glazer, Kenji Kawaguchi, Gal Chechik, Ethan Fetaya
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:30689-30705, 2023.

Abstract

Auxiliary learning is an effective method for enhancing the generalization capabilities of trained models, particularly when dealing with small datasets. However, this approach may present several difficulties: (i) optimizing multiple objectives can be more challenging, and (ii) how to balance the auxiliary tasks to best assist the main task is unclear. In this work, we propose a novel approach, named AuxiNash, for balancing tasks in auxiliary learning by formalizing the problem as generalized bargaining game with asymmetric task bargaining power. Furthermore, we describe an efficient procedure for learning the bargaining power of tasks based on their contribution to the performance of the main task and derive theoretical guarantees for its convergence. Finally, we evaluate AuxiNash on multiple multi-task benchmarks and find that it consistently outperforms competing methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-shamsian23a, title = {Auxiliary Learning as an Asymmetric Bargaining Game}, author = {Shamsian, Aviv and Navon, Aviv and Glazer, Neta and Kawaguchi, Kenji and Chechik, Gal and Fetaya, Ethan}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {30689--30705}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/shamsian23a/shamsian23a.pdf}, url = {https://proceedings.mlr.press/v202/shamsian23a.html}, abstract = {Auxiliary learning is an effective method for enhancing the generalization capabilities of trained models, particularly when dealing with small datasets. However, this approach may present several difficulties: (i) optimizing multiple objectives can be more challenging, and (ii) how to balance the auxiliary tasks to best assist the main task is unclear. In this work, we propose a novel approach, named AuxiNash, for balancing tasks in auxiliary learning by formalizing the problem as generalized bargaining game with asymmetric task bargaining power. Furthermore, we describe an efficient procedure for learning the bargaining power of tasks based on their contribution to the performance of the main task and derive theoretical guarantees for its convergence. Finally, we evaluate AuxiNash on multiple multi-task benchmarks and find that it consistently outperforms competing methods.} }
Endnote
%0 Conference Paper %T Auxiliary Learning as an Asymmetric Bargaining Game %A Aviv Shamsian %A Aviv Navon %A Neta Glazer %A Kenji Kawaguchi %A Gal Chechik %A Ethan Fetaya %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-shamsian23a %I PMLR %P 30689--30705 %U https://proceedings.mlr.press/v202/shamsian23a.html %V 202 %X Auxiliary learning is an effective method for enhancing the generalization capabilities of trained models, particularly when dealing with small datasets. However, this approach may present several difficulties: (i) optimizing multiple objectives can be more challenging, and (ii) how to balance the auxiliary tasks to best assist the main task is unclear. In this work, we propose a novel approach, named AuxiNash, for balancing tasks in auxiliary learning by formalizing the problem as generalized bargaining game with asymmetric task bargaining power. Furthermore, we describe an efficient procedure for learning the bargaining power of tasks based on their contribution to the performance of the main task and derive theoretical guarantees for its convergence. Finally, we evaluate AuxiNash on multiple multi-task benchmarks and find that it consistently outperforms competing methods.
APA
Shamsian, A., Navon, A., Glazer, N., Kawaguchi, K., Chechik, G. & Fetaya, E.. (2023). Auxiliary Learning as an Asymmetric Bargaining Game. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:30689-30705 Available from https://proceedings.mlr.press/v202/shamsian23a.html.

Related Material