Collective Model Fusion for Multiple Black-Box Experts

Minh Hoang, Nghia Hoang, Bryan Kian Hsiang Low, Carleton Kingsford
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:2742-2750, 2019.

Abstract

Model fusion is a fundamental problem in collec-tive machine learning (ML) where independentexperts with heterogeneous learning architecturesare required to combine expertise to improve pre-dictive performance. This is particularly chal-lenging in information-sensitive domains whereexperts do not have access to each other’s internalarchitecture and local data. This paper presentsthe first collective model fusion framework formultiple experts with heterogeneous black-box ar-chitectures. The proposed method will enable thisby addressing the key issues of how black-boxexperts interact to understand the predictive be-haviors of one another; how these understandingscan be represented and shared efficiently amongthemselves; and how the shared understandingscan be combined to generate high-quality consen-sus prediction. The performance of the resultingframework is analyzed theoretically and demon-strated empirically on several datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-hoang19a, title = {Collective Model Fusion for Multiple Black-Box Experts}, author = {Hoang, Minh and Hoang, Nghia and Low, Bryan Kian Hsiang and Kingsford, Carleton}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {2742--2750}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/hoang19a/hoang19a.pdf}, url = {https://proceedings.mlr.press/v97/hoang19a.html}, abstract = {Model fusion is a fundamental problem in collec-tive machine learning (ML) where independentexperts with heterogeneous learning architecturesare required to combine expertise to improve pre-dictive performance. This is particularly chal-lenging in information-sensitive domains whereexperts do not have access to each other’s internalarchitecture and local data. This paper presentsthe first collective model fusion framework formultiple experts with heterogeneous black-box ar-chitectures. The proposed method will enable thisby addressing the key issues of how black-boxexperts interact to understand the predictive be-haviors of one another; how these understandingscan be represented and shared efficiently amongthemselves; and how the shared understandingscan be combined to generate high-quality consen-sus prediction. The performance of the resultingframework is analyzed theoretically and demon-strated empirically on several datasets.} }
Endnote
%0 Conference Paper %T Collective Model Fusion for Multiple Black-Box Experts %A Minh Hoang %A Nghia Hoang %A Bryan Kian Hsiang Low %A Carleton Kingsford %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-hoang19a %I PMLR %P 2742--2750 %U https://proceedings.mlr.press/v97/hoang19a.html %V 97 %X Model fusion is a fundamental problem in collec-tive machine learning (ML) where independentexperts with heterogeneous learning architecturesare required to combine expertise to improve pre-dictive performance. This is particularly chal-lenging in information-sensitive domains whereexperts do not have access to each other’s internalarchitecture and local data. This paper presentsthe first collective model fusion framework formultiple experts with heterogeneous black-box ar-chitectures. The proposed method will enable thisby addressing the key issues of how black-boxexperts interact to understand the predictive be-haviors of one another; how these understandingscan be represented and shared efficiently amongthemselves; and how the shared understandingscan be combined to generate high-quality consen-sus prediction. The performance of the resultingframework is analyzed theoretically and demon-strated empirically on several datasets.
APA
Hoang, M., Hoang, N., Low, B.K.H. & Kingsford, C.. (2019). Collective Model Fusion for Multiple Black-Box Experts. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:2742-2750 Available from https://proceedings.mlr.press/v97/hoang19a.html.

Related Material