Understanding Clinical Collaborations Through Federated Classifier Selection

Sebastian Caldas, Joo Heung Yoon, Michael R. Pinsky, Gilles Clermont, Artur Dubrawski
Proceedings of the 6th Machine Learning for Healthcare Conference, PMLR 149:126-145, 2021.

Abstract

Deriving true clinical utility from models trained on multiple hospitals’ data is a key challenge in the adoption of Federated Learning (FL) systems in support of clinical collaborations. When utility is equated to predictive power, population heterogeneity between centers becomes a key bottleneck in training performant models. Nevertheless, there are other aspects to clinical utility that have frequently been overlooked in this context. Among them, we argue for the importance of understanding how a collaboration may be affecting the quality of a center’s predictions. Insights into how and when external knowledge is being useful can lead to strategic decisions by stakeholders, such as better allocation of local resources or even identifying best practices outside of the current organization. We take a step towards deriving such utility through FedeRated CLassifier Selection (FRCLS, pronounced “freckles”): an algorithm that reuses classifiers trained in outside institutions. It identifies regions of the feature space where the collaborators’ models will outperform the local center’s classifier, and can provide interpretable rules to describe these regions of beneficial expertise. We apply FRCLS to a sepsis prediction task in two different hospital systems, demonstrating its benefits in terms of understanding the types of patients for which the collaboration is useful and reasoning about the strategic decisions that may stem out of these analyses.

Cite this Paper


BibTeX
@InProceedings{pmlr-v149-caldas21a, title = {Understanding Clinical Collaborations Through Federated Classifier Selection}, author = {Caldas, Sebastian and Yoon, Joo Heung and Pinsky, Michael R. and Clermont, Gilles and Dubrawski, Artur}, booktitle = {Proceedings of the 6th Machine Learning for Healthcare Conference}, pages = {126--145}, year = {2021}, editor = {Jung, Ken and Yeung, Serena and Sendak, Mark and Sjoding, Michael and Ranganath, Rajesh}, volume = {149}, series = {Proceedings of Machine Learning Research}, month = {06--07 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v149/caldas21a/caldas21a.pdf}, url = {https://proceedings.mlr.press/v149/caldas21a.html}, abstract = {Deriving true clinical utility from models trained on multiple hospitals’ data is a key challenge in the adoption of Federated Learning (FL) systems in support of clinical collaborations. When utility is equated to predictive power, population heterogeneity between centers becomes a key bottleneck in training performant models. Nevertheless, there are other aspects to clinical utility that have frequently been overlooked in this context. Among them, we argue for the importance of understanding how a collaboration may be affecting the quality of a center’s predictions. Insights into how and when external knowledge is being useful can lead to strategic decisions by stakeholders, such as better allocation of local resources or even identifying best practices outside of the current organization. We take a step towards deriving such utility through FedeRated CLassifier Selection (FRCLS, pronounced “freckles”): an algorithm that reuses classifiers trained in outside institutions. It identifies regions of the feature space where the collaborators’ models will outperform the local center’s classifier, and can provide interpretable rules to describe these regions of beneficial expertise. We apply FRCLS to a sepsis prediction task in two different hospital systems, demonstrating its benefits in terms of understanding the types of patients for which the collaboration is useful and reasoning about the strategic decisions that may stem out of these analyses.} }
Endnote
%0 Conference Paper %T Understanding Clinical Collaborations Through Federated Classifier Selection %A Sebastian Caldas %A Joo Heung Yoon %A Michael R. Pinsky %A Gilles Clermont %A Artur Dubrawski %B Proceedings of the 6th Machine Learning for Healthcare Conference %C Proceedings of Machine Learning Research %D 2021 %E Ken Jung %E Serena Yeung %E Mark Sendak %E Michael Sjoding %E Rajesh Ranganath %F pmlr-v149-caldas21a %I PMLR %P 126--145 %U https://proceedings.mlr.press/v149/caldas21a.html %V 149 %X Deriving true clinical utility from models trained on multiple hospitals’ data is a key challenge in the adoption of Federated Learning (FL) systems in support of clinical collaborations. When utility is equated to predictive power, population heterogeneity between centers becomes a key bottleneck in training performant models. Nevertheless, there are other aspects to clinical utility that have frequently been overlooked in this context. Among them, we argue for the importance of understanding how a collaboration may be affecting the quality of a center’s predictions. Insights into how and when external knowledge is being useful can lead to strategic decisions by stakeholders, such as better allocation of local resources or even identifying best practices outside of the current organization. We take a step towards deriving such utility through FedeRated CLassifier Selection (FRCLS, pronounced “freckles”): an algorithm that reuses classifiers trained in outside institutions. It identifies regions of the feature space where the collaborators’ models will outperform the local center’s classifier, and can provide interpretable rules to describe these regions of beneficial expertise. We apply FRCLS to a sepsis prediction task in two different hospital systems, demonstrating its benefits in terms of understanding the types of patients for which the collaboration is useful and reasoning about the strategic decisions that may stem out of these analyses.
APA
Caldas, S., Yoon, J.H., Pinsky, M.R., Clermont, G. & Dubrawski, A.. (2021). Understanding Clinical Collaborations Through Federated Classifier Selection. Proceedings of the 6th Machine Learning for Healthcare Conference, in Proceedings of Machine Learning Research 149:126-145 Available from https://proceedings.mlr.press/v149/caldas21a.html.

Related Material