Ensembling Neural Networks for Improved Prediction and Privacy in Early Diagnosis of Sepsis

Shigehiko Schamoni, Michael Hagmann, Stefan Riezler
Proceedings of the 7th Machine Learning for Healthcare Conference, PMLR 182:123-145, 2022.

Abstract

Ensembling neural networks is a long-standing technique for improving the generalization error of neural networks by combining networks with orthogonal properties via a committee decision. We show that this technique is an ideal fit for machine learning on medical data: First, ensembles are amenable to parallel and asynchronous learning, thus enabling efficient training of patient-specific component neural networks. Second, building on the idea of minimizing generalization error by selecting uncorrelated patient-specific networks, we show that one can build an ensemble of a few selected patient-specific models that outperforms a single model trained on much larger pooled datasets. Third, the non-iterative ensemble combination step is an optimal low-dimensional entry point to apply output perturbation to guarantee the privacy of the patient-specific networks. We exemplify our framework of differentially private ensembles on the task of early prediction of sepsis, using real-life intensive care unit data labeled by clinical experts.

Cite this Paper

BibTeX
@InProceedings{pmlr-v182-schamoni22a, title = {Ensembling Neural Networks for Improved Prediction and Privacy in Early Diagnosis of Sepsis}, author = {Schamoni, Shigehiko and Hagmann, Michael and Riezler, Stefan}, booktitle = {Proceedings of the 7th Machine Learning for Healthcare Conference}, pages = {123--145}, year = {2022}, editor = {Lipton, Zachary and Ranganath, Rajesh and Sendak, Mark and Sjoding, Michael and Yeung, Serena}, volume = {182}, series = {Proceedings of Machine Learning Research}, month = {05--06 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v182/schamoni22a/schamoni22a.pdf}, url = {https://proceedings.mlr.press/v182/schamoni22a.html}, abstract = {Ensembling neural networks is a long-standing technique for improving the generalization error of neural networks by combining networks with orthogonal properties via a committee decision. We show that this technique is an ideal fit for machine learning on medical data: First, ensembles are amenable to parallel and asynchronous learning, thus enabling efficient training of patient-specific component neural networks. Second, building on the idea of minimizing generalization error by selecting uncorrelated patient-specific networks, we show that one can build an ensemble of a few selected patient-specific models that outperforms a single model trained on much larger pooled datasets. Third, the non-iterative ensemble combination step is an optimal low-dimensional entry point to apply output perturbation to guarantee the privacy of the patient-specific networks. We exemplify our framework of differentially private ensembles on the task of early prediction of sepsis, using real-life intensive care unit data labeled by clinical experts.} }
Endnote
%0 Conference Paper %T Ensembling Neural Networks for Improved Prediction and Privacy in Early Diagnosis of Sepsis %A Shigehiko Schamoni %A Michael Hagmann %A Stefan Riezler %B Proceedings of the 7th Machine Learning for Healthcare Conference %C Proceedings of Machine Learning Research %D 2022 %E Zachary Lipton %E Rajesh Ranganath %E Mark Sendak %E Michael Sjoding %E Serena Yeung %F pmlr-v182-schamoni22a %I PMLR %P 123--145 %U https://proceedings.mlr.press/v182/schamoni22a.html %V 182 %X Ensembling neural networks is a long-standing technique for improving the generalization error of neural networks by combining networks with orthogonal properties via a committee decision. We show that this technique is an ideal fit for machine learning on medical data: First, ensembles are amenable to parallel and asynchronous learning, thus enabling efficient training of patient-specific component neural networks. Second, building on the idea of minimizing generalization error by selecting uncorrelated patient-specific networks, we show that one can build an ensemble of a few selected patient-specific models that outperforms a single model trained on much larger pooled datasets. Third, the non-iterative ensemble combination step is an optimal low-dimensional entry point to apply output perturbation to guarantee the privacy of the patient-specific networks. We exemplify our framework of differentially private ensembles on the task of early prediction of sepsis, using real-life intensive care unit data labeled by clinical experts.
APA
Schamoni, S., Hagmann, M. & Riezler, S.. (2022). Ensembling Neural Networks for Improved Prediction and Privacy in Early Diagnosis of Sepsis. Proceedings of the 7th Machine Learning for Healthcare Conference, in Proceedings of Machine Learning Research 182:123-145 Available from https://proceedings.mlr.press/v182/schamoni22a.html.

Related Material