Hidden in Plain Sight: Undetectable Adversarial Bias Attacks on Vulnerable Patient Populations

Pranav Kulkarni, Andrew Chan, Nithya Navarathna, Skylar Chan, Paul Yi, Vishwa Sanjay Parekh
Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning, PMLR 250:793-821, 2024.

Abstract

The proliferation of artificial intelligence (AI) in radiology has shed light on the risk of deep learning (DL) models exacerbating clinical biases towards vulnerable patient populations. While prior literature has focused on quantifying biases exhibited by trained DL models, demographically targeted adversarial bias attacks on DL models and its implication in the clinical environment remains an underexplored field of research in medical imaging. In this work, we demonstrate that demographically targeted label poisoning attacks can introduce undetectable underdiagnosis bias in DL models. Our results across multiple performance metrics and demographic groups like sex, age, and their intersectional subgroups show that adversarial bias attacks demonstrate high-selectivity for bias in the targeted group by degrading group model performance without impacting overall model performance. Furthermore, our results indicate that adversarial bias attacks result in biased DL models that propagate prediction bias even when evaluated with external datasets.

Cite this Paper


BibTeX
@InProceedings{pmlr-v250-kulkarni24a, title = {Hidden in Plain Sight: Undetectable Adversarial Bias Attacks on Vulnerable Patient Populations}, author = {Kulkarni, Pranav and Chan, Andrew and Navarathna, Nithya and Chan, Skylar and Yi, Paul and Parekh, Vishwa Sanjay}, booktitle = {Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning}, pages = {793--821}, year = {2024}, editor = {Burgos, Ninon and Petitjean, Caroline and Vakalopoulou, Maria and Christodoulidis, Stergios and Coupe, Pierrick and Delingette, Hervé and Lartizien, Carole and Mateus, Diana}, volume = {250}, series = {Proceedings of Machine Learning Research}, month = {03--05 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v250/main/assets/kulkarni24a/kulkarni24a.pdf}, url = {https://proceedings.mlr.press/v250/kulkarni24a.html}, abstract = {The proliferation of artificial intelligence (AI) in radiology has shed light on the risk of deep learning (DL) models exacerbating clinical biases towards vulnerable patient populations. While prior literature has focused on quantifying biases exhibited by trained DL models, demographically targeted adversarial bias attacks on DL models and its implication in the clinical environment remains an underexplored field of research in medical imaging. In this work, we demonstrate that demographically targeted label poisoning attacks can introduce undetectable underdiagnosis bias in DL models. Our results across multiple performance metrics and demographic groups like sex, age, and their intersectional subgroups show that adversarial bias attacks demonstrate high-selectivity for bias in the targeted group by degrading group model performance without impacting overall model performance. Furthermore, our results indicate that adversarial bias attacks result in biased DL models that propagate prediction bias even when evaluated with external datasets.} }
Endnote
%0 Conference Paper %T Hidden in Plain Sight: Undetectable Adversarial Bias Attacks on Vulnerable Patient Populations %A Pranav Kulkarni %A Andrew Chan %A Nithya Navarathna %A Skylar Chan %A Paul Yi %A Vishwa Sanjay Parekh %B Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning %C Proceedings of Machine Learning Research %D 2024 %E Ninon Burgos %E Caroline Petitjean %E Maria Vakalopoulou %E Stergios Christodoulidis %E Pierrick Coupe %E Hervé Delingette %E Carole Lartizien %E Diana Mateus %F pmlr-v250-kulkarni24a %I PMLR %P 793--821 %U https://proceedings.mlr.press/v250/kulkarni24a.html %V 250 %X The proliferation of artificial intelligence (AI) in radiology has shed light on the risk of deep learning (DL) models exacerbating clinical biases towards vulnerable patient populations. While prior literature has focused on quantifying biases exhibited by trained DL models, demographically targeted adversarial bias attacks on DL models and its implication in the clinical environment remains an underexplored field of research in medical imaging. In this work, we demonstrate that demographically targeted label poisoning attacks can introduce undetectable underdiagnosis bias in DL models. Our results across multiple performance metrics and demographic groups like sex, age, and their intersectional subgroups show that adversarial bias attacks demonstrate high-selectivity for bias in the targeted group by degrading group model performance without impacting overall model performance. Furthermore, our results indicate that adversarial bias attacks result in biased DL models that propagate prediction bias even when evaluated with external datasets.
APA
Kulkarni, P., Chan, A., Navarathna, N., Chan, S., Yi, P. & Parekh, V.S.. (2024). Hidden in Plain Sight: Undetectable Adversarial Bias Attacks on Vulnerable Patient Populations. Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning, in Proceedings of Machine Learning Research 250:793-821 Available from https://proceedings.mlr.press/v250/kulkarni24a.html.

Related Material