Using Expert Gaze for Self-Supervised and Supervised Contrastive Learning of Glaucoma from OCT Data

Wai Tak Lau, Ye Tian, Roshan Kenia, Saanvi Aima, Kaveri A Thakoor
Proceedings of the fifth Conference on Health, Inference, and Learning, PMLR 248:427-445, 2024.

Abstract

In this work, we address the challenge of limited data availability common in healthcare settings by using clinician (ophthalmologist) gaze data on optical coherence tomography (OCT) report images as they diagnose glaucoma, a top cause of irreversible blindness worldwide. We directly learn gaze representations with our ‘GazeFormerMD’ model to generate pseudo-labels using a novel multi-task objective, combining triplet and cross-entropy losses. We use these pseudo-labels for weakly supervised contrastive learning (WSupCon) to detect glaucoma from a partially-labeled dataset of OCT report images. Our natural-language-inspired region-based-encoding GazeFormerMD model pseudo-labels, trained using our multi-task objective, enable downstream glaucoma detection accuracy via WSupCon exceeding 91% even with only 70% labeled training data. Furthermore, a model pre-trained with GazeFormerMD-generated pseudo-labels and used for linear evaluation on an unseen OCT-report dataset achieved comparable performance to a fully-supervised, trained-from-scratch model while using only 25% labeled data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v248-lau24a, title = {Using Expert Gaze for Self-Supervised and Supervised Contrastive Learning of Glaucoma from OCT Data}, author = {Lau, Wai Tak and Tian, Ye and Kenia, Roshan and Aima, Saanvi and Thakoor, Kaveri A}, booktitle = {Proceedings of the fifth Conference on Health, Inference, and Learning}, pages = {427--445}, year = {2024}, editor = {Pollard, Tom and Choi, Edward and Singhal, Pankhuri and Hughes, Michael and Sizikova, Elena and Mortazavi, Bobak and Chen, Irene and Wang, Fei and Sarker, Tasmie and McDermott, Matthew and Ghassemi, Marzyeh}, volume = {248}, series = {Proceedings of Machine Learning Research}, month = {27--28 Jun}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v248/main/assets/lau24a/lau24a.pdf}, url = {https://proceedings.mlr.press/v248/lau24a.html}, abstract = {In this work, we address the challenge of limited data availability common in healthcare settings by using clinician (ophthalmologist) gaze data on optical coherence tomography (OCT) report images as they diagnose glaucoma, a top cause of irreversible blindness worldwide. We directly learn gaze representations with our ‘GazeFormerMD’ model to generate pseudo-labels using a novel multi-task objective, combining triplet and cross-entropy losses. We use these pseudo-labels for weakly supervised contrastive learning (WSupCon) to detect glaucoma from a partially-labeled dataset of OCT report images. Our natural-language-inspired region-based-encoding GazeFormerMD model pseudo-labels, trained using our multi-task objective, enable downstream glaucoma detection accuracy via WSupCon exceeding 91% even with only 70% labeled training data. Furthermore, a model pre-trained with GazeFormerMD-generated pseudo-labels and used for linear evaluation on an unseen OCT-report dataset achieved comparable performance to a fully-supervised, trained-from-scratch model while using only 25% labeled data.} }
Endnote
%0 Conference Paper %T Using Expert Gaze for Self-Supervised and Supervised Contrastive Learning of Glaucoma from OCT Data %A Wai Tak Lau %A Ye Tian %A Roshan Kenia %A Saanvi Aima %A Kaveri A Thakoor %B Proceedings of the fifth Conference on Health, Inference, and Learning %C Proceedings of Machine Learning Research %D 2024 %E Tom Pollard %E Edward Choi %E Pankhuri Singhal %E Michael Hughes %E Elena Sizikova %E Bobak Mortazavi %E Irene Chen %E Fei Wang %E Tasmie Sarker %E Matthew McDermott %E Marzyeh Ghassemi %F pmlr-v248-lau24a %I PMLR %P 427--445 %U https://proceedings.mlr.press/v248/lau24a.html %V 248 %X In this work, we address the challenge of limited data availability common in healthcare settings by using clinician (ophthalmologist) gaze data on optical coherence tomography (OCT) report images as they diagnose glaucoma, a top cause of irreversible blindness worldwide. We directly learn gaze representations with our ‘GazeFormerMD’ model to generate pseudo-labels using a novel multi-task objective, combining triplet and cross-entropy losses. We use these pseudo-labels for weakly supervised contrastive learning (WSupCon) to detect glaucoma from a partially-labeled dataset of OCT report images. Our natural-language-inspired region-based-encoding GazeFormerMD model pseudo-labels, trained using our multi-task objective, enable downstream glaucoma detection accuracy via WSupCon exceeding 91% even with only 70% labeled training data. Furthermore, a model pre-trained with GazeFormerMD-generated pseudo-labels and used for linear evaluation on an unseen OCT-report dataset achieved comparable performance to a fully-supervised, trained-from-scratch model while using only 25% labeled data.
APA
Lau, W.T., Tian, Y., Kenia, R., Aima, S. & Thakoor, K.A.. (2024). Using Expert Gaze for Self-Supervised and Supervised Contrastive Learning of Glaucoma from OCT Data. Proceedings of the fifth Conference on Health, Inference, and Learning, in Proceedings of Machine Learning Research 248:427-445 Available from https://proceedings.mlr.press/v248/lau24a.html.

Related Material