OSLAT: Open Set Label Attention Transformer for Medical Entity Retrieval and Span Extraction

Raymond Li, Ilya Valmianski, Li Deng, Xavier Amatriain, Anitha Kannan
Proceedings of the 2nd Machine Learning for Health symposium, PMLR 193:373-390, 2022.

Abstract

Medical entity span extraction and linking are critical steps for many healthcare NLP tasks. Most existing entity extraction methods either have a fixed vocabulary of medical entities or require span annotations. In this paper, we propose a method for linking an open set of entities that does not require any span annotations. Our method, Open Set Label Attention Transformer (OSLAT), uses the label-attention mechanism to learn candidate-entity contextualized text representations. We find that OSLAT can not only link entities but is also able to implicitly learn spans associated with entities. We evaluate OSLAT on two tasks: (1) span extraction trained without explicit span annotations, and (2) entity linking trained without span-level annotation. We test the generalizability of our method by training two separate models on two datasets with low entity overlap and comparing cross-dataset performance.

Cite this Paper

BibTeX
@InProceedings{pmlr-v193-li22a, title = {OSLAT: Open Set Label Attention Transformer for Medical Entity Retrieval and Span Extraction}, author = {Li, Raymond and Valmianski, Ilya and Deng, Li and Amatriain, Xavier and Kannan, Anitha}, booktitle = {Proceedings of the 2nd Machine Learning for Health symposium}, pages = {373--390}, year = {2022}, editor = {Parziale, Antonio and Agrawal, Monica and Joshi, Shalmali and Chen, Irene Y. and Tang, Shengpu and Oala, Luis and Subbaswamy, Adarsh}, volume = {193}, series = {Proceedings of Machine Learning Research}, month = {28 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v193/li22a/li22a.pdf}, url = {https://proceedings.mlr.press/v193/li22a.html}, abstract = {Medical entity span extraction and linking are critical steps for many healthcare NLP tasks. Most existing entity extraction methods either have a fixed vocabulary of medical entities or require span annotations. In this paper, we propose a method for linking an open set of entities that does not require any span annotations. Our method, Open Set Label Attention Transformer (OSLAT), uses the label-attention mechanism to learn candidate-entity contextualized text representations. We find that OSLAT can not only link entities but is also able to implicitly learn spans associated with entities. We evaluate OSLAT on two tasks: (1) span extraction trained without explicit span annotations, and (2) entity linking trained without span-level annotation. We test the generalizability of our method by training two separate models on two datasets with low entity overlap and comparing cross-dataset performance.} }
Endnote
%0 Conference Paper %T OSLAT: Open Set Label Attention Transformer for Medical Entity Retrieval and Span Extraction %A Raymond Li %A Ilya Valmianski %A Li Deng %A Xavier Amatriain %A Anitha Kannan %B Proceedings of the 2nd Machine Learning for Health symposium %C Proceedings of Machine Learning Research %D 2022 %E Antonio Parziale %E Monica Agrawal %E Shalmali Joshi %E Irene Y. Chen %E Shengpu Tang %E Luis Oala %E Adarsh Subbaswamy %F pmlr-v193-li22a %I PMLR %P 373--390 %U https://proceedings.mlr.press/v193/li22a.html %V 193 %X Medical entity span extraction and linking are critical steps for many healthcare NLP tasks. Most existing entity extraction methods either have a fixed vocabulary of medical entities or require span annotations. In this paper, we propose a method for linking an open set of entities that does not require any span annotations. Our method, Open Set Label Attention Transformer (OSLAT), uses the label-attention mechanism to learn candidate-entity contextualized text representations. We find that OSLAT can not only link entities but is also able to implicitly learn spans associated with entities. We evaluate OSLAT on two tasks: (1) span extraction trained without explicit span annotations, and (2) entity linking trained without span-level annotation. We test the generalizability of our method by training two separate models on two datasets with low entity overlap and comparing cross-dataset performance.
APA
Li, R., Valmianski, I., Deng, L., Amatriain, X. & Kannan, A.. (2022). OSLAT: Open Set Label Attention Transformer for Medical Entity Retrieval and Span Extraction. Proceedings of the 2nd Machine Learning for Health symposium, in Proceedings of Machine Learning Research 193:373-390 Available from https://proceedings.mlr.press/v193/li22a.html.

Related Material