Biomedical Relation Extraction Based on Deep Transfer Learning and Prompt Learning

Ma Rongrong, Liu Huan
Proceedings of 2024 International Conference on Machine Learning and Intelligent Computing, PMLR 245:8-15, 2024.

Abstract

Biomedical Relationship Extraction (BioRE) is an important task for electronic health record mining and biomedical knowledge base construction. Due to the complexity of biomedical information in the literature, it is difficult to realize the construction of large-scale datasets. Although previous models showed good results in a fully supervised environment, they did not generalize well to the low resource situation common in this field. We propose a biomedical relationship extraction model based on deep transfer learning and prompt learning. We use deep neural network for transfer learning, using large-scale biomedical field data set (BioREx) training model. The trained model was applied to low-resource datasets in biomedical field, which can make the model to learn more rich knowledge, so as to alleviate the problem of insufficient training data. In addition, we used a prompt learning method by appending a prompt sentence describing the desired relationship label to the beginning of the input sentence, which can reduce the gap between the pretrained language model (PLM) and downstream tasks. Experiments show that using deep transfer learning and prompt learning can effectively improve the prediction results. Experiments on three BioRE benchmark datasets DrugProt, DDI and BC5CDR, F1 values are significantly improved compared to previous models.

Cite this Paper

BibTeX
@InProceedings{pmlr-v245-rongrong24a, title = {Biomedical Relation Extraction Based on Deep Transfer Learning and Prompt Learning}, author = {Rongrong, Ma and Huan, Liu}, booktitle = {Proceedings of 2024 International Conference on Machine Learning and Intelligent Computing}, pages = {8--15}, year = {2024}, editor = {Nianyin, Zeng and Pachori, Ram Bilas}, volume = {245}, series = {Proceedings of Machine Learning Research}, month = {26--28 Apr}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v245/main/assets/rongrong24a/rongrong24a.pdf}, url = {https://proceedings.mlr.press/v245/rongrong24a.html}, abstract = {Biomedical Relationship Extraction (BioRE) is an important task for electronic health record mining and biomedical knowledge base construction. Due to the complexity of biomedical information in the literature, it is difficult to realize the construction of large-scale datasets. Although previous models showed good results in a fully supervised environment, they did not generalize well to the low resource situation common in this field. We propose a biomedical relationship extraction model based on deep transfer learning and prompt learning. We use deep neural network for transfer learning, using large-scale biomedical field data set (BioREx) training model. The trained model was applied to low-resource datasets in biomedical field, which can make the model to learn more rich knowledge, so as to alleviate the problem of insufficient training data. In addition, we used a prompt learning method by appending a prompt sentence describing the desired relationship label to the beginning of the input sentence, which can reduce the gap between the pretrained language model (PLM) and downstream tasks. Experiments show that using deep transfer learning and prompt learning can effectively improve the prediction results. Experiments on three BioRE benchmark datasets DrugProt, DDI and BC5CDR, F1 values are significantly improved compared to previous models.} }
Endnote
%0 Conference Paper %T Biomedical Relation Extraction Based on Deep Transfer Learning and Prompt Learning %A Ma Rongrong %A Liu Huan %B Proceedings of 2024 International Conference on Machine Learning and Intelligent Computing %C Proceedings of Machine Learning Research %D 2024 %E Zeng Nianyin %E Ram Bilas Pachori %F pmlr-v245-rongrong24a %I PMLR %P 8--15 %U https://proceedings.mlr.press/v245/rongrong24a.html %V 245 %X Biomedical Relationship Extraction (BioRE) is an important task for electronic health record mining and biomedical knowledge base construction. Due to the complexity of biomedical information in the literature, it is difficult to realize the construction of large-scale datasets. Although previous models showed good results in a fully supervised environment, they did not generalize well to the low resource situation common in this field. We propose a biomedical relationship extraction model based on deep transfer learning and prompt learning. We use deep neural network for transfer learning, using large-scale biomedical field data set (BioREx) training model. The trained model was applied to low-resource datasets in biomedical field, which can make the model to learn more rich knowledge, so as to alleviate the problem of insufficient training data. In addition, we used a prompt learning method by appending a prompt sentence describing the desired relationship label to the beginning of the input sentence, which can reduce the gap between the pretrained language model (PLM) and downstream tasks. Experiments show that using deep transfer learning and prompt learning can effectively improve the prediction results. Experiments on three BioRE benchmark datasets DrugProt, DDI and BC5CDR, F1 values are significantly improved compared to previous models.
APA
Rongrong, M. & Huan, L.. (2024). Biomedical Relation Extraction Based on Deep Transfer Learning and Prompt Learning. Proceedings of 2024 International Conference on Machine Learning and Intelligent Computing, in Proceedings of Machine Learning Research 245:8-15 Available from https://proceedings.mlr.press/v245/rongrong24a.html.

Related Material