GlossAdapter: Enhancing word sense disambiguation via LoRA adapters

Vijayalakshmi Manikandan, Dunwei Wen, M. Ali Akber Dewan
Proceedings of the The 39th Canadian Conference on Artificial Intelligence, PMLR 318:1044-1051, 2026.

Abstract

Word sense disambiguation (WSD) is a long-standing problem in natural language processing (NLP). Recently, fine-tuned large pre-trained models with gloss and other lexical information have been used for WSD. But these models are parameter inefficient as the entire model needs to be trained. To deal with the problem, we propose GlossAdapter to WSD via Low-Rank Adaptation (LoRA) adapter modules and Part of Speech (POS) filtering. LoRA modules are parameter-efficient as they add only a few trainable parameters for a task, keeping the original weights of the pre-trained model frozen while maintaining the model quality. The proposed POS filtering aligns target word context with WordNet lexical categories to construct sentence-gloss pairs for effective model training. We fine-tune our model with SemCor3.0 dataset, and evaluated it with benchmark datasets Senseval-2, Senseval-3, SemEval-2013, and SemEval-2015. We perform experiments based on BERTbase and RoBERTalarge models. By adding only 0.5% of the parameters for RoBERTalarge, the results show that our LoRA adapter-based model combined with POS filtering outperforms the other state-of-the-art models.

Cite this Paper

BibTeX
@InProceedings{pmlr-v318-manikandan26a, title = {GlossAdapter: Enhancing word sense disambiguation via LoRA adapters}, author = {Manikandan, Vijayalakshmi and Wen, Dunwei and Dewan, M. Ali Akber}, booktitle = {Proceedings of the The 39th Canadian Conference on Artificial Intelligence}, pages = {1044--1051}, year = {2026}, editor = {Bouzar-Benlabiod, Lydia and Leung, Carson}, volume = {318}, series = {Proceedings of Machine Learning Research}, month = {25--29 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v318/main/assets/manikandan26a/manikandan26a.pdf}, url = {https://proceedings.mlr.press/v318/manikandan26a.html}, abstract = {Word sense disambiguation (WSD) is a long-standing problem in natural language processing (NLP). Recently, fine-tuned large pre-trained models with gloss and other lexical information have been used for WSD. But these models are parameter inefficient as the entire model needs to be trained. To deal with the problem, we propose GlossAdapter to WSD via Low-Rank Adaptation (LoRA) adapter modules and Part of Speech (POS) filtering. LoRA modules are parameter-efficient as they add only a few trainable parameters for a task, keeping the original weights of the pre-trained model frozen while maintaining the model quality. The proposed POS filtering aligns target word context with WordNet lexical categories to construct sentence-gloss pairs for effective model training. We fine-tune our model with SemCor3.0 dataset, and evaluated it with benchmark datasets Senseval-2, Senseval-3, SemEval-2013, and SemEval-2015. We perform experiments based on BERTbase and RoBERTalarge models. By adding only 0.5% of the parameters for RoBERTalarge, the results show that our LoRA adapter-based model combined with POS filtering outperforms the other state-of-the-art models.} }
Endnote
%0 Conference Paper %T GlossAdapter: Enhancing word sense disambiguation via LoRA adapters %A Vijayalakshmi Manikandan %A Dunwei Wen %A M. Ali Akber Dewan %B Proceedings of the The 39th Canadian Conference on Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Lydia Bouzar-Benlabiod %E Carson Leung %F pmlr-v318-manikandan26a %I PMLR %P 1044--1051 %U https://proceedings.mlr.press/v318/manikandan26a.html %V 318 %X Word sense disambiguation (WSD) is a long-standing problem in natural language processing (NLP). Recently, fine-tuned large pre-trained models with gloss and other lexical information have been used for WSD. But these models are parameter inefficient as the entire model needs to be trained. To deal with the problem, we propose GlossAdapter to WSD via Low-Rank Adaptation (LoRA) adapter modules and Part of Speech (POS) filtering. LoRA modules are parameter-efficient as they add only a few trainable parameters for a task, keeping the original weights of the pre-trained model frozen while maintaining the model quality. The proposed POS filtering aligns target word context with WordNet lexical categories to construct sentence-gloss pairs for effective model training. We fine-tune our model with SemCor3.0 dataset, and evaluated it with benchmark datasets Senseval-2, Senseval-3, SemEval-2013, and SemEval-2015. We perform experiments based on BERTbase and RoBERTalarge models. By adding only 0.5% of the parameters for RoBERTalarge, the results show that our LoRA adapter-based model combined with POS filtering outperforms the other state-of-the-art models.
APA
Manikandan, V., Wen, D. & Dewan, M.A.A.. (2026). GlossAdapter: Enhancing word sense disambiguation via LoRA adapters. Proceedings of the The 39th Canadian Conference on Artificial Intelligence, in Proceedings of Machine Learning Research 318:1044-1051 Available from https://proceedings.mlr.press/v318/manikandan26a.html.

Related Material