Enhancing Textbook Question Answering with Knowledge Graph-Augmented Large Language Models

Mengliang He, Aimin Zhou, Xiaoming Shi
Proceedings of the 16th Asian Conference on Machine Learning, PMLR 260:639-654, 2025.

Abstract

Previous works on Textbook Question Answering suffer from limited performance due to the small-scale neural network based backbone. To alleviate the issue, we propose to utilize LLMs as the backbone of TQA tasks. To this end, we utilize two methods, the raw-context based prompting method and the knowledge graph based prompting method. Specifically, we introduce the Textbook Question Answering-Knowledge Graph (TQA-KG) method, which first converts textbook content into structural knowledge graphs and then combining knowledge graph into LLM prompting, thereby enhancing the model’s reasoning capabilities and answer accuracy. Extensive experiments conducted on the CK12-QA dataset illustrate the effectiveness of the method, achieving an improvement of 5.67% in accuracy compared to current state-of-the-art methods on average.

Cite this Paper

BibTeX
@InProceedings{pmlr-v260-he25a, title = {Enhancing Textbook Question Answering with Knowledge Graph-Augmented Large Language Models}, author = {He, Mengliang and Zhou, Aimin and Shi, Xiaoming}, booktitle = {Proceedings of the 16th Asian Conference on Machine Learning}, pages = {639--654}, year = {2025}, editor = {Nguyen, Vu and Lin, Hsuan-Tien}, volume = {260}, series = {Proceedings of Machine Learning Research}, month = {05--08 Dec}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v260/main/assets/he25a/he25a.pdf}, url = {https://proceedings.mlr.press/v260/he25a.html}, abstract = {Previous works on Textbook Question Answering suffer from limited performance due to the small-scale neural network based backbone. To alleviate the issue, we propose to utilize LLMs as the backbone of TQA tasks. To this end, we utilize two methods, the raw-context based prompting method and the knowledge graph based prompting method. Specifically, we introduce the Textbook Question Answering-Knowledge Graph (TQA-KG) method, which first converts textbook content into structural knowledge graphs and then combining knowledge graph into LLM prompting, thereby enhancing the model’s reasoning capabilities and answer accuracy. Extensive experiments conducted on the CK12-QA dataset illustrate the effectiveness of the method, achieving an improvement of 5.67% in accuracy compared to current state-of-the-art methods on average.} }
Endnote
%0 Conference Paper %T Enhancing Textbook Question Answering with Knowledge Graph-Augmented Large Language Models %A Mengliang He %A Aimin Zhou %A Xiaoming Shi %B Proceedings of the 16th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Vu Nguyen %E Hsuan-Tien Lin %F pmlr-v260-he25a %I PMLR %P 639--654 %U https://proceedings.mlr.press/v260/he25a.html %V 260 %X Previous works on Textbook Question Answering suffer from limited performance due to the small-scale neural network based backbone. To alleviate the issue, we propose to utilize LLMs as the backbone of TQA tasks. To this end, we utilize two methods, the raw-context based prompting method and the knowledge graph based prompting method. Specifically, we introduce the Textbook Question Answering-Knowledge Graph (TQA-KG) method, which first converts textbook content into structural knowledge graphs and then combining knowledge graph into LLM prompting, thereby enhancing the model’s reasoning capabilities and answer accuracy. Extensive experiments conducted on the CK12-QA dataset illustrate the effectiveness of the method, achieving an improvement of 5.67% in accuracy compared to current state-of-the-art methods on average.
APA
He, M., Zhou, A. & Shi, X.. (2025). Enhancing Textbook Question Answering with Knowledge Graph-Augmented Large Language Models. Proceedings of the 16th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 260:639-654 Available from https://proceedings.mlr.press/v260/he25a.html.

Related Material