Enhancing Non-Cognitive Assessments with GPT: Innovations in Item Generation and Translation for the University Belonging Questionnaire

Mingfeng Xue, Yunting Liu, HuaXia Xiong
Proceedings of Large Foundation Models for Educational Assessment, PMLR 264:157-172, 2025.

Abstract

This study explores the application of GPT-3.5-turbo for item generation and translation in non-cognitive educational assessments, specifically focusing on the University Belonging Questionnaire (UBQ). The UBQ, designed to measure university students’ sense of belonging across three dimensions, was expanded to include a new dimension on peer relationships, and translated into Chinese using GPT-3.5-turbo. A total of 25 new items, including those for the new dimension were generated and translated into Chinese, out of which 14 items passed the expert review. Psychometric analyses of the expanded and translated UBQ were conducted to evaluate reliability, internal structure, and external validity. The results demonstrate that the UBQ, with its new and translated items, maintains strong reliability and satisfactory internal structure, although the new items introduced some noise. Correlation analyses with the general belongingness scale revealed moderate associations with the acceptance dimension but weak associations with the overall scale. The study highlights GPT’s potential in efficiently expanding and translating non-cognitive assessment tools. This work addresses crucial needs in educational assessments and provides a foundation for future advancements in item development and translation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v264-xue25a, title = {Enhancing Non-Cognitive Assessments with GPT: Innovations in Item Generation and Translation for the University Belonging Questionnaire}, author = {Xue, Mingfeng and Liu, Yunting and Xiong, HuaXia}, booktitle = {Proceedings of Large Foundation Models for Educational Assessment}, pages = {157--172}, year = {2025}, editor = {Li, Sheng and Cui, Zhongmin and Lu, Jiasen and Harris, Deborah and Jing, Shumin}, volume = {264}, series = {Proceedings of Machine Learning Research}, month = {15--16 Dec}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v264/main/assets/xue25a/xue25a.pdf}, url = {https://proceedings.mlr.press/v264/xue25a.html}, abstract = {This study explores the application of GPT-3.5-turbo for item generation and translation in non-cognitive educational assessments, specifically focusing on the University Belonging Questionnaire (UBQ). The UBQ, designed to measure university students’ sense of belonging across three dimensions, was expanded to include a new dimension on peer relationships, and translated into Chinese using GPT-3.5-turbo. A total of 25 new items, including those for the new dimension were generated and translated into Chinese, out of which 14 items passed the expert review. Psychometric analyses of the expanded and translated UBQ were conducted to evaluate reliability, internal structure, and external validity. The results demonstrate that the UBQ, with its new and translated items, maintains strong reliability and satisfactory internal structure, although the new items introduced some noise. Correlation analyses with the general belongingness scale revealed moderate associations with the acceptance dimension but weak associations with the overall scale. The study highlights GPT’s potential in efficiently expanding and translating non-cognitive assessment tools. This work addresses crucial needs in educational assessments and provides a foundation for future advancements in item development and translation.} }
Endnote
%0 Conference Paper %T Enhancing Non-Cognitive Assessments with GPT: Innovations in Item Generation and Translation for the University Belonging Questionnaire %A Mingfeng Xue %A Yunting Liu %A HuaXia Xiong %B Proceedings of Large Foundation Models for Educational Assessment %C Proceedings of Machine Learning Research %D 2025 %E Sheng Li %E Zhongmin Cui %E Jiasen Lu %E Deborah Harris %E Shumin Jing %F pmlr-v264-xue25a %I PMLR %P 157--172 %U https://proceedings.mlr.press/v264/xue25a.html %V 264 %X This study explores the application of GPT-3.5-turbo for item generation and translation in non-cognitive educational assessments, specifically focusing on the University Belonging Questionnaire (UBQ). The UBQ, designed to measure university students’ sense of belonging across three dimensions, was expanded to include a new dimension on peer relationships, and translated into Chinese using GPT-3.5-turbo. A total of 25 new items, including those for the new dimension were generated and translated into Chinese, out of which 14 items passed the expert review. Psychometric analyses of the expanded and translated UBQ were conducted to evaluate reliability, internal structure, and external validity. The results demonstrate that the UBQ, with its new and translated items, maintains strong reliability and satisfactory internal structure, although the new items introduced some noise. Correlation analyses with the general belongingness scale revealed moderate associations with the acceptance dimension but weak associations with the overall scale. The study highlights GPT’s potential in efficiently expanding and translating non-cognitive assessment tools. This work addresses crucial needs in educational assessments and provides a foundation for future advancements in item development and translation.
APA
Xue, M., Liu, Y. & Xiong, H.. (2025). Enhancing Non-Cognitive Assessments with GPT: Innovations in Item Generation and Translation for the University Belonging Questionnaire. Proceedings of Large Foundation Models for Educational Assessment, in Proceedings of Machine Learning Research 264:157-172 Available from https://proceedings.mlr.press/v264/xue25a.html.

Related Material