A Unified Adaptive Testing System Enabled by Hierarchical Structure Search

Junhao Yu, Yan Zhuang, Zhenya Huang, Qi Liu, Xin Li, Rui Li, Enhong Chen
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:57803-57817, 2024.

Abstract

Adaptive Testing System (ATS) is a promising testing mode, extensively utilized in standardized tests like the GRE. It offers personalized ability assessment by dynamically adjusting questions based on individual ability levels. Compared to traditional exams, ATS can improve the accuracy of ability estimates while simultaneously reducing the number of questions required. Despite the diverse testing formats of ATS, tailored to different adaptability requirements in various testing scenarios, there is a notable absence of a unified framework for modeling them. In this paper, we introduce a unified data-driven ATS framework that conceptualizes the various testing formats as a hierarchical test structure search problem. It can learn directly from data to solve for the optimal questions for each student, eliminating the need for manual test design. The proposed solution algorithm comes with theoretical guarantees for estimation error and convergence. Empirical results show that our framework maintains assessment accuracy while reducing question count by 20% on average and improving training stability.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-yu24r, title = {A Unified Adaptive Testing System Enabled by Hierarchical Structure Search}, author = {Yu, Junhao and Zhuang, Yan and Huang, Zhenya and Liu, Qi and Li, Xin and Li, Rui and Chen, Enhong}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {57803--57817}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/yu24r/yu24r.pdf}, url = {https://proceedings.mlr.press/v235/yu24r.html}, abstract = {Adaptive Testing System (ATS) is a promising testing mode, extensively utilized in standardized tests like the GRE. It offers personalized ability assessment by dynamically adjusting questions based on individual ability levels. Compared to traditional exams, ATS can improve the accuracy of ability estimates while simultaneously reducing the number of questions required. Despite the diverse testing formats of ATS, tailored to different adaptability requirements in various testing scenarios, there is a notable absence of a unified framework for modeling them. In this paper, we introduce a unified data-driven ATS framework that conceptualizes the various testing formats as a hierarchical test structure search problem. It can learn directly from data to solve for the optimal questions for each student, eliminating the need for manual test design. The proposed solution algorithm comes with theoretical guarantees for estimation error and convergence. Empirical results show that our framework maintains assessment accuracy while reducing question count by 20% on average and improving training stability.} }
Endnote
%0 Conference Paper %T A Unified Adaptive Testing System Enabled by Hierarchical Structure Search %A Junhao Yu %A Yan Zhuang %A Zhenya Huang %A Qi Liu %A Xin Li %A Rui Li %A Enhong Chen %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-yu24r %I PMLR %P 57803--57817 %U https://proceedings.mlr.press/v235/yu24r.html %V 235 %X Adaptive Testing System (ATS) is a promising testing mode, extensively utilized in standardized tests like the GRE. It offers personalized ability assessment by dynamically adjusting questions based on individual ability levels. Compared to traditional exams, ATS can improve the accuracy of ability estimates while simultaneously reducing the number of questions required. Despite the diverse testing formats of ATS, tailored to different adaptability requirements in various testing scenarios, there is a notable absence of a unified framework for modeling them. In this paper, we introduce a unified data-driven ATS framework that conceptualizes the various testing formats as a hierarchical test structure search problem. It can learn directly from data to solve for the optimal questions for each student, eliminating the need for manual test design. The proposed solution algorithm comes with theoretical guarantees for estimation error and convergence. Empirical results show that our framework maintains assessment accuracy while reducing question count by 20% on average and improving training stability.
APA
Yu, J., Zhuang, Y., Huang, Z., Liu, Q., Li, X., Li, R. & Chen, E.. (2024). A Unified Adaptive Testing System Enabled by Hierarchical Structure Search. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:57803-57817 Available from https://proceedings.mlr.press/v235/yu24r.html.

Related Material