Automatic Scoring of Students’ Science Writing Using Hybrid Neural Network

Ehsan Latif, Xiaoming Zhai
Proceedings of the 2024 AAAI Conference on Artificial Intelligence, PMLR 257:97-106, 2024.

Abstract

This study explores the efficacy of a multi-perspective hybrid neural network (HNN) for scoring student responses in science education with an analytic rubric. We compared the accuracy of the HNN model with four ML approaches (BERT, ANN, Naive Bayes, and Logistic Regression). The results have shown that HHN achieved 8%, 3%, 1%, and 0.12% higher accuracy than Naive Bayes, Logistic Regression, ANN, and BERT, respectively, for five scoring aspects (p < 0.001). The overall HNN’s perceived accuracy (M = 96.23%, SD = 1.45%) is comparable to the (training and inference) expensive BERT model’s accuracy (M = 96.12%, SD = 1.52%). We also have observed that HNN is x2 more efficient in terms of training and inferencing than BERT and has comparable efficiency to the lightweight but less accurate Naive Bayes model. Our study confirmed the accuracy and efficiency of using HNN to automatically score students’ science writing.

Cite this Paper

BibTeX
@InProceedings{pmlr-v257-latif24a, title = {Automatic Scoring of Students’ Science Writing Using Hybrid Neural Network}, author = {Latif, Ehsan and Zhai, Xiaoming}, booktitle = {Proceedings of the 2024 AAAI Conference on Artificial Intelligence}, pages = {97--106}, year = {2024}, editor = {Ananda, Muktha and Malick, Debshila Basu and Burstein, Jill and Liu, Lydia T. and Liu, Zitao and Sharpnack, James and Wang, Zichao and Wang, Serena}, volume = {257}, series = {Proceedings of Machine Learning Research}, month = {26--27 Feb}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v257/main/assets/latif24a/latif24a.pdf}, url = {https://proceedings.mlr.press/v257/latif24a.html}, abstract = {This study explores the efficacy of a multi-perspective hybrid neural network (HNN) for scoring student responses in science education with an analytic rubric. We compared the accuracy of the HNN model with four ML approaches (BERT, ANN, Naive Bayes, and Logistic Regression). The results have shown that HHN achieved 8%, 3%, 1%, and 0.12% higher accuracy than Naive Bayes, Logistic Regression, ANN, and BERT, respectively, for five scoring aspects (p < 0.001). The overall HNN’s perceived accuracy (M = 96.23%, SD = 1.45%) is comparable to the (training and inference) expensive BERT model’s accuracy (M = 96.12%, SD = 1.52%). We also have observed that HNN is x2 more efficient in terms of training and inferencing than BERT and has comparable efficiency to the lightweight but less accurate Naive Bayes model. Our study confirmed the accuracy and efficiency of using HNN to automatically score students’ science writing.} }
Endnote
%0 Conference Paper %T Automatic Scoring of Students’ Science Writing Using Hybrid Neural Network %A Ehsan Latif %A Xiaoming Zhai %B Proceedings of the 2024 AAAI Conference on Artificial Intelligence %C Proceedings of Machine Learning Research %D 2024 %E Muktha Ananda %E Debshila Basu Malick %E Jill Burstein %E Lydia T. Liu %E Zitao Liu %E James Sharpnack %E Zichao Wang %E Serena Wang %F pmlr-v257-latif24a %I PMLR %P 97--106 %U https://proceedings.mlr.press/v257/latif24a.html %V 257 %X This study explores the efficacy of a multi-perspective hybrid neural network (HNN) for scoring student responses in science education with an analytic rubric. We compared the accuracy of the HNN model with four ML approaches (BERT, ANN, Naive Bayes, and Logistic Regression). The results have shown that HHN achieved 8%, 3%, 1%, and 0.12% higher accuracy than Naive Bayes, Logistic Regression, ANN, and BERT, respectively, for five scoring aspects (p < 0.001). The overall HNN’s perceived accuracy (M = 96.23%, SD = 1.45%) is comparable to the (training and inference) expensive BERT model’s accuracy (M = 96.12%, SD = 1.52%). We also have observed that HNN is x2 more efficient in terms of training and inferencing than BERT and has comparable efficiency to the lightweight but less accurate Naive Bayes model. Our study confirmed the accuracy and efficiency of using HNN to automatically score students’ science writing.
APA
Latif, E. & Zhai, X.. (2024). Automatic Scoring of Students’ Science Writing Using Hybrid Neural Network. Proceedings of the 2024 AAAI Conference on Artificial Intelligence, in Proceedings of Machine Learning Research 257:97-106 Available from https://proceedings.mlr.press/v257/latif24a.html.

Related Material