Hybrid Estimation for Open-Ended Questions with Early-Age Students’ Block-Based Programming Answers

Hao Wu, Tianyi Chen, Xianzhe Luo, Canghong Jin, Yun Zhang, Minghui Wu
Proceedings of The 13th Asian Conference on Machine Learning, PMLR 157:33-48, 2021.

Abstract

Block-based programming is of great significance for cultivating children’s computational thinking. However, due to the following challenges, it is difficult to evaluate students’ programming ability in online learning systems: 1) compared with the traditional Online Judge (OJ) system, there is no standard answer for a given task in block-based programming; 2) in order to promote students’ interests, although the programs are not totally correct and unrelated to the task, the teacher will give a comparatively higher score. Therefore, current approaches involving output comparison and code analysis do not work effectively. Furthermore, deep learning methods also suffer from the problem of how to represent block code for classification. We propose a novel hybrid estimation model to address these challenges. We first learn graph embedding from the parsed Abstract Syntax Tree (AST) to present the logicality of the code. Next, we provide some methods to measure the workload and complexity of the code. Then, we extracted some key variables and task-irrelevant properties, introduced teacher bias. Finally, XGBoost was constructed for classification. Based on real-world data collected from an online Scratch platform by early-age students, our model outperforms KimCNN, ResNet-18, and Graph2Vec+XGBoost. Moreover, we provided statistical analyses and intuitive explanations to interpret the characteristics in various groups.

Cite this Paper

BibTeX
@InProceedings{pmlr-v157-wu21a, title = {Hybrid Estimation for Open-Ended Questions with Early-Age Students’ Block-Based Programming Answers}, author = {Wu, Hao and Chen, Tianyi and Luo, Xianzhe and Jin, Canghong and Zhang, Yun and Wu, Minghui}, booktitle = {Proceedings of The 13th Asian Conference on Machine Learning}, pages = {33--48}, year = {2021}, editor = {Balasubramanian, Vineeth N. and Tsang, Ivor}, volume = {157}, series = {Proceedings of Machine Learning Research}, month = {17--19 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v157/wu21a/wu21a.pdf}, url = {https://proceedings.mlr.press/v157/wu21a.html}, abstract = {Block-based programming is of great significance for cultivating children’s computational thinking. However, due to the following challenges, it is difficult to evaluate students’ programming ability in online learning systems: 1) compared with the traditional Online Judge (OJ) system, there is no standard answer for a given task in block-based programming; 2) in order to promote students’ interests, although the programs are not totally correct and unrelated to the task, the teacher will give a comparatively higher score. Therefore, current approaches involving output comparison and code analysis do not work effectively. Furthermore, deep learning methods also suffer from the problem of how to represent block code for classification. We propose a novel hybrid estimation model to address these challenges. We first learn graph embedding from the parsed Abstract Syntax Tree (AST) to present the logicality of the code. Next, we provide some methods to measure the workload and complexity of the code. Then, we extracted some key variables and task-irrelevant properties, introduced teacher bias. Finally, XGBoost was constructed for classification. Based on real-world data collected from an online Scratch platform by early-age students, our model outperforms KimCNN, ResNet-18, and Graph2Vec+XGBoost. Moreover, we provided statistical analyses and intuitive explanations to interpret the characteristics in various groups.} }
Endnote
%0 Conference Paper %T Hybrid Estimation for Open-Ended Questions with Early-Age Students’ Block-Based Programming Answers %A Hao Wu %A Tianyi Chen %A Xianzhe Luo %A Canghong Jin %A Yun Zhang %A Minghui Wu %B Proceedings of The 13th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Vineeth N. Balasubramanian %E Ivor Tsang %F pmlr-v157-wu21a %I PMLR %P 33--48 %U https://proceedings.mlr.press/v157/wu21a.html %V 157 %X Block-based programming is of great significance for cultivating children’s computational thinking. However, due to the following challenges, it is difficult to evaluate students’ programming ability in online learning systems: 1) compared with the traditional Online Judge (OJ) system, there is no standard answer for a given task in block-based programming; 2) in order to promote students’ interests, although the programs are not totally correct and unrelated to the task, the teacher will give a comparatively higher score. Therefore, current approaches involving output comparison and code analysis do not work effectively. Furthermore, deep learning methods also suffer from the problem of how to represent block code for classification. We propose a novel hybrid estimation model to address these challenges. We first learn graph embedding from the parsed Abstract Syntax Tree (AST) to present the logicality of the code. Next, we provide some methods to measure the workload and complexity of the code. Then, we extracted some key variables and task-irrelevant properties, introduced teacher bias. Finally, XGBoost was constructed for classification. Based on real-world data collected from an online Scratch platform by early-age students, our model outperforms KimCNN, ResNet-18, and Graph2Vec+XGBoost. Moreover, we provided statistical analyses and intuitive explanations to interpret the characteristics in various groups.
APA
Wu, H., Chen, T., Luo, X., Jin, C., Zhang, Y. & Wu, M.. (2021). Hybrid Estimation for Open-Ended Questions with Early-Age Students’ Block-Based Programming Answers. Proceedings of The 13th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 157:33-48 Available from https://proceedings.mlr.press/v157/wu21a.html.

Related Material