Conformal Prediction for Students’ Grades in a Course Recommender System
Proceedings of the Eighth Symposium on Conformal and Probabilistic Prediction and Applications, PMLR 105:196-213, 2019.
Course selection can be challenging for students of Liberal Arts programs. In particular, due to the highly personalized curricula of these students, it is often difficult to assess whether or not a particular course is too advanced given their academic background. To assist students of the liberal arts program of the University College Maastricht, Morsomme and Vazquez (2019) developed a course recommender system that suggests courses whose content matches the student’s academic interests, and issues warnings for courses that it deems too advanced. To issue warnings, the system produces point predictions for the grades that a student will receive in the courses that she/he is considering for the following term. Point predictions are estimated with regression models specific to each course which take into account the academic performance of the student along with the knowledge that she/he has acquired in previous courses. A warning is issued if the predicted grade is a fail. In this paper, we complement the system’s point predictions for grades with prediction intervals constructed using the conformal prediction framework (Vovk et al., 2005). We use the Inductive Confidence Machine (ICM) (Papadopoulos et al., 2002) with normalized nonconformity scores to construct prediction intervals that are tailored to each student. We find that the prediction intervals constructed with the ICM are valid and that their widths are related to the accuracy of the underlying regression model.