A red button does it all: An adaptive notification badge to nudge students to close their knowledge gaps in intelligent tutoring systems

Background: Intelligent tutoring systems (ITSs) often adaptively provide students with additional learning materials to close knowledge gaps recognized automatically by the system. In addition, students may be nudged with adaptive notification badges to close their knowledge gaps. Aim: This study evaluated the effectiveness of implementing an adaptive notification badge (small red button) reminding students to close their knowledge gaps. We also investigated the association between closing knowledge gaps and students’ overall performance, while controlling for other relevant factors (e.g., students’ usage behavior, system features, and teachers’ assignment types).Sample: We considered 105,081 students (in 6,668 classes working on ~25 million problems) who used an ITS for learning mathematics.Methods: We employed hierarchical linear regression analysis to evaluate whether students were more likely to close their knowledge gaps in a period in which notification badges were implemented in the ITS, as compared to a period without notification badges. Moreover, we used psychological network analysis to evaluate whether closing knowledge gaps was associated with students’ performance relative to other relevant factors.Results: Students closed 28% of their knowledge gaps when the adaptive red button notification badge nudged them but only closed 5% when there was no red button. Closing knowledge gaps predicted better student performance and was a more important predictor than students’ usage behavior, other system features, and teachers’ assignment types.Conclusions: Our study highlights the relevance of adaptive nudges within ITS for closing knowledge gaps and shows that closing knowledge gaps is linked to students’ performance.