Intelligent tutoring systems need real teachers

The development and distribution of digital learning software, such as intelligent tutoring systems (ITSs), have evolved into a billion-dollar industry, impacting a vast number of students worldwide. While ITSs have significant potential to effectively transform learning and support teachers, the extent to which ITSs rely on teacher support to maintain students’ engagement with the ITS is less well understood. This study evaluates exceptionally rich data from an ITS for learning mathematics used in Germany and the Netherlands (~139,000,000 problems; n ~ 194,000 students) between 2016 and 2023. We compared whether students’ engagement varied based on how they used the ITS: with teacher support (i.e., teachers assigned mathematics problems to students) versus without teacher support (i.e., students self-assigned problems). Our results indicated that students in Germany and the Netherlands consistently (i) dropped out later, (ii) were active for significantly more weeks, and (iii) worked through more mathematics problems each week with teacher support as compared to without teacher support. We observed these results robustly across academic school years within the examined period. Overall, our study challenges the usefulness of digital learning software as stand-alone tools and emphasizes the need for embedding such software in real-life learning scenarios involving teachers.