Beyond the "Wow" factor: Using Generative AI for Increasing Generative Sense-Making

Generative Artificial Intelligence (GenAI) has emerged as a transformative tool in education, offering scalable, individualized learning experiences. However, there is a notable lack of theoretically informed and methodologically rigorous research on how GenAI can effectively augment learning. This study addresses this gap by investigating the potential of a theoretically informed GenAI chatbot, ChatTutor, to facilitate generative sense-making, leveraging principles from generative learning theory. The study had two primary goals: first, to build on theory to propose how GenAI could be used to support generative sense-making; and second, to empirically test its impact on conceptual knowledge, self-efficacy, and trust immediately after the intervention, as well as on conceptual knowledge, enjoyment, and behavioral intentions in a follow-up test four weeks later. Conducted in an authentic university course, the pre-registered experiment with 175 students compared ChatTutor to a generic GenAI system (ChatGPT) and a teaching-as-usual condition. Results show that ChatTutor significantly enhanced trust, enjoyment, and behavioral intentions but not self-efficacy. While ChatTutor improved conceptual knowledge over ChatGPT immediately after the intervention, it did not outperform teaching-as-usual. Four weeks later, ChatTutor significantly outperformed teaching-as-usual but not ChatGPT in conceptual knowledge. The manuscript underscores the importance of integrating human-centered design and educational psychology theories into GenAI applications to optimize learning outcomes and proposes future research and practical implications.