A pilot implementation of Studiosty’s AI enabled Writing Feedback + service - exploring students’ experiences, approaches to utilisation and outcomes

The rise of AI (Artificial Intelligence) as an assistive technology has seen a significant increase in the last two years in Higher Education and the wider educational sector. Institutions are exploring ways to harness the power of AI as an approach to automate and enhance students’ learning. This study examines the change from a human-led feedback service – Studiosity’s WF (Writing Feedback) to an AI-led service – WF+ (Writing Feedback+). Students’ initial experiences of the change to AI-led feedback are evaluated using four data sources: Studiosity’s own internal survey – offered to students upon the reception of feedback; an excerpt from a University-level survey conducted to perform a deep-dive into students’ experiences; utilisation timing and finally examining the impact of service utilisation upon retention and attainment. At a high-level the findings indicate exclusive experience of human or AI feedback tends to yield equal levels of satisfaction, but AI is rated less well by students experiencing both approaches. Students’ priorities and timing approaches to system utiliation differ very little, but there is evidence to suggest enhanced attainment and retention amongst those who engage with both human and AI feedback – indicating the potential to utilise the services to detect assessment engagement. The paper is designed to serve as an example of the types of early and mid-term evaluation other practitioners may wish to use to monitor their own switch to an AI-led feedback service.