Prompting for Creative Problem-Solving: A Process-Mining Study

Read the full article See related articles

Listed in

This article is not in any list yet, why not save it to one of your lists.
Log in to save this article

Abstract

Background: Although generative‑AI systems are increasingly used to solve non‑routine problems, effective prompting strategies remain largely underexplored.Aims: The present study investigates how university students prompt ChatGPT to solve complex ill-defined problems, specifically examining which prompts are associated with higher or lower problem-solving performance.Sample: Seventy-seven university students (53 women; Mage = 22.4 years) participated in the study.Methods: To identify various prompt types employed by students, the study utilized qualitative analysis of interactions with ChatGPT 3.5 during the resolution of the creative problem-solving task. Participants’ performance was measured by the quality, elaboration, and originality of their ideas. Subsequently, two-step clustering was employed to identify groups of low- and high-performing students. Finally, process-mining techniques (heuristics miner) were used to analyze the interactions of low- and high-performing students.Results: The findings suggest that including clear evaluation criteria when prompting ChatGPT to generate ideas (rs = .38), providing ChatGPT with an elaborated context for idea generation (rs = .47), and offering specific feedback (rs = .45), enhances the quality, elaboration, and originality of the solutions. Successful problem-solving involves iterative human-AI regulation, with high performers using an overall larger number of prompts (d = 0.82). High performers interacted with ChatGPT through dialogue, where they monitored and regulated the generation of ideas, while low performers used ChatGPT as an information resource.Conclusions: These results emphasize the importance of active and iterative engagement for creative problem-solving and suggest that educational practices should foster metacognitive monitoring and regulation to maximize the benefits of human-AI collaboration.

Article activity feed