Measurement and Automated Scoring of Scientific Creative Thinking

Scientific creativity, a key driver of modern scientific innovation, plays a crucial role in shaping education, fostering innovation and enhancing problem-solving abilities. Research on scientific creative thinking is vital for advancing higher education reform and optimizing talent development models. To better understand scientific creativity, it is essential to have reliable assessment tools. In this study, we developed a scientific creativity test based on these real-world instances of scientific creativity. By utilizing authentic scientific invention scenarios and adopting a learning-testing paradigm, we constructed the Scientific Divergent Application Task (SDAT). The psychometric properties of the SDAT were examined through various reliability and validity indices. The results indicated that the SDAT demonstrates good internal consistency, as indicated by Cronbach’s α and confirmatory factor analysis. Furthermore, SDAT scores showed significant correlations with originality scores on the AUT, openness to experience, scientific creative behavior scores on the ICAA, and semantic distance, providing strong evidence for its construct validity in measuring individual scientific creativity. Test-retest reliability, assessed with an independent sample, demonstrated the temporal stability of the SDAT. Additionally, the study established an automated scoring system by fine-tuning multiple large language models for the original evaluation of SDAT responses, thereby improving both the objectivity of scoring and the efficiency of the test.The SDAT not only offers a valuable tool for measuring scientific creativity but also provides important insights into the cognitive mechanisms of scientific creativity, particularly through the dynamic relationship between knowledge acquisition and innovation.Keywords: creativity assessment; scientific creativity; scientific creative thinking;semantic networks; large language models