A preliminary, large-scale evaluation of the collaborative potential of human and machine creativity

Artificial intelligence research focuses on machine reasoning and knowledge-based problem-solving. Yet, Einstein observed that knowledge stagnates without human imagination. "Imagination,” he said, “is more important than knowledge. Knowledge is limited. Imagination encircles the world.” In this study, we examined whether machines can have the potential to improve innovation. We employed the validated Divergent Association Task (DAT), an objective and reproducible measure of the ability to generate solutions that diverge from known or taken-for-granted solutions already in use. Our experiments indicated that, contrary to speculations, machines are no more creative on average than diverse human samples. Further, human creativity surpasses that of large-language model (LLM) in the right-hand tail of the distribution, indicating that humans collectively possess more creative brilliance relative to machines. When we collaborated with LLM partners by instructing LLMs to take the DAT as if they were creative geniuses like da Vinci, Curie, or Jobs, or from the perspective of different demographic groups such as a black person or a female, DAT scores significantly dropped relative to independent human or baseline LLM scores. Moreover, when we repeatedly instructed LLMs to think more creatively, machines were unintentionally and unwittingly pushed to create fake solutions that could mislead uninformed human partners. These findings were based on about 10,000 human observations from diverse international, educational, and cultural backgrounds, and hundreds of thousands of test results from different LLMs. The findings suggest machine creativity differs substantially from knowledge-based problem-solving and that human-machine creative partnerships may potentially inhibit innovation.