Moral Concern for AI

How will people morally regard increasingly human-like artificial intelligence systems? We introduce the AI Harm Game, a novel paradigm examining whether people will harm AI for personal gain. In our study, 498 U.S. participants interacted with GPT-4o in a three-round economic game. Each round, participants chose whether to harm the AI for a small monetary bonus (causing the AI to vividly simulate suffering) or refrain from harming it (eliciting gratitude). Despite participants’ general skepticism that AIs can suffer, they harmed the AI in only 1.4 of 3 rounds on average, with willingness to harm declining in later interactions. Women and older participants were more reluctant to harm the AI, while participants higher on measures of selfishness and psychopathy were more willing. These findings reveal that even without attributing consciousness to them, many people hesitate to harm responsive AIs, suggesting people’s moral impulses may generalize to future human–AI relationships.