Think Twice Before You Search: Deterrence Messaging Designs to Prevent Searches for Non-Consensual Intimate Images

Non-consensual distribution of intimate images (NDII) occurs when explicit images or content of a person, whether authentic or synthetic (AI generated), are distributed online or shared with others without their consent. This can lead to significant and long-lasting psychological, financial, privacy, and other harms. The ease with which machine learning tools are becoming accessible and affordable through websites and apps that allow users to produce and share synthetic NDII increases the prevalence and scale of harm. One way of addressing this problem is through deterrence messaging — notifications or other text aimed at influencing human decision-making and behavior. However, one dimension that is under-studied is the use of search to seek out NDII on social media platforms and search engines, which can be a pathway to sharing, creating, or normalizing the practice of NDII. In this report we examine deterrence messaging in search as one strategy for reducing NDII.