The Online Misinformation Susceptibility Scale: Development and Initial Validation

Background/Objectives: Considering the negative consequences of widespread online misinformation, there is a need for valid tools to measure online misinformation susceptibility. Thus, our aim was to develop and validate a tool to measure susceptibility to online misinformation, i.e. the Online Misinformation Susceptibility Scale (OMISS). Methods: First, we conducted a complete and thorough literature review to create a preliminary list of items for our scale. Subsequently, a multidisciplinary expert panel evaluated the content validity of these initial items. To establish face validity, we conducted cognitive interviews and computed the item-level face validity index. Moreover, we performed both exploratory and confirmatory factor analyses to validate the underlying construct structure of the OMISS. We examined the concurrent validity of the OMISS by using the fake news detection scale, the Trust in Scientists Scale, the single-item trust in scientists scale, the Conspiracy Mentality Questionnaire, and the single-item conspiracy belief scale. Additionally, we used Cronbach’s alpha, McDonald’s Omega, Cohen’s kappa, and intraclass correlation coefficient to examine the reliability of the OMISS. Results: Psychometric evaluation through both exploratory and confirmatory factor analyses supported a one-factor 9-item model for the OMISS. The OMISS demonstrated strong concurrent validity, evidenced by statistically significant correlations with the fake news detection scale (r = -0.135), the Trust in Scientists Scale (r = -0.304), the single-item trust in scientists scale (r = -0.280), the Conspiracy Mentality Questionnaire (r = 0.159), and the single-item conspiracy belief scale (r = 0.095). Cronbach’s alpha and McDonald’s Omega for the OMISS were 0.920 and 0.921, respectively. Intraclass correlation coefficient for the OMISS was 0.994 (p < 0.001). Cohen’s kappa for the nine items ranged from 0.732 to 0.968 (p < 0.001 in all cases) in test-retest study. Conclusions: Our comprehensive psychometric evaluations confirm the OMISS as a valid tool for measuring online misinformation susceptibility. Our findings support a one-factor 9-item model for the OMISS. The OMISS holds promise as an effective tool for identifying susceptibility to misinformation and may support policymakers, health educators, healthcare professionals, and stakeholders in pinpointing high-risk groups.