There’s More to the Story: The Number of Response Options in Likert Scale Influences Only Internal Consistency, Not Criterion Validity

The existing research on the number of options in the Likert-type response format has focused primarily on reliability and descriptive statistics, often overlooking validity or examining it with limitations. This study addressed this gap through a within-subject experiment (N = 846, 69% women), manipulating response options (two, six, and ten) in two Likert-type scales: the Height Inventory and the autonomy subscale of the Basic Needs Satisfaction in General Scale. Two-point variants significantly differed in means and reliability compared to six- and ten-point versions, although the magnitudes of differences were small. On the other hand, validity (criterion, measurement model, and trait criterion validity) remained unaffected. Thus, the increased reliability may stem from systematic but attribute-irrelevant variance related to response format (i.e., method variance). These findings suggest that response formats with fewer options can be viable, particularly in scales with more items. Future research should explore differences in cognitive processes across response formats.