Comparison of Self-Report Data Validity in Undergraduate Samples Using Remote Versus In-Person Administration Methods

In the Internet age, recruitment, participation, and compensation for survey research can occur remotely, away from a laboratory setting. Although this method of data collection offers notable benefits such as access to more diverse samples and lower study costs, it is possible that rates of inattentive or otherwise invalid response patterns are more common when survey completion occurs without any oversight. To answer this question, undergraduate student participants (final n = 678; 65% women, 76% White/European American) were randomly assigned to complete a battery of self-report surveys either in a typical laboratory administration setting (in person) or remotely from the location of their choosing. Following a preregistered analytic plan, data from both conditions were screened using multiple methods for self-report data validation, including the use of embedded and stand-alone validity scales, response timing, and identification of multivariate outliers. Results showed null-to-small differences between survey administration conditions. However, differences between screening methods in the proportion of data flagged as “invalid” emerged. The implications of these findings for study design and planned analyses are discussed.