Cross-Nationally, Non-Probability Web Surveys Demonstrate Poorer Demographic Coverage and Yield More Liberal Estimates of Public Opinion than F2F Surveys

Non-probability web surveys offer several advantages over face-to-face (F2F) interviews—they are cheaper, faster, more accessible, and reduce interviewer effects and desirability bias. As such, they are increasingly popular in both academic and commercial research. However, they often yield demographically biased samples, raising concerns about the accuracy of the resulting public opinion estimates. Most studies on web surveys focus on developed countries, with limited attention to non-WEIRD contexts. Addressing this gap, we compare data from the Values in Crisis (VIC) survey (2020–2021, opt-in online panels with quota sampling) and the 7th wave of the World Values Survey (WVS; 2017–2022, probability-based multi-stage stratified sampling) across nine countries: Brazil, Chile, China, Germany, Greece, Japan, Kazakhstan, Russia, and the United Kingdom. We assess differences in demographic coverage and national mean scores on 15 attitudinal items, including trust, national pride, religiosity, life satisfaction, postmaterialism, and support for gender equality and reproductive freedom. WVS samples more closely match national demographics, though VIC performs comparably in some cases. Substantively, web surveys show higher (up to 20–30 p.p.) acceptance of gender equality, abortion, and homosexuality and lower religiosity, national pride, and life satisfaction. These differences persist even after adjusting for demographic imbalances and restricting WVS samples to internet users.