Measuring what matters: An environmental scan and quality appraisal of population well-being surveys

Population well-being surveys collect information from large, representative groups of people living in a defined geographic region on various social, emotional, and psychological factors that contribute to overall quality of life. Surveys provide relevant data to shape policies and practices to improve overall quality of life, such as informing the development of national quality of life frameworks. Ensuring that high-quality measures are being used in surveys is essential, because population level data inform policies that affect entire populations. The purpose of this study was to identify all existing population well-being surveys and analyze the quantity and quality of the established measures used in each survey. We conducted an environmental scan and identified 14 population well-being surveys, representing four continents, administered at monthly to 5-year intervals, and on samples of 3,928 to 65,000 individuals. We systematically appraised both the psychometry and pragmatism of each measure used in population well-being surveys through double-blind extraction using an established quality assessment tool. Overall measure quality was 33.05 (range: 20-46) and we found comparable domain scores for psychometric (M = 16.21) and pragmatic quality (M = 16.84). Psychometrically, measures tended to have high convergent validity and lower predictive validity. Practically, measures tended to have high readability scores and be briefer rather than longer. Our findings should be interpreted with our quasi-systematic method in mind, and our English language restriction. By highlighting how survey measures can be improved or refined in order to better measure population well-being, we hope these findings can support future population well-being surveys conducted across the world.