How accurate are high resolution settlement maps at predicting population counts in data scarce settings?

Despite the recent milestone of the world population surpassing 8 billion, disparities in population data reliability persist, with many countries facing outdated or incomplete census data. Such inaccuracies have far-reaching implications for various sectors, including public health, urban planning, and resource allocation. The study capitalizes on the rich data environment provided by the detailed 2018 Colombian census data and census coverage indicators, offering a unique opportunity to assess census-independent population estimation approaches. Drawing from a diverse range of environmental landscapes in Colombia, the research evaluates the effectiveness of satellite imagery-derived settlement maps in conjunction with various modeling techniques. We explore two census-independent population estimation approaches based on settlement maps: pure data driven machine learning approach exemplified by a random forest model and a process-structure driven probabilistic approach exemplified by a hierarchical Bayesian model. Our findings underscore the efficacy of Bayesian modeling in addressing data scarcity and bias, providing robust estimates and quantifying model uncertainty. Nonetheless the random forest model is better suited when the data inputs are detailed and unbiased. Additionally, we highlight the importance of considering settlement map characteristics and population sample sizes in the modeling process. Through rigorous evaluation at different stages of the population modeling pipeline, including data input, model selection, and outcome assessment, this study offers insights into the requirements and challenges of leveraging satellite imagery-derived settlement maps for population estimation in data-scarce contexts.