Machine Learning Approaches for Meta-Analytic Estimates of Important Predictors in Behavioral Science Studies: An Analysis of Cooperation in Social Dilemmas

Research in the social and behavioral sciences is accumulating at an exponential rate. One of the challenges facing scientists is how to quantitatively determine, in an unbiased manner, which predictors contribute the most to explaining variation in a specific phenomenon. To address these issues and improve the predictor importance estimation, we propose an enhanced grouped permutation feature importance (GPFI) method using a state-of-the-art ensemble machine learning regression model. A simulation study utilizing four artificial datasets demonstrated that the mean absolute percentage error of importance estimation was reduced to 5.0% with the proposed method compared to 30.0% with the conventional GPFI method. As an applied example, we utilized the Cooperation Databank to assess the relative importance of 106 predictors of cooperation, including parameters of the study’s experimental paradigm (e.g., group size, incentive structure, and repeated interaction) and sample characteristics (e.g., gender, age, and ethnicity), and found that the proposed technique was able to identify the top predictors of cooperation. These results clarify the implications of such information for understanding and promoting cooperation. The analytical methods developed in this study can be applied across the social and behavioral sciences, especially in well-developed topics that involve accumulated empirical studies.