Large Samples, Small Effects: A Diagnostic Framework Using Nonlinear Modeling and Sensitivity Analysis

Research on student burnout often relies on observational associations and predictive models, with limited attention to the robustness of inferred relationships. Using a large synthetic dataset of 150,000 students, this study examines the extent to which commonly cited predictors of burnout exhibit stable and substantively meaningful associations. We propose a general diagnostic framework for evaluating substantive significance and robustness in large-sample observational studies. We estimate ordinal regression models to preserve the ordered structure of burnout levels, explore nonlinear relationships using spline-based generalized additive models (GAMs), and assess robustness to unmeasured confounding using formal sensitivity analysis. Nonlinear relationships are examined using spline-based methods implemented via the MultiSpline R package. Across all modeling approaches, we find minimal evidence of strong or substantively meaningful associations. Nonlinear effects are largely flat, and predicted probabilities vary only marginally across the observed range of predictors. Sensitivity analysis further reveals extreme fragility: even very small unobserved confounding—explaining as little as 0.38% of residual variance in both the predictor and outcome—would be sufficient to eliminate estimated effects entirely. These findings demonstrate that large sample sizes do not guarantee meaningful inference and underscore the importance of combining flexible modeling with formal robustness assessment. The study is intended as a methodological demonstration, emphasizing the need for caution in interpreting observational predictors of complex outcomes such as burnout. Implications for research reporting conventions in education and educational psychology are discussed.