The Danger of Unscientific Surveys Swaying Public Policy: The Project Justice Colorado Family Court Reform Report, Voice of Survivors and a Call for Action-Preprint

Objectives: This policy-oriented critique examines the methodological weaknesses of the Project Justice USA survey and accompanying report, which have been used to advocate for legislative reform in the Colorado family court system. The study investigates whether the survey meets accepted scientific standards for sampling, measurement, and reporting, and assesses the risk of misinformation influencing public policy.Hypotheses: It was expected that the Project Justice survey would demonstrate significant methodological flaws, including biased recruitment, unvalidated measures, and unsupported policy conclusions.Methods: Two independent researchers reviewed the full Project Justice survey instrument and report. A structured content analysis was conducted on seventy-four social media recruitment posts collected between March and June 2025. Posts were evaluated for targeting bias, framing bias, emotional/priming language, and source credibility. Additional evaluations included methodological review of survey design, question phrasing, statistical reporting, and transparency of analytic procedures.Results: The analysis found that 90.5% of recruitment posts demonstrated targeting bias, and 94.6% included framing bias. The survey failed to report demographic information, relied on a self-selected sample recruited from advocacy networks, and included over sixty flawed survey items. Claims in the report often misinterpreted correlation as causation and lacked statistical transparency. No effect sizes or confidence intervals were reported.Conclusions: The Project Justice survey exhibits critical methodological deficiencies that undermine the validity and generalizability of its findings. Its unscientific presentation of advocacy-driven data poses a risk of misleading policymakers, family courts, and the public. Recommendations are offered to promotemethodological rigor, transparency as well as accountability when using survey data to inform legislative reform.