Adult Online Assessments of Arithmetic, Vocabulary, Reasoning, and Math Self-Assessment: An Evaluation of the Feasibility, Reliability and Validity.

Online, at-home testing of scholastic skills can widen participation in research, but needs evidence on quality and practicality. We ran two pilots in samples of Dutch adults to: (a) compare a digital version to the official pen & paper version on a short arithmetic test (TempoToets Rekenen; TTR), (b) evaluate two online vocabulary tests, (c) benchmark a developing abstract-reasoning test (OMSS), and (d) examine the Dutch translation of the Math Ability Rating Questionnaire (MARQ) self-report.Pilot I (N = 55) included home online testing and supervised lab testing (online and pen/paper). Home: TTR, Matrix Reasoning Item Bank (MaRs-IB), synonym vocabulary test, Dutch Auditory & Image Vocabulary Test (DAIVT). Lab: TTR, synonym test, DAIVT, the Peabody Picture Vocabulary Test (PPVT-III) and the OMSS. Pilot II (N=61) repeated TTR across three settings; home-digital, lab-digital, lab-pen/paper and compared it to (subsets of) the MARQ. Outcomes included internal consistency (α), Pearson correlations (r), intraclass correlation coefficients (ICC) from mixed models, administration time, and feasibility indicators.Arithmetic (TTR): In Pilot I, cross-format associations were moderate (r = .63). In Pilot II, home-digital vs pen/paper r = .69, lab-digital vs pen/paper r = .70, and home- vs lab-digital r = .79. Scores were generally higher on pen/paper than on the digital versions (modality effect), but the two digital settings were more consistent.Vocabulary: Both tests were reliable and convergent with PPVT: DAIVT α = .90; r = .76; synonym α = .85; r = .66. The synonym test was shorter than the DAIVT (~4.5 vs. 7 min) and required no audio, favouring remote feasibility; DAIVT provided stronger convergence, likely reflecting the similar format to the PPVT.Abstract reasoning: OMSS showed acceptable preliminary indices versus MaRs-IB (α = .67; r = .56; ICC = .51; ~11 min). A ~20-item OMSS form appears efficient with limited expected loss in precision.Self-reported math (MARQ): Internal consistency was strong (α > .80 for MARQ-19/5/3). Higher MARQ scores (poorer self-reported arithmetic ability) were weakly to, at best, moderately correlated with TTR (timed arithmetic performance); the strongest being for MARQ-5 (r = −.33, p = .011).For the remote, online version of the TTR standardized administration (clear on-screen instructions, short practice, consistent device) is crucial to limit modality-related variance. For vocabulary, both online tests are suitable; the synonym test is pragmatic for remote use, whereas DAIVT offers stronger convergence at greater time cost. OMSS is promising pending item refinement and shortening. MARQ–TTR associations were weaker than expected, suggesting differences in construct coverage (broad self-report vs. timed operations), translation and sample effects. Within MARQ, the MARQ5 subset appears most suitable in our context due to its brevity, high internal consistency, and the strongest, albeit modest, correlation with TTR.