Exploring the Construct Validity of Integrated Speaking Tasks: The Case of a Large-scale High-stakes Computer-based Listening-Speaking Task

Integrated speaking tasks have been widely used in many large-scale high-stakes tests. However, little is known about their application among low- or intermediate-level English second language learners, such as in the Guangdong Version of the Computer-based English Listening and Speaking Test of the National Matriculation English Test. This misalignment is particularly problematic given the substantial impact of integrated speaking tasks on teaching, learning, and assessing of English language education in China and internationally. To address this gap, the present study employed a bi-factor Exploratory Structural Equation Modeling (ESEM) and hierarchical multiple regression analyses, with data from a sample of 360 participants in a real test, and probed whether and to what extent the test actually assesses the ability it is supposed to test as manifested in its official test specifications issued by the Education Examinations Authority of Guangdong Province. Findings indicated that the test did measure students’ ability of accomplishing certain tasks in specific contexts by acquiring and applying various knowledge sources (e.g. tasks prompts, encyclopedic knowledge of English and the world, source material, communicative strategies, etc.) in English. Specifically, parameters from the five domain-specific textual factors and two communicative strategies, extracted from the participants’ oral output, co-worked on the participants’ performance in the test, with varying weights across factors. This variability highlights the comprehensiveness and contextual specificity of the test. These findings could provide empirical evidence supporting the validity of score interpretations and offer important implications for the teaching, learning, and assessment of integrated speaking tasks in senior high schools in China.