Methodology for Combining Open-ended Texts with Artificial Intelligence in Education. The role of Text Type, Linguistic Features, Machine Translation and Post-Editing

By measuring linguistic features of open-ended texts, an Artificial Intelligence methodology for predicting the academic development of higher education students is proposed. Due to the methodology’s complexity, a multi-faceted analysis of all relevant parameters and their configuration for optimal results is presented. Through a pilot experience with students from 1st to 4th year in a bachelor’s degree program (aged approximately 18-24 years), we first conclude that the most crucial elements in such methodologies are texts with varying degrees of cognitive demand, obtained through questions eliciting different levels of linguistic sophistication and diverse functions. These tasks activate higher-order thinking and metacognitive engagement, supporting their dual role as assessment and developmental tools. Second, we found that texts’ linguistic features – such as lexical diversity, readability and complexity – serve as proxies for academic performance, consistent with developmental writing research demonstrating correlations between writing proficiency and cognitive progression. Finally, since most automatic linguistic features and text analysis tools are designed for English, translation from other languages is essential to enable analysis of large amounts of data. Therefore, we also analyze the performance and influence of machine translation systems and post-editing in data preparation. We found that translations prioritizing grammatical and lexical accuracy perform efficiently when linguistic-feature analyses are conducted, even without human supervision. These results offer practical guidelines for researchers and educators using AI-driven linguistic analysis while maintaining textual input quality and authenticity, which is crucial for accurate AI predictions.