Large Language Models and Shifts in Scholarly Writing Style: A Cross-Journal Quantitative Analysis of Ophthalmology Research Articles
Discuss this preprint
Start a discussion What are Sciety discussions?Listed in
This article is not in any list yet, why not save it to one of your lists.Abstract
Large language models (LLMs) are increasingly integrated into scientific writing workflows, raising questions about whether their widespread availability may influence the language of the published scientific record. We conducted a longitudinal text analysis to examine whether stylistic features of research articles changed following the introduction of widely accessible LLM tools. A corpus of 862 full-length original research articles was assembled from four general ophthalmology journals representing Clarivate Journal Citation Reports quartiles Q1–Q4. Articles were sampled systematically by journal-month from pre-LLM (January 2018–December 2020) and post-LLM (January 2023–July 2025) periods. Using an automated text-processing workflow, we quantified lexical discourse markers and punctuation features associated with editorial and connective phrasing patterns in scientific writing. Feature frequencies were normalized by article length, and a composite stylistic divergence index was constructed using standardized feature values within each quartile. Post-LLM articles showed measurable stylistic shifts, most pronounced in Q3 and Q4 journals. Several discourse and editorial markers increased in prevalence, punctuation patterns shifted, and the composite stylistic divergence index increased significantly in lower-quartile journals while remaining stable in Q1. Explicit disclosure of generative tool use was rare, occurring in fewer than 3% of post-LLM articles. These findings suggest that corpus-level stylistic patterns in scientific writing may be evolving in the post-LLM era and illustrate how quantitative analysis of linguistic features can help monitor technological influences on scholarly communication.