Estimating Dimensional Structure in Generative Psychometrics: Comparing PCA and Network Methods Using Large Language Model Item Embeddings

As large language models (LLMs) increasingly inform psychometric practice through item generation and semantic analysis, embedding-based approaches offer a pre-empirical pathway for assessing dimensional structure before human response data become available. However, the methodological choices used to recover dimensions from embedding spaces remain underexamined. The present study compared the performance of principal component analysis (PCA), which currently dominates generative psychometric applications, with network-based exploratory graph analysis (EGA) for estimating dimensional structure from item embeddings. Using Monte Carlo simulations across six personality facets nested within two domains, we generated 180 items per condition via LLM prompting and represented them as semantic embeddings. Dimensional recovery was evaluated across four analytic pipelines: Kaiser's rule with PCA extraction and varimax rotation, parallel analysis with PCA extraction and varimax rotation, EGA, and EGA combined with network-integrated item filtering. Results showed that PCA-based methods severely overestimated dimensionality and yielded poorer recovery of the generating structure. EGA demonstrated markedly improved performance relative to PCA-based approaches, whereas EGA combined with network filtering procedures achieved near-perfect dimensional accuracy and structural recovery. These findings indicate that network psychometric workflows provide more accurate structural representations than component-based approaches when analyzing embedding spaces, with direct implications for generative psychometric workflows in which item pools are evaluated prior to data collection. More broadly, the findings demonstrate that analytic choices made during pre-empirical assessment have substantive consequences for item selection, construct validity, taxonomic clarity, and downstream theoretical interpretation.