Convex Hull Applications to Natural Language Psychometrics

Psychological measurement plays a vital role in many areas of science. Traditional methods for developing and scoring measurement instruments require large sets of human responses, making them time-consuming and costly. Recent advances in artificial intelligence (AI) offer new ways to tackle these challenges. In this brief note, we explore the use of convex hulls—a concept from computational geometry—in combination with AI-driven large language models to enhance psychometric practice. A convex hull is the smallest convex boundary around a set of points. By treating language embeddings as high-dimensional coordinates and forming convex hulls, we can interpret the structure of items and free text responses in new ways. We propose two novel applications of convex hulls: (1) item analysis without data: We use convex hulls to check if a test item “belongs” to a scale, providing a new indicator of item quality; and (2) scoring free text: By interpreting candidate responses in relation to the convex hull of other responses, we propose an objective way to score psychological constructs from natural language. Each application is possible in supervised and unsupervised modes. These methods are experimental and yet to be validated. We outline, but do not implement, brief methods for testing their efficacy. We discuss open questions that we expect will impact the utility of these methods, such as a) why we do not ask the LLM to score the items and responses, b) what the measurement scale of the proximity scores is c) why we preferred convex hull centroids over clustering and scale embedding means d) what happens of the construct hull does not match the intended target and e) what centroids and other hull attributes represent (e.g., the intensity or essence of constructs) and the implications for item analysis and scoring.