Rethinking Input Complexity in Transformer-Based Clinical Prediction: Implications for Feature Dimensionality and Sequence Length in Longitudinal EHR Data

Objective

Transformer-based models for clinical prediction using longitudinal electronic health record (EHR) data are often developed with large feature sets and long patient histories under the assumption that more data improves performance. However, high-dimensional inputs and long sequences increase computational burden, potentially limiting scalability. We evaluated how feature dimensionality and sequence length affect predictive performance, calibration, risk stratification, and computational efficiency in EHR prediction.

Methods

Using longitudinal EHR data from adults with mild asthma in an integrated healthcare system, we evaluated input representation design for predicting acute asthma exacerbation. Feature dimensionality was reduced using Integrated Gradients attribution scores, univariate performance-based selection, and clinically guided selection strategies. Sequence length was varied using percentile-based truncation of patient histories. Performance was assessed using discrimination, calibration, high-risk classification, threshold-based event capture, and computational efficiency across regions.

Results

Models using fewer features achieved discrimination comparable to the 80-feature reference model, with AUROC values ranging from 0.843 to 0.864 versus 0.870 for the full model. Moderate sequence-length truncation reduced training time by more than 70% with minimal loss in discrimination. Although reduced-dimensional models showed attenuation of predicted risk at the upper tail, they identified similar high-risk populations and captured comparable proportions of asthma exacerbation events at clinically relevant thresholds.

Conclusion

Transformer-based prediction models maintained strong performance across reduced feature sets. While dimensionality reduction modestly affected calibration at the highest risk levels, moderate sequence-length reduction substantially reduced computational burden with limited change in overall discrimination. These findings highlight trade-offs between input complexity, predictive performance, and computational efficiency.

Funding

National Heart, Lung, and Blood Institute, National Institutes of Health (R01 HL163049).

Lay Summary

Artificial intelligence models are increasingly used to predict clinical outcomes from electronic health record (EHR) data. Many modern deep learning models are built using large numbers of clinical variables and long patient histories because more data is often assumed to improve prediction. However, these complex models require substantial computing resources and may be difficult to implement in real-world healthcare systems.

In this study, we examined whether simpler input designs could maintain strong predictive performance while reducing computational burden. Using transformer-based models and longitudinal EHR data from adults with mild asthma, we systematically evaluated how the number of clinical features and the length of patient histories affected prediction performance and efficiency.

We found that models using fewer features and shorter patient histories performed similarly to more complex models for identifying patients at high risk of acute asthma exacerbation. Although simplified models showed modest differences in predicted risk estimates at the highest risk levels, they identified similar high-risk patient groups and captured similar numbers of asthma exacerbation events.

These findings suggest that deep learning models for healthcare prediction may not require highly complex input designs to achieve clinically useful performance, which could improve scalability and real-world implementation.