The Trainee Digital Growth Chart - An Electronic Health Record Audit Log-Based Dashboard for Assessing Resident Information Gathering Skills: An Observation Cohort Study

Background Information gathering is the foundational skill of clinical reasoning. However, residents and attending physicians lack objective insights into resident competencies in information gathering from the electronic health record (EHR). The EHR audit logs, time stamped records of user activities, may offer a wealth of information about how residents gather patient information. In this work, we sought to: 1. Establish a benchmark of attending physician expectations of residents’ EHR-based information gathering activities at different stages of residency, 2. Develop and pilot a dashboard - the Trainee Digital Growth Chart - to feedback an audit log-derived peer-benchmarked composite measure of information gathering performance to residents and their attending physicians in pediatric hospital medicine (PHM), and 3. Measure if feedback and assessment on objectively derived information gathering performance promotes future improvement. Methods We convened a focus group of PHM attending physicians to establish information gathering benchmarks for residents at each stage of their training. We queried EHR audit logs for the information gathering activities of each resident against these benchmarks and piloted the dashboard in a 4-arm observational cohort study during the 2023-24 academic year, with arms based on different degrees of exposure to the dashboard (including none). We used an interrupted time series-like analysis to assess for change in performance before and after Growth Chart exposure, and we conducted a subgroup analysis for interns. Results 56.9% (29/51) of residents who had historical Growth Chart data available to examine 30 days before or during a PHM block, logged into the Growth Chart, and 24% of these (7/29) also discussed the performance with their attendings (comprising the group with the highest exposure). Interns in this highest exposure group were noted to have significant improvement in future information gathering during daily rounding and inpatient activities. Conclusion Through the EHR audit logs, it is possible to passively measure information gathering and to use this information in resident assessment and training. In our study, the most opportune time to positively influence information gathering behavior was during the intern year when EHR information-gathering habits are still being formed.