The effect of introducing an electronic medical record system on data quality and factors associated with data quality across 187 HIV clinics in Kenya: An interrupted time series analysis from 2011-2018

Background: The objective of this evaluation was to estimate the effect of electronic medical record system (EMR) implementation on the quality of data uploaded to the District Health Information System Version 2 platform (DHIS2). Methods: This was an interrupted time series analysis of DHIS2 data quality. Data were extracted from 187 Kenyan health facilities from January 2011 to June 2018 (i.e., spanning 30 quarters). The primary exposure was presence of EMR, and the primary data quality outcomes were quarterly composite discrepancy scores and composite completeness scores. The composite discrepancy score depicted the extent of deviation of observed values from plausible values based on internal consistency checks. Higher discrepancy scores reflected worse data quality. The composite completeness score (CCS score) was a percentage measure of the extent of documentation of pre-selected variables. A 2017 cross-sectional facility survey was used to assess factors associated with data quality. We conducted an interrupted time series analysis to determine changes in the trend of data quality scores before and after EMR implementation. We conducted multivariable linear regression analyses to determine factors associated with data quality. Results: There was no statistically significant level change or effect in composite discrepancy scores comparing pre-EMR period and the post-EMR period. In the cross-sectional analysis, on average health centers had higher composite discrepancy scores compared to dispensaries thus worse data quality (0.066; 95% CI: 0.002-0.130, p=0.045), high volume facilities (>500 patients) had higher discrepancy scores than low volume facilities (0.090; 95% CI: 0.043-0.138, p<0.001), and operating the KenyaEMR system was associated with less discrepancy scores and thus better data quality (0.058; 95% CI: -0.107- -0.008, p=0.024] than the IQCare system. Regarding CCS, there was a significant drop in composite completeness scores (CCS) after transitioning to EMR. The average CCS in the first quarter post-EMR was lower than the average CCS in the quarter preceding EMR implementation (6.96; 95% CI: -9.15 – -4.77, p<0.001). After six quarters post-EMR implementation, CCS declined steadily with an average quarterly change in CCS that was 1.20 percentage points lower than the average quarterly trend pre-EMR (95% CI: -1.70 – -0.69, <0.001). In cross-sectional analysis, health centers (8.16; 95% CI: 3.94 – 12.37, p<0.001) and hospitals (10.39; 95% CI: 5.96 – 14.80, p<0.001), high facility volume (4.54; 95% CI: 1.06 – 8.02, p=0.010) and high HIV burden county (3.95; 95% CI: 0.19 – 7.70, p= 0.039) were associated with higher CCS compared to dispensaries, low facility volume, and low HIV burden, respectively. Conclusions: EMR implementation did not demonstrate evidence for significant positive impact on DHIS2 data quality, as indicated by the lack of improvement in composite discrepancy scores and a drop in composite completeness scores post-EMR implementation. Our findings suggest that EMRs are not sufficient to ensure high-quality data. Facility characteristics (like higher level facility, high volume, and being in a high HIV burden county), and KenyaEMR use appear to be associated with discrepancy and completeness of data. Further research to explore the mechanistic link between EMRs, data quality, and context will be necessary to optimize the use of EMRs to improve data quality in routine health information system data in LMICs.