Promoting the Use of Research Evidence from Websites: Optimizing Microsurveys As Feedback Loops To Drive Improvement

Educators’ use of research evidence (URE) from intermediary websites is not well understood. Current studies measure URE from websites through periodic, retrospective user reports or by passively tracking website usage, neither of which adequately inform the continuous improvement efforts of intermediaries. This paper describes a series of pilot studies undertaken to understand the feasibility of microsurveys – brief website embedded surveys triggered by specific user behaviors – to gather real-time insights for improving the promotion of URE. Specifically, this report describes various microsurvey configurations to optimize microsurvey response rates.A plug-in embedded microsurveys across web pages. Microsurveys used two questions: a 5-point Likert emoticon rating scale and an open-ended follow-up. Four pilot studies were conducted: (a) question-varying pilot, (b) time-varying pilot, (c) click-varying pilot, and (d) question-varying under optimized conditions pilot. Chi-square tests of independence and logistic regression were used to assess differences in response rates and the relationship between conditions, scores, and response rates. Response rates were improved by discarding low performing microsurvey questions, selecting an appropriate microsurvey time trigger, and reducing the number of clicks to complete the microsurvey. Under optimized conditions, Likert scale question response rates increased from 4.18% to 11.31%. The response rate to the open-ended follow-up question, however, remained stable with engagement predicted by favorability score on the preceding Likert scale question. This study establishes the potential of microsurveys for measuring URE from intermediary websites by achieving response rates understood to yield reliable estimates for the sake of improving the promotion of evidence in practice. Future research should explore additional configurations to further optimize response rate, integrate microsurveys with observational and behavioral data to generate validity evidence, and study the integration of microsurvey feedback into organizational change processes.