On the responsible use of metrics and indicators in research assessment

There is growing awareness that research assessment in many countries and disciplines creates an incentive system that contributes to making science less reliable and academia more unfair. This jeopardizes the bond of trust between the community at large and the world of research and tertiary education, with dangerous and widespread societal implications. Much of the blame is attributed to the inappropriate use of quantitative indicators, which spurred several initiatives to limit their impact and/or encourage their more appropriate use. We propose a set of guidelines for using indicators responsibly. We argue that indicators should reliably measure a well-defined quality dimension (e.g., impact, innovation, methodological rigour); be transparent, reproducible and non-proprietary; be readily available and economically accessible; and be adapted appropriately according to the context of the assessment exercise (e.g., seniority, field of research, evaluation goals). We also argue that the use of indicators should always be integrated with a qualitative, in-depth evaluation, and that a multi-dimensional approach – where a multiplicity of indicators inform on a multiplicity of constructs – is fundamental in most assessment exercises. These principles are presented with a focus on practical implementation in realistic evaluation settings, and with a focus on their psychometric foundations.