Public lecture by Professor Trish Greenhalgh and Professor Anne Kelso
21 March 2018.
Measuring the impact of research is receiving ever-greater priority from research institutes, funding bodies and even researchers themselves. Yet an over-emphasis on many commonly used impact metrics can paradoxically encourage short-term thinking that camouflages the real value of research.
That was the message from celebrated UK academic and widely published researcher Professor Trish Greenhalgh, who addressed a packed lecture hall at the University of Sydney on the tensions inherent in measuring research impact and lessons for Australia from the UK experience.
Speaking at an event co-hosted by the Sax Institute and the three National Health and Medical Research Council (NHMRC) partnership centres on 19 March, Professor Greenhalgh urged listeners to “move beyond the metrics game”, and cautioned against a tendency to rely exclusively on short-term or surrogate metrics. While these had the advantage of being relatively easy to measure, they did not necessarily align well with the strategic mission of the research group or to impacts likely to bring beneficial change.
Having a paper accepted for publication by a journal with a high impact factor was an example of a potentially misleading metric, as this said little about the quality of the paper or the degree to which the paper would be read or its findings adopted. When attempting to measure impact, she said researchers would do well to include non-linear and longer-term mechanisms, which had been shown to play a big role in achieving beneficial change. Strong and ongoing links to policy makers was one approach that had proven helpful in ensuring researchers remained aligned to the bigger picture behind a research program.
Co-presenter and NHMRC CEO Professor Anne Kelso AO told the meeting, co-hosted by the Australian Prevention Partnership Centre at the Sax Institute, the Cognitive Decline Partnership Centre at the University of Sydney and the Partnership Centre for Health System Sustainability at Macquarie University, that the NHMRC was also seeking to improve its focus on impacts.
While impact “may be indirect, difficult to attribute and take time”, Professor Kelso said the NHMRC was moving “very deliberately” to include more rigorous measures in its grant assessment processes, including by no longer considering the number of prior grants won by a researcher as a criterion for the assessment of the researcher’s track record in applications for future funding.
Watch the recording
- You can download a PDF of Professor Trish Greenhalgh and Professor Anne Kelso’s lecture slides from the event here.