Evaluation can be a transformational step in improving the implementation and impact of a health service or program. While the focus is often on the all-important results, evaluation can also be a crucial tool for supporting change along the way.
The Sax Institute’s Director of Evaluate, Anne Redman, has over two decades of experience in the field of evaluation, and leads a team skilled in identifying if, how and why a program or policy is working. Here she shares her tips for making the most of evaluation evidence.
See it as a tool for feedback and continual improvement
“Collecting good evidence is the first step in figuring out if your policy or program is making an impact,” says Anne. “That’s really what evaluation is all about – looking at the nuts and bolts of a program or service, and asking: what’s working? What isn’t? What can you tweak to make things better? And in the end, what impact did this have?”
Anne points out that while the evaluation effort is often focused on the final outcomes, which of course are critical, evaluation can also provide valuable real-time feedback. “We encourage people to see their evaluation as a tool for improvement. It’s not just about the end result – you really can be using your evaluation during the evaluation period to help understand your progress and improve the program as you’re implementing.
“For example, we’ve been working with the Victorian place-based suicide prevention trials since 2017. Over those four years, ongoing evaluation evidence has provided opportunities for learning, which have supported enhancements along the way. This included evidence from the 2020 evaluation report contributing to the Victorian Government deciding to extend the trials for an additional two years to enhance their sustainability.
“While evaluation helps you understand what has been achieved – it can also create opportunities for change that can ultimately increase the value and impact of the services and programs.”
Use your evaluation to map the future
An evaluation brings together a wide range of data sources and evidence to understand how well a service or program is working and ways in which it can be improved. Anne says this wealth of information can offer unique insights into how to steer a program or policy into the future.
She points to her team’s recent work with the NSW Ministry of Health on a secondary triage initiative for aged care residents. Introduced in March 2020, in partnership with NSW Ambulance, this initiative aims to reduce clinically avoidable low‑acuity transfers from residential aged care facilities to emergency departments, to minimise the risks for vulnerable patients.
A core component of the initiative is a secondary triage line staffed by emergency medicine specialists, who consult with the aged care facility Registered Nurse to determine if the resident does require transport to hospital (via ambulance or the Patient Transport Service) or whether an appropriate care plan could support them being cared for in situ.
“The evaluation found considerable support for the secondary triage model and made recommendations for enhancing and adapting the service for different health contexts,” explains Anne. The evaluation recommendations also drew on existing evidence regarding secondary triage systems. “That’s the value of evaluation. It can adapt to your needs and answer very specific questions to help map the future of a service.”
Failure can be just as insightful as success
Naturally we look to evaluations to show what’s working, but Anne says it’s just as important to learn from what’s not working.
“This requires an environment with a good level of trust,” she says. “Working in a transparent and collaborative way can create some safety around that and enable people to review any shortfalls with an open mind.”
“Often it’s the things that aren’t working that provide a powerful opportunity for change. And for this reason, we recommend that people embed an evaluation early in a program or policy. This way you can see right from the beginning how things are working, and change and implement as you go. Ultimately, evaluation is about helping people make beneficial change in their health service or program.”
Find out more about the Sax Institute’s Evaluate service here.
If you have a program or policy that needs evaluating, contact anne.redman@saxinstitute.org.au
