Evaluating Volunteering for Health: part 1

Sarah: 00:00:05

Hello and welcome to Volunteering Discovery.

Sarah: 00:00:08

This is a podcast which takes you behind the scenes of

Sarah: 00:00:10

volunteering in health and care.

Sarah: 00:00:12

In Norfolk and Waveney volunteers enhance the experience of people accessing health

Sarah: 00:00:16

and care in a huge variety of ways in our hospitals, our communities, and even

Sarah: 00:00:21

from the comfort of their own homes.

Sarah: 00:00:23

In each podcast, we'll be hearing from those who give their

Sarah: 00:00:26

time supporting others and the people who work alongside them.

Sarah: 00:00:30

I am your host, Sarah, a volunteer coordinator working in the NHS.

Sarah: 00:00:34

Why not subscribe to this podcast to make sure you hear all of the

Sarah: 00:00:38

upcoming episodes in the series.

Sarah: 00:00:40

In this two-part special, we're diving into the evaluation of volunteering for

Sarah: 00:00:45

health in Norfolk and Waveney, a program that brings together innovation in

Sarah: 00:00:49

volunteering in the healthcare outcomes.

Sarah: 00:00:51

In this first episode, producer Jules is joined by Dr. Jurgen Grotz, director

Sarah: 00:00:57

of the Institute for Volunteering Research and Evaluation partner in the

Sarah: 00:01:01

program, helping the partnership to develop and undertake the evaluation.

Sarah: 00:01:06

Jurgen brings a wealth of experience in understanding how volunteering

Sarah: 00:01:10

contributes to health and wellbeing, and today we'll explore the purpose, approach

Sarah: 00:01:14

and early insights from the evaluation.

Sarah: 00:01:17

Whether you're a practitioner, policymaker, or simply curious about

Sarah: 00:01:21

the impact of volunteering in health and care settings, this conversation

Sarah: 00:01:25

will offer valuable perspectives on how we measure what matters.

Jules: 00:01:31

Thank you very much for joining us today.

Jules: 00:01:34

Could I start by asking you to please introduce yourself?

Jules: 00:01:37

Who are you?

Jules: 00:01:38

What is your role, and what is your connection to volunteering for health?

Jurgen: 00:01:43

Well, first of all, thanks for inviting me, Jules.

Jurgen: 00:01:46

I am Dr. Jurgen Grotz, and I'm the director of the Institute for

Jurgen: 00:01:50

Volunteering Research here at the University of East Anglia in Norwich.

Jurgen: 00:01:57

Maybe I should say a little bit about what that institute actually is, as

Jurgen: 00:02:01

the name suggests, uh, my main job is to undertake research, research about

Jurgen: 00:02:07

everything to do with volunteering.

Jurgen: 00:02:11

Now volunteering is a pretty big thing in society here in England.

Jurgen: 00:02:16

If you ask 10 people on the street, more likely than not, seven or more

Jurgen: 00:02:22

will tell you that sometime in their life, they have already volunteered.

Jurgen: 00:02:26

They know what it is.

Jurgen: 00:02:28

That means it's, it's common.

Jurgen: 00:02:30

We find it everywhere, but that wouldn't tell us what difference it makes.

Jurgen: 00:02:35

To the volunteers themselves, but also to the courses that they volunteer for.

Jurgen: 00:02:40

That's sometimes where evaluations come in, and that's second

Jurgen: 00:02:45

part of your question is what is my role in this project?

Jurgen: 00:02:49

Obviously, my main job is doing research, but as part of that,

Jurgen: 00:02:53

I've done many evaluations over the years evaluations, looking at

Jurgen: 00:02:59

what difference volunteering makes.

Jurgen: 00:03:01

And that's why I was invited to work with colleagues of this program to look at

Jurgen: 00:03:06

how an evaluation could be undertaken.

Jules: 00:03:10

Thinking about the evaluation of the volunteering for Health

Jules: 00:03:13

program in Norfolk and Waveney, could you explain what an evaluation is

Jules: 00:03:18

and what it means specifically in this context, which is looking at

Jules: 00:03:22

volunteering connected with health?

Jurgen: 00:03:25

An evaluation is.

Jurgen: 00:03:27

Making a judgment, but at the very beginning, you have to decide what

Jurgen: 00:03:31

you're making a judgment about.

Jurgen: 00:03:33

I think in the majority of cases of the work that I'm involved in now,

Jurgen: 00:03:36

it is a question of making a judgment about whether or not a program or an

Jurgen: 00:03:41

activity actually makes a difference.

Jurgen: 00:03:45

Now in order to do that, and that explains a little bit more of my role

Jurgen: 00:03:49

within the program, is you have to ask yourself a number of questions,

Jurgen: 00:03:55

and so you already know me as the one who's always asking awkward questions.

Jurgen: 00:04:00

The first question here is, why do you actually want to evaluate.

Jurgen: 00:04:06

What is this about?

Jurgen: 00:04:08

What do you make a judgment about or who do you make that judgment for?

Jurgen: 00:04:13

Is it because the funder has said you have to do it?

Jurgen: 00:04:16

Is it because you want to make a case for more funding after the program

Jurgen: 00:04:22

finishes, or is it about learning how to do that program better?

Jurgen: 00:04:27

So there's a whole number of reasons why you want to evaluate,

Jurgen: 00:04:31

and that reflects then as well how you undertake your evaluation.

Jurgen: 00:04:35

That's the first awkward.

Jurgen: 00:04:36

Now, the second really awkward question is, what do you actually

Jurgen: 00:04:41

want to achieve with your program?

Jurgen: 00:04:44

So at one of our initial meetings, I was asking the other partnership

Jurgen: 00:04:49

members of the partnership what success for them would look like.

Jurgen: 00:04:54

What do they want to have achieved in two, three years time with the funding,

Jurgen: 00:04:59

with the activities of this program?

Jurgen: 00:05:02

So what do we actually want to achieve?

Jurgen: 00:05:04

That is a really important question.

Jurgen: 00:05:07

And it is absolutely great that here in this program we have started to

Jurgen: 00:05:11

ask this question at the beginning.

Jurgen: 00:05:13

There's no point asking it later on when starting your evaluation

Jurgen: 00:05:17

two thirds through the program.

Jurgen: 00:05:18

So we are very well set up there for that.

Jurgen: 00:05:20

But then the next stage, and this is, this is the.

Jurgen: 00:05:23

Awkward questions that I'm currently asking.

Jurgen: 00:05:26

This is about how do you actually get the information and what do you do to

Jurgen: 00:05:32

make that judgment, to make an informed judgment, an evidence-based judgment.

Jurgen: 00:05:37

People sometimes call it so.

Jurgen: 00:05:39

The first thing here would be asking the question, where do we

Jurgen: 00:05:43

actually get the information from?

Jurgen: 00:05:44

And, and then maybe at the very same time, especially if we're talking

Jurgen: 00:05:48

about volunteering in health and social care, is the information already here?

Jurgen: 00:05:53

Do we, you know, do, do we, do we need to watch out that we don't duplicate?

Jurgen: 00:05:56

And then also.

Jurgen: 00:05:58

I guess looking at how we use that information and especially then later

Jurgen: 00:06:04

on how we can share that information.

Jurgen: 00:06:07

No point just collecting information, putting it all into a nice folder

Jurgen: 00:06:12

saying we've collected all that information at the end and then

Jurgen: 00:06:15

saying, you know, that's, that's it.

Jurgen: 00:06:17

So those would be questions, awkward questions that I'm asking and that

Jurgen: 00:06:22

sort of, I guess is my, my role.

Jules: 00:06:25

Could you tell us in practical terms how you've gone about working

Jules: 00:06:28

with the group to start to develop the idea of what success would look like and

Jules: 00:06:33

therefore what we might need to measure?

Jurgen: 00:06:35

Okay, so this is, and this is again where the, where it was really

Jurgen: 00:06:38

nice that we started this very early.

Jurgen: 00:06:40

We set up actually a group within the partnership to,

Jurgen: 00:06:45

to consider the evaluation.

Jurgen: 00:06:48

And we've already had a number of meetings and we spent those meetings systematically

Jurgen: 00:06:53

working through those questions.

Jurgen: 00:06:56

At the very first meeting, we were looking at why we, why we evaluate.

Jurgen: 00:07:00

That was helping us.

Jurgen: 00:07:02

Together with what we want to achieve with thinking about what we want

Jurgen: 00:07:06

to achieve to establish the aim.

Jurgen: 00:07:08

This helps us to look at the overall picture at the end.

Jurgen: 00:07:13

What do we want to achieve with this?

Jurgen: 00:07:16

What we would then sometimes refer to as impact's, difficult to

Jurgen: 00:07:20

capture impact, but where we can certainly define some outcomes.

Jurgen: 00:07:25

Something that is.

Jurgen: 00:07:26

A direct result of the the program that isn't just what we

Jurgen: 00:07:31

sometimes refer to as an output.

Jurgen: 00:07:33

I say a little bit more about outputs in a minute, but we have been able,

Jurgen: 00:07:37

as a group to work through the process of asking these awkward questions

Jurgen: 00:07:42

to do that or to help with that.

Jurgen: 00:07:45

I'm using what's called as a Theory of change approach.

Jurgen: 00:07:50

I don't know whether it helps to, to speak a little bit to that.

Jurgen: 00:07:53

The Theory of change approach helps us to start at the end, to define that

Jurgen: 00:07:58

overall aim, to just define what we.

Jurgen: 00:08:01

What we want to achieve and then slowly walk back or think backwards.

Jurgen: 00:08:06

What do we need to to do to achieve that?

Jurgen: 00:08:10

When individual steps, looking at outcomes outputs, and then actually looking at the

Jurgen: 00:08:16

activities that we undertake to achieve those outputs, which then hopefully

Jurgen: 00:08:22

lead to those outcomes and together help us to achieve our overall aim

Jules: 00:08:28

at what stage?

Jules: 00:08:30

Are we at in the evaluation process at the moment?

Jurgen: 00:08:33

We've, we've got the theory of change, so that's just

Jurgen: 00:08:36

a simple visual, uh, reflection of how we want to achieve that.

Jurgen: 00:08:41

Uh, we also got all the necessary definitions.

Jurgen: 00:08:45

So we've got our aim, we've got the objectives very clearly defined,

Jurgen: 00:08:50

and really importantly, one of the first steps we've defined what we

Jurgen: 00:08:55

are talking about, the activities that we think of as volunteering.

Jurgen: 00:08:59

We've got all of that in place and what we're currently working on is

Jurgen: 00:09:03

what we call an outcome framework.

Jurgen: 00:09:06

That means we, we look at all the activities that we are undertaking.

Jurgen: 00:09:10

We are looking at how.

Jurgen: 00:09:12

They connect up to the outcomes that we want to achieve, and we ask questions

Jurgen: 00:09:20

from the activities that help us hopefully to make a judgment on whether

Jurgen: 00:09:25

or not we've achieved those outcomes.

Jules: 00:09:28

And of course, we are not yet at the stage of gathering the data,

Jules: 00:09:32

although we're very close to it.

Jules: 00:09:34

But during this process, have you gained any insights or had any challenges so far?

Jurgen: 00:09:42

I think the main insight that I have, and this is a nice, this is

Jurgen: 00:09:45

really nice with this project, is the, the, the ownership that that

Jurgen: 00:09:50

partnership members have taken of this.

Jurgen: 00:09:53

No evaluator or nobody in my role can sort of make up the ideas of the outputs

Jurgen: 00:09:59

or the particular measures that has to be owned by the people who actually.

Jurgen: 00:10:04

Collect the data, who work with the data and who know what they want to

Jurgen: 00:10:09

achieve overall, and this, this has been a real strength of this project,

Jurgen: 00:10:14

that right from the beginning with key members of the network, we have been

Jurgen: 00:10:18

working these things out together.

Jurgen: 00:10:20

It is a collaborative endeavor, and I think that's the main achievement at

Jurgen: 00:10:24

the moment in terms of the evaluation, that we get to the point where we

Jurgen: 00:10:28

can test the outcome framework.

Jurgen: 00:10:30

After a genuinely collaborative effort to get the outcome framework together.

Jules: 00:10:35

How about challenges?

Jurgen: 00:10:37

Well, the main challenge comes with my awkward questions.

Jurgen: 00:10:40

You know, not everybody at the outset agreed on why we need an evaluation,

Jurgen: 00:10:45

frankly, and uh, I think it's also fair to say that not everybody that was

Jurgen: 00:10:49

necessarily having the same view on what we want to achieve at the end of this.

Jurgen: 00:10:54

Now, we might still not have shared opinions on everything, but

Jurgen: 00:10:59

we've got a shared understanding.

Jurgen: 00:11:01

We are now evaluating the thing that we all know what we want to evaluate.

Jurgen: 00:11:05

That will avoid disappointment at the end.

Jurgen: 00:11:08

Because if somebody thinks we're evaluating X, but what we're actually

Jurgen: 00:11:12

doing is evaluating y, then that does not look good at the end.

Jurgen: 00:11:16

At the moment, we have achieved this sort of shared understanding, um, and it

Jurgen: 00:11:20

was a bit of, well, that's why it takes time, and so it was a bit of a challenge.

Jurgen: 00:11:25

It's difficult with point of time just to get everybody in a room.

Jurgen: 00:11:30

To have the time to make these, to, to, to have these conversations

Jurgen: 00:11:35

is really, really challenging.

Jurgen: 00:11:37

But people have been willing to give their time and it's been great.

Jules: 00:11:41

I'm skipping to the end now 'cause obviously there's still quite a

Jules: 00:11:45

considerable period of the program to run where we'll actually be gathering the data

Jules: 00:11:49

and, and starting the evaluation itself.

Jules: 00:11:52

But if we do skip towards the end, having now worked out what the.

Jules: 00:11:58

Outcome framework will be in having a theory of change, what do you

Jules: 00:12:02

hope that the program team and our wider stakeholders will be able to

Jules: 00:12:08

learn or use the final evaluation for at the end of the program?

Jurgen: 00:12:14

I think you already used the main word here.

Jurgen: 00:12:17

This the, the reason why we evaluate is learning, not just trying to

Jurgen: 00:12:23

find out what we can learn about the things that we undertake.

Jurgen: 00:12:27

That is quite an important part as well.

Jurgen: 00:12:29

We're actually doing innovative, we are undertaking innovative activities to learn

Jurgen: 00:12:34

whether or not they make a difference or not, or how they make a difference

Jurgen: 00:12:38

or not, but the overall program.

Jurgen: 00:12:42

It's about learning whether or not we can be better at achieving our overall aim.

Jurgen: 00:12:49

It's important to remember here, the overall aim isn't just numbers.

Jurgen: 00:12:53

You know, like more volunteers or more This, more that.

Jurgen: 00:12:56

The overall aim is reflects the aim of the funder.

Jurgen: 00:12:59

It's to test and if possible, show that people who don't usually.

Jurgen: 00:13:05

Volunteer in health and social care can actually be involved, or

Jurgen: 00:13:10

that those who are involved can be involved in different ways.

Jurgen: 00:13:13

This is about showing and making sure that volunteers are seen to be

Jurgen: 00:13:19

involved in health and social care in different and innovative ways.

Jurgen: 00:13:24

If we can learn how to do that or how not to do that for that matter, then

Jurgen: 00:13:29

that is the key part that I hope that people will take away at the same time.

Jurgen: 00:13:34

And let me just add that every time I do this in partnership with people,

Jurgen: 00:13:38

I hope that they also learn something about evaluating and about asking

Jurgen: 00:13:43

the awkward questions themselves.

Jurgen: 00:13:45

I don't always want to be the one who's asking the awkward questions,

Jurgen: 00:13:48

but if they can then take that away, not just the learning from the program.

Jurgen: 00:13:54

To actually go into the next program.

Jurgen: 00:13:56

That's obviously a, I always hope that that happens.

Jurgen: 00:13:59

Doesn't always, but take that into the next program.

Jurgen: 00:14:02

But also for them to already at the next program level, to be able to think about

Jurgen: 00:14:09

these evaluations themselves, to have that and that, that's one thing that I would.

Jurgen: 00:14:14

Suggest is it reflects sustainability, sustainability of the program as well,

Jurgen: 00:14:20

where we use the learning of the program to help define and, and, uh, create

Jurgen: 00:14:26

the next program and where we use the learning from, about the activity

Jurgen: 00:14:31

of evaluation to, to inform the next evaluation and to get, to make it better.

Jules: 00:14:37

I've come to the end of my questions for today, but is there

Jules: 00:14:39

anything else that you would like to add?

Jules: 00:14:41

Is there anything else that you think our listeners should

Jules: 00:14:43

know about this evaluation?

Jurgen: 00:14:46

Well, the one thing that I would like to add is the, what I hope

Jurgen: 00:14:51

is always part when we look at volunteer involvement, and that is the respect

Jurgen: 00:14:56

of the volunteer and what they do, the volunteers achieve what they achieve.

Jurgen: 00:15:02

Without being evaluated.

Jurgen: 00:15:04

Sometimes despite being evaluated or they don't come to a place to fill in our

Jurgen: 00:15:09

forms or to answer our questions, they come there to do what they want to do and

Jurgen: 00:15:14

to, um, to, to help where they can help.

Jurgen: 00:15:16

That, that respect is essential.

Jurgen: 00:15:19

That does not mean that we can't ask questions, awkward questions,

Jurgen: 00:15:22

or that we can't work together to find out the difference they make,

Jurgen: 00:15:27

and they themselves will be very.

Jurgen: 00:15:29

Interested in hearing about that difference and seeing how that works.

Jurgen: 00:15:33

But all work and all evaluation in this context has to start with the

Jurgen: 00:15:39

respect of the volunteer and the achievements that they actually make.

Jules: 00:15:43

Thank you, Jurgen.

Jules: 00:15:45

It would be great to talk to you again later in the, uh, the process

Jules: 00:15:48

of the program to check in and see how well everything is going

Jules: 00:15:51

and what learning we have so far.

Jules: 00:15:53

But until then, thank you very much.

Jurgen: 00:15:56

It's my absolute pleasure.

Sarah: 00:15:57

That brings us to the end of part one of our special

Sarah: 00:16:00

on evaluating, volunteering for health in Norfolk and Waveney.

Sarah: 00:16:04

A huge thank you to Dr. Jurgen Grots for sharing his insights into the

Sarah: 00:16:08

evaluation process and the importance of understanding the impact of

Sarah: 00:16:12

volunteering in health and care settings.

Sarah: 00:16:14

In part two, we'll hear from a member of the evaluation working

Sarah: 00:16:18

group and someone from the delivery team who has hands-on experience

Sarah: 00:16:22

in evaluating volunteer programs.

Sarah: 00:16:25

We'll explore how evaluation is being shaped collaboratively and what it means

Sarah: 00:16:29

for those involved in delivering and supporting volunteering in practice.

Sarah: 00:16:33

Stay tuned and don't forget to subscribe so you don't miss the next episode.

Sarah: 00:16:37

Thanks for listening.

Sarah: 00:16:39

Please don't forget to subscribe, rate and review this podcast.

Sarah: 00:16:43

It all helps people find us and spread the word about volunteering.

Sarah: 00:16:47

This podcast was produced for the Norfolk Wave in the integrated care

Sarah: 00:16:50

system by Hospital Radio Norwich.

Sarah: 00:16:53

Hosted by Sarah Briggs, producer was Jules Orson.

Sarah: 00:16:57

Original music, composed and performed by Philip Aldridge, interviews

Sarah: 00:17:01

by Sarah Briggs and Jules Odeon.