Using NVivo in participatory evaluation
17 June 2015 - IN creative, evaluation, methods, NVivo, participatory, research
Evaluation is a particular type of research that is designed to assess the value of a service, intervention, practice or policy. Evaluation rarely draws on academic literature or requires formal ethical approval, and it usually requires the researcher to write recommendations for future action. Apart from that, it has much in common with other kinds of research, requiring a plan, data collection and analysis, report writing, and presentation and dissemination of findings
A creative approach
As long as you have a basic understanding of research methods, it is possible to do evaluation without thinking very hard. Work out what your population is and how to sample that population; decide whether to use questionnaires, interviews, focus groups or a combination; collect and analyse data; write it up. But evaluation research, like all research, can also be done using more creative methods.
Creative research methods include arts-based and technological methods, mixed methods, and transformative research frameworks such as participatory or decolonising methodologies. Also, many creative researchers will involve participants or other stakeholders. I love to involve non-researchers in research because often they have really good ideas. I did this in a participatory evaluation I worked on for a Sure Start children's centre in England over 10 years ago.
Involving non-researchers in all stages
Children's centres provide a range of services for families with young children, from ante-natal groups and baby massage to childcare and nursery education. They have an inclusive ethos and like to involve family members in all kinds of ways. So this particular children's centre's manager was very interested when I suggested involving family members in the evaluation. We asked around to find out who might be interested, and ended up with a small group of three young mothers who had five pre-school children between them.
I explained to the mothers that they were free to be involved as much or as little as they liked. They originally signed up for involvement in the planning stage. The children's centre helped by finding somewhere for us to meet – though as the actual centre was still in the process of being built, we ended up meeting round a table in the local pub before opening time! The children's centre also provided a creche worker and toys to occupy the children in a side room while the mothers worked on the evaluation with me and another researcher.
We planned a series of interviews with various stakeholders, and wrote and tested the interview questions. Then I asked the mums whether they would like to conduct the interviews. They were nervous, particularly about interviewing 'important people', but they were also keen, and eventually decided they would do the interviews if they could work in pairs. We trained them in interviewing techniques and research ethics, then off they went. And they loved it! They did a great job, too.
Analyzing in NVivo
Then it was time for data analysis. In my experience, this is usually the time when participants say "We've had enough now, you do it and then tell us what you've found." Not these young mums: they were up for the challenge. And of course we were using NVivo.
So I took my laptop to the pub and coached them through the process. I'm not an NVivo trainer, but I found it fairly straightforward to explain how to do emergent coding, and reassure them that they couldn't do it wrong because whatever they saw in the data was valid. When they'd done the coding, I helped them to identify themes and outliers and patterns in the data, and we used annotations throughout to keep track of our thought processes. They soon became confident with the software, and found the analysis an exciting process, enjoying the revelations NVivo can help researchers to see.
Did they want to write the research report too, I asked? Yes indeed they did! So I explained how to use the results from NVivo to jump-start, inform, and shape the writing process. And off they went again, with guidance and support when they needed it from my colleague and myself – including a morning's hands-on childcare, one day, when the creche worker was off sick. I will never forget playing with the children on the pub carpet while the mums debated and typed at the table above us.
Those young mothers did a fine job. They presented the findings and recommendations, too, at a local meeting. The word about our participatory evaluation spread as far as the then Minister for Education who sent an approving message back to the children's centre manager. One of the mums gained enough confidence, from her experience, to go on and do a college course.
Participatory research is resource-intensive. It often takes quite a lot longer than non-participatory research, and requires extra funding, too. The mums were all on benefits, so being paid fees would have caused problems, but the funders paid for their transport to and from meetings, the childcare, mobile phone top-ups so they could keep in touch with the researchers and the centre, and drinks and snacks and lunches for all of us.
And when it was all over, to say 'thank you', the funders gave them each vouchers for an all-expenses-paid spa day at a local hotel, and arranged and paid for childcare and transport for that too.
Creativity in managing relationships
But participatory research is also highly creative. The creativity lies in how you manage relationships within the project. And this kind of commissioned research is able to be more fully participatory than funded academic research, where the research usually has to be designed and ethical approval gained before participants can be involved.
We were able to involve participants from the very earliest stage, before any work had been done on the research design. Then we were able to work flexibly throughout. We were willing to take over from the mums at any point, and we might have needed to if circumstances had been different: say if one mum had become unwell, another had got a job, and the third didn't want to continue without her friends.
So we were lucky, too. I did many evaluations in children's centres, most of them with a participatory element, but this was the only one where participants were willing to take part in the data analysis. But that proves it can be done. Also, it was enormous fun!