Find themes in your open-ended survey responses
11 July 2017 - BY Kath McNiff
Free text responses can go a long way toward humanizing your structured survey data.
Respondents can use their own words to express opinions and elaborate on ideas - adding a new dimension to your data and delivering rich insight that you might otherwise miss.
The anonymity of a survey also encourages participants to be more open than they might be in a face-to-face interview. This can lend rigour and nuance to your findings.
But open-ended responses can add a layer of analytical complexity too.
Unlike scaled or closed responses, they’re not easily reduced to numbers and frequencies. Instead, you’ll need to think more qualitatively - reviewing the responses looking for themes, patterns and connections.
If you’re using NVivo, here’s an approach you could take.
1. Choose the survey you want to analyze
You can start by bringing your survey results into NVivo so that you have your open-ended responses and demographic details all together in one place.
If you used Survey Monkey or Qualtrics to create your survey, you can connect to your online account and bring the results into NVivo.
If your results are in an Excel spreadsheet, you can locate the file and import it.
Options available on the Data tab under Survey
Once you choose the survey you want to import, the Survey Import Wizard guides you through a few simple choices:
The Import Survey Wizard
Behind the scenes, the wizard organizes your open-ended content by question and respondent.
This makes it easier to explore all the answers to a question or all the responses by a participant.
What’s more, the demographic attributes from the survey are recorded against each participant – so you can compare the responses of different groups of people.
3. Explore the survey results in NVivo
Once imported, your survey dataset looks something like this:
Straight away, you can scroll through the responses and use features for filtering and sorting columns.
So, for example, you could display only the comments made by women or by respondents who think the pace of development is too fast.
In this view, you could read through all the responses for a given question to get a sense of what people are saying. You could even begin coding the text to gather material by theme.
But there is an easier way to approach this.
On import, the wizard created a node (or container) for each open-ended question.
Having the content in a node offers a more stream-lined way to explore the responses and it gives you immediate access to useful queries.
4. Review the responses to each open-ended question
The nodes created by the wizard are stored in the Nodes area of the workspace:
Nodes for open-ended responses displayed in list view
Each node is like a container that you can open to explore all the responses to a question:
Responses to ‘The natural environment Down East is’
Looking at all the responses in one place makes it easier to get a handle on the data and may spark new ways of thinking.
5. Code the themes
As you read through the responses, you can code the common themes or interesting ideas:
Coding stripes show how the text has been coded
6. Review your node structure
After coding at new nodes, your node structure may start to look something like this:
New nodes under auto coded response nodes
In the above example, you can see that a picture of the natural environment is starting to emerge.
Quite a few respondents describe the environment as ‘precious but fragile’ and ‘development’ is coming through as a common concern.
You could create a word cloud to check for frequently used words.
This can be a useful way to validate your own findings and identify themes you may have missed:
Words frequently used by respondents in relation to the natural environment
For example, this word cloud shows the most common terms people are using in response to the question about the natural environment.
The word ‘destroyed’ is coming out quite strongly – you can double-click to explore it in context.
7. Use queries and visualizations to make comparisons
During import, the wizard also creates a case for each survey respondent.
You can open a case node to review all of a participant’s free text responses.
The wizard assigns attributes to the case based on details from the survey:
Cases displayed in list view along with attributes
These cases and attributes come in handy when you want to make comparisons between different types of people.
For example, you could run a Matrix Coding Query to compare responses based on where a respondent is from:
Matrix Coding query results
Or, you could create a chart to check the spread of responses to a closed-ended question.
For example, this chart shows that around 50 respondents (cases) find the pace of development too fast:
A chart showing the spread responses to a scaled question
Develop your understanding and broaden your perspective
Once you’ve coded the themes and reviewed the data from different angles using queries, you’ll have a clearer sense of how people are feeling about the issues at hand.
The results may confirm what your structured data already tells you or may offer surprises that deepen your understanding and open-up avenues for further exploration.
Remember to write memos along the way so that your insights are carefully stored with the data that inspired them.
This post focuses on working with surveys in NVivo 11 Pro for Windows.
NVivo 11 Plus offers extra features for helping you uncover themes and sentiment – it’s especially useful when you’re dealing with large datasets. Refer to the QSR website for details on how to upgrade or watch this video to learn more: