Find themes in your open-ended survey responses

11 July 2017 - BY Kath McNiff

Find themes in your open-ended survey responses

Free text responses can go a long way toward humanizing your structured survey data.

Respondents can use their own words to express opinions and elaborate on ideas - adding a new dimension to your data and delivering rich insight that you might otherwise miss.

The anonymity of a survey also encourages participants to be more open than they might be in a face-to-face interview. This can lend rigour and nuance to your findings.

But open-ended responses can add a layer of analytical complexity too.

Unlike scaled or closed responses, they’re not easily reduced to numbers and frequencies. Instead, you’ll need to think more qualitatively - reviewing the responses looking for themes, patterns and connections.

If you’re using NVivo, here’s an approach you could take.

1. Choose the survey you want to analyze

You can start by bringing your survey results into NVivo so that you have your open-ended responses and demographic details all together in one place.

If you used Survey Monkey or Qualtrics to create your survey, you can connect to your online account and bring the results into NVivo.

If your results are in an Excel spreadsheet, you can locate the file and import it.

make-sense-of-open-ended-survey-responses-options-jpg.png

Options available on the Data tab under Survey

Once you choose the survey you want to import, the Survey Import Wizard guides you through a few simple choices: 

make-sense-of-open-ended-survey-responses-wizard-jpg.png
The Import Survey Wizard

Behind the scenes, the wizard organizes your open-ended content by question and respondent.

This makes it easier to explore all the answers to a question or all the responses by a participant.

What’s more, the demographic attributes from the survey are recorded against each participant – so you can compare the responses of different groups of people.

3. Explore the survey results in NVivo

Once imported, your survey dataset looks something like this:

make-sense-of-open-ended-survey-responses-dataset-jpg-(1).png

Straight away, you can scroll through the responses and use features for filtering and sorting columns.

So, for example, you could display only the comments made by women or by respondents who think the pace of development is too fast.

In this view, you could read through all the responses for a given question to get a sense of what people are saying. You could even begin coding the text to gather material by theme.

But there is an easier way to approach this.

On import, the wizard created a node (or container) for each open-ended question.

Having the content in a node offers a more stream-lined way to explore the responses and it gives you immediate access to useful queries.  

4. Review the responses to each open-ended question

The nodes created by the wizard are stored in the Nodes area of the workspace:

make-sense-of-open-ended-survey-responses-nodes-jpg.png
Nodes for open-ended responses displayed in list view

Each node is like a container that you can open to explore all the responses to a question:

make-sense-of-open-ended-survey-responses-review-node-jpg-(1).png
Responses to ‘The natural environment Down East is’

Looking at all the responses in one place makes it easier to get a handle on the data and may spark new ways of thinking.

5. Code the themes

As you read through the responses, you can code the common themes or interesting ideas:

make-sense-of-open-ended-survey-responses-review-code.jpg
Coding stripes show how the text has been coded

6. Review your node structure

After coding at new nodes, your node structure may start to look something like this:

make-sense-of-open-ended-survey-responses-review-node-structure-jpg.png
New nodes under auto coded response nodes

In the above example, you can see that a picture of the natural environment is starting to emerge.

Quite a few respondents describe the environment as ‘precious but fragile’ and ‘development’ is coming through as a common concern.

You could create a word cloud to check for frequently used words.

This can be a useful way to validate your own findings and identify themes you may have missed:

make-sense-of-open-ended-survey-responses-word-cloud-(1).jpg
Words frequently used by respondents in relation to the natural environment

For example, this word cloud shows the most common terms people are using in response to the question about the natural environment.

The word ‘destroyed’ is coming out quite strongly – you can double-click to explore it in context.

7. Use queries and visualizations to make comparisons

During import, the wizard also creates a case for each survey respondent.

You can open a case node to review all of a participant’s free text responses.

The wizard assigns attributes to the case based on details from the survey:

make-sense-of-open-ended-survey-responses-cases.jpg
Cases displayed in list view along with attributes

These cases and attributes come in handy when you want to make comparisons between different types of people.

For example, you could run a Matrix Coding Query to compare responses based on where a respondent is from:

make-sense-of-open-ended-survey-responses-matrix-(1).jpg
Matrix Coding query results

Or, you could create a chart to check the spread of responses to a closed-ended question.

For example, this chart shows that around 50 respondents (cases) find the pace of development too fast:

make-sense-of-open-ended-survey-responses-chart-jpg.png 
A chart showing the spread responses to a scaled question

Develop your understanding and broaden your perspective

Once you’ve coded the themes and reviewed the data from different angles using queries, you’ll have a clearer sense of how people are feeling about the issues at hand.

The results may confirm what your structured data already tells you or may offer surprises that deepen your understanding and open-up avenues for further exploration.

Remember to write memos along the way so that your insights are carefully stored with the data that inspired them.

This post focuses on working with surveys in NVivo 11 Pro for Windows.

NVivo 11 Plus offers extra features for helping you uncover themes and sentiment – it’s especially useful when you’re dealing with large datasets. Refer to the QSR website for details on how to upgrade or watch this video to learn more: