Analyze SurveyMonkey results using NVivo
In my last post, I covered how to import survey results from SurveyMonkey into NVivo. (Update: NVivo 10 Service Pack 3 now lets you import your SurveyMonkey data directly into NVivo—read Jason's new post Connect to SurveyMonkey and bring your data into NVivo.)
But what can you do with your survey results once you’ve imported them? In this post, I’ll take you through a few suggestions for analyzing this data using NVivo 10.
There are many different approaches you can take to analysis in NVivo, and which approach is right for you will depend very much on your research methodology.
Using the example of a simple course evaluation survey, I’ll be aiming to identify the prevalence of particular themes in responses to free text survey questions, and compare the prevalence of those themes across different groups of respondents (e.g. different age groups or course subjects).
1. Descriptive coding
Descriptive coding of survey data is the process of reading through the responses and coding (tagging) pieces of text with nodes (tags) representing different themes.
NVivo provides a number of tools to make this process as efficient as possible.
Some tips when coding survey data:
- Hide any dataset fields (e.g. classifying fields) that you don’t need to see while coding. This reduces clutter and also, importantly, reduces any unintentional bias that you might bring to your coding if you could see this data. (To do this, open the Properties for the Dataset representing your survey results, using the Properties button on the Home tab of the ribbon, and go to the Dataset tab of the Properties dialog to select which fields should be visible.)
- Use the Form view of the dataset to view all responses from one respondent at at a time. (Open the Dataset source for your survey results, and click the Form tab on the right hand side.) You can use the navigation buttons at the bottom to step through each respondent in turn.
- I like to create new nodes as I go using the List View of the Nodes folder, and to code content at those nodes using the Quick Coding Toolbar at the bottom of the screen. I use a node hierarchy with a parent node for each question on the survey (e.g. Likes, Dislikes, Suggestions), and a child node for each theme in response to a question.
- You can use the Coding Stripes pane to display your coding as you go. (From the View tab of the ribbon, under Coding Stripes select Nodes Recently Coding.) This helps to remind you where you’re up to with your coding, and what you’ve just done, as well as a convenient way to correct your coding (you can click the right mouse button on a coding stripe and select Uncode to change your mind). Hovering the mouse button over the Coding Density bar will display a full list of all nodes coding a response.
2. Auto coding
In addition to descriptive coding, you can automatically code the responses in your survey to create nodes containing all responses
- to a particular free text question, or
- by a particular respondent, or
- by respondents in a particular group (e.g. age group or course subject).
You can then use these nodes to filter and ask questions of your data.
To allow me to tabulate the prevalence of themes (see Step 4 below), at a minimum I usually create a node for each respondent (select the Dataset source, click Auto Code on the Analyze tab of the ribbon, choose “Code at nodes for each value in a column”, and select the RespondentID column).
For more information on auto coding datasets, see the NVivo help topic “Automatic coding in dataset sources”.
3. Classify respondent nodes
At step 3, I created a node for each respondent in the survey. In order to tabulate the prevalence of themes across different groups of respondents, it’s necessary to assign a classification and attribute values to those nodes, representing the data from the classifying columns of the survey dataset (e.g. age, subject).
To do this:
- First, create a Node Classification called e.g. “Respondent” (from the Create tab of the ribbon, click Node Classification).
- Next, open the Properties of the Dataset and set all fields to Visible.
- With the Dataset source selected in the List View, click Classify Nodes from Dataset on the Create tab of the ribbon to open the Classify Nodes from Dataset Wizard. Press Next.
- At step 2 of the wizard, at “Select the location…”, select the option Under Existing Node. Press the Select… button and choose the parent node you created when Auto Coding the respondent nodes above. At “Select a classification…”, select the Respondent classification and check the “Update the classification of existing nodes” checkbox. Press Next.
- At step 3, select the RespondentID column. Press Next.
- At step 4, move all the fields you want to use as attributes (e.g. age, subject) from the Available Columns box to the Selected Columns box, and check the “Update the attribute values of existing nodes” checkbox. Press Finish.
Your respondent nodes will now belong to the classification “Respondent” and will have attribute values copied from the classifying columns of the survey dataset.
4. Tabulate results
NVivo’s Matrix Coding Queries can be used to tabulate the prevalence of particular themes, and to compare the prevalence of those themes across different groups of respondents.
For example, if in my descriptive coding I’ve coded responses at a variety of theme nodes that are children of the parent node 'Likes', and I want to see how the prevalence of these themes varies by course subject, I could construct a Matrix Coding Query as follows:
- From the Query tab of the ribbon, select Matrix Coding to open the Matrix Coding Query dialog.
- In the Rows tab, press the Select button to open the Select Project Items dialog. On the left, select Nodes. On the right, expand the Likes parent node, check all the boxes for its child nodes, and press OK. Back in the Query dialog, press Add To List.
- In the Columns tab, press the Select button to open the Select Project Items dialog. On the left, select Node Classifications. On the right, expand the Respondent classification and the Subject attribute, check all the boxes for different Subject values, and press OK. Back in the Query dialog, press Add To List.
- Press Run to run the query.
A matrix of numbers is displayed, representing the number of coded references for each theme for each group of respondents.
Importantly, the underlying qualitative data behind these numbers is readily accessible from this matrix. Double-click the mouse on a cell of the matrix, or click Open on the Home tab of the ribbon, to open a view of all the references coded at that cell.
When working with matrices you can also:
- Choose to display the number of respondents for each theme for each group of respondents – click on Node Matrix on the View tab of the ribbon, and under Nodes Coded, choose the Respondent classification.
- Display colored shading in the matrix – click on Node Matrix on the View tab of the ribbon, and choose a Cell Shading option.
- View a chart of this data – click the Chart tab on the right hand side of the detail view
- Export the matrix data to Excel or another tool for further analysis – click on Export Items in the External Data tab of the ribbon.
5. Further analysis
The suggestions above are just a few of the possibilities for analyzing SurveyMonkey results in NVivo.
NVivo 10 offers a wide range of analysis features, including sophisticated queries for finding particular terms in your text (including synonyms and related terms), finding frequently occurring terms or finding content based on coding. It also offers a suite of visualisations including charts, word trees, tag clouds and cluster analysis to help you explore and make sense of your data.
Take at look at the movie Work with Survey Results for a step-by-step demo of how to import and explore survey data.
Do you have any tips to share on analyzing SurveyMonkey results in NVivo? Do add your comments below.