Located in the Melbourne metropolitan area, the Council and management team at the City of Kingston decided to create a shared, long term vision for its diverse community. The project was named Living Kingston 2035 and provided the community with an opportunity to share their needs, interests and aspirations. The Project Team, led by Dr. Robyn Cochrane, used NVivo to progressively analyze and securely manage a significant volume of data which included survey responses, online submissions and forum discussions. By using NVivo, the City of Kingston could efficiently and systematically sort the community’s input to make informed evidence-based decisions and develop plans for the medium to long term.
Benefits of using NVivo:
- Minimized researcher subjectivity and bias
- Managed and analyzed large volumes of unstructured data
- Data were stored securely and reported throughout the project
- Research timeframes were reduced significantly
- Improved the research outcomes and findings to inform decisions and plans
- Reports, visualizations and data summaries could be quickly generated
- Decisions and plans for the future could be made with confidence
- NVivo 10 will be used to create a ‘community knowledge bank’ and generate customized reports to inform other operational and planning processes.
Introducing Living Kingston 2035
In March 2011, Council resolved to create a long-term community vision for Kingston. The vision was to become a key legacy for the outgoing Council and provide useful input to the decision-making and planning processes of the incoming Council (following elections in October 2012). After considering Officers’ advice, Council resolved to establish a Steering Group and Reference Panel to oversee and guide the community visioning project as well as a Project Team to implement the project.
Planning the consultation
The Project Team recognized the many challenges posed by effectively engaging with the community, particularly about a topic as far-reaching and abstract as a “community vision”. A review of a community visioning exercises undertaken nationally uncovered a range of consultation approaches that had been usefully applied. The Project Team noted the importance of carefully framing key questions that could be used across multiple participation options and decided upon:
- What do you like about Kingston?
- What needs to be improved in Kingston?
- Looking forward to 2035, what is your wish for Kingston?
Early decisions were made to use several communication channels to inform the community about the project and to offer multiple participation options to enable interested community members to submit their views in the consultation. The consultation was envisaged to comprise traditional paper based surveys, telephone-in options, staff forums, public community forums as well as the use of online, web based and mobile channels. A small number of local government authorities had experimented with web-based technologies and it was quickly agreed that these technologies could provide an additional participation option that might overcome some of traditional consultation challenges. Web-based technologies were regarded as a legitimate and innovative way to meaningfully engage with the community, encourage broader participation and more diverse inputs into the consultation.
Sourcing project funding
The project was initiated and funded by the City of Kingston. However, the Project Team was aware of a community support grants program offered by the Victorian Government for community support initiatives. Contact was made with representatives of the Department of Planning and Community Development to seek advice as to the potential suitability of this project for funding. Following an intensive grant application process and lengthy and competitive external evaluation process, the City of Kingston’s application was successful. A grant of $48,000 was secured to support the consultation and, in particular, the community strengthening and capacity building features of the project.
Turning unstructured data into an informed direction
The Project Team recognized that it was critical to the integrity of the project that the data were efficiently managed and reported to key stakeholders throughout the duration of the project.
NVivo was used at three key milestone stages:
- Pre-testing of initial broad themes/template: following a review of other community visioning exercises, the Project Team identified nine key themes to guide Kingston’s consultation: Arts, Culture and Heritage; Built Environment; Community and Support Services; Connection and Learning Communities; Free Moving and Accessible City; Living Environment; Recreational Spaces; Safe and Clean City and Vibrant Local Economy. In the early stages of the consultation NVivo was used to analyze a sub-set of the survey data to test the relevance and coverage of the nine broad themes. Two new themes became evident (“Council Operations” and “Other/Out of the Box”) and were subsequently incorporated into the coding template and consultation approach.
- Preliminary analysis of the data for “checking in” with the community: In early November we used NVivo to conduct a review and preliminary analysis of the data gathered to 31 October 2012. Thousands of LIKE, IMPROVE and WISH ideas were sorted into 10 broad themes. By using NVivo we could provide key stakeholders and participants with verbatim data and 100 broad WISH statements. The WISH statements attempted to capture the essence of the ideas gathered and were used to check and validate what had been heard. Ideas classified as “Other/Out of the Box” were also captured and reported.
- Analysis and formal report of full consultation: over 13,500 ideas were gathered via multiple participation options for the entire consultation. Our challenge was to manage and make sense of a substantial volume of ‘messy’ data while maintaining the integrity of the data and anonymity of participants. Overall we gathered:
- 1,538 completed surveys
- 399 online submissions via the Living Kingston website
- Information from over 300 forum attendees
- 4,101 votes, 41 comments and 16 new ideas from 115 participants via an online voting platform
Using NVivo our Project Team had the capacity to produce multiple reports encompassing over 1,000 pages of “raw data”. The reports presented the findings for each broad theme, geographical areas (Council ward) and participation option (survey, online submissions and forum discussions). High-level descriptive reports as well as detailed research reports were produced to suit the needs of specific audiences.
The role of NVivo
Dr. Cochrane was already a proficient user of QSR International’s NVivo having used the product prior to working with the City of Kingston. Thus, the Project Team recognized that NVivo was a relatively inexpensive computer-assisted qualitative data-analysis software tool that could efficiently and effectively manage large volumes of consultation data. Consequently, the Project Team elected to use NVivo 10 because it:
- Was compatible with Microsoft excel and word files
- Had the capacity to save evolving versions of the research project.
- Could manage the analysis process more systematically, effectively and efficiently than manual sorting.
- Offered a secure project file hub for cataloguing consultation data gathered via multiple options from thousands of participants.
- Ensure the secure storage of sensitive and personal data.
- Enhanced the trustworthiness and transparency of the research process and findings.
- Enabled the production of robust and defensible reports that could stand up to close scrutiny.
Using NVivo 10 the Project Team were able to:
- Manage the largest community consultation project ever undertaken by the City of Kingston and deliver findings to the management team and Councilors within four weeks.
- Conduct preliminary analyses of the thousands of ideas gathered to 31 October and create 100 broad WISH statements that were then explored via a second series of community forums and an online voting platform created specifically for the project.
- Easily sort and analyze a significant volume of data in a transparent and robust manner.
- Work through the messiness of the open-ended responses and carefully write up of the research.
- Monitor the occurrence and recurrence of ideas and phases in the data
- Generate frequency counts and visual images to highlight consistency in coding and reporting.
- Connect and segment ideas with participants’ demographic attributes (ie. age group and residential suburb).
- Have ongoing access to an interactive community knowledge bank to inform other operational and planning processes thereby maximizing the organizational return on investment in the consultation.
Although it is difficult to estimate, it may have taken the Project Team around three months to manually analyze the consultation data without NVivo. Managing, analyzing and subsequently reporting the data without NVivo would have been an extremely labor-intensive task. It is highly unlikely that the findings would have been formally reported with the same level of confidence, rigor and visual aids.
The use of best practice computer-assisted qualitative data-analysis software minimizes subjectivity within the sorting and presenting of the data and allows this to occur without researcher bias. It was believed a computer-assisted approach would provide better and clearer outcomes from the consultation.
Living Kingston 2035 Program Leader Dr. Robyn Cochrane stated “We could quickly and confidently report to our management team and Councilors that all of the data had been carefully sorted using NVivo 10. There was little room for researcher bias and loss of data – we could confidently say that every idea was coded, sorted and reported.
Sarah Bishop, Manager – Communications and Public Affairs commented “It was crucial for us to quickly turnaround the consultation data so our Councilors could develop their draft 2013-17 Council Plan. By using NVivo our project team could quickly generate visualizations, frequency tables and customized detailed reports. When these reports were considered alongside the participant profiles for each data set and other information about the municipality of Kingston, Councilors could confidently use the consultation data to make decisions and plan for the future."
Word frequency query for the Arts, Culture and Heritage WISH ideas displayed visually as a tag cloud. The size and density of each word reflects the frequency reported in the WISH ideas. For the purposes of this example, the 100 most frequently reported words (comprising four or more letters) were selected.
Dr. Cochrane added “Sharing in-depth and high-level descriptive versions of the consultation findings with key stakeholders and the community is important to the integrity of the project and is best practice in the public participation sphere. Through this project, the City of Kingston has informed, consulted, involved, collaborated and empowered its community (see iap2.org.au)”.
Related products:
- SurveyMonkey
- Uservoice crowdsourcing online voting platform
About City of Kingston:
City of Kingston is responsible for ensuring that its 142,000 residents are well governed and serviced. To find out more: kingston.vic.gov.au. The Victorian Government has created a guide which details the roles of local Councils. To find out more: localgovernment.vic.gov.au
About Dr. Robyn Cochrane:
Dr. Robyn Cochrane was employed by the City of Kingston as Project Leader of Living Kingston 2035. Robyn works as a Research/Teaching Associate with Monash University and is the lead researcher with Cochrane Research Solutions. Robyn holds a PhD in Management and has over 25 years of experience in the local government, vocational education and training, higher education and community sectors. Robyn has published articles in academic and practitioner journals and produced a range of research reports for industry partners. Dr. Cochrane’s interests include: qualitative and quantitative research designs and analysis; online and traditional surveying approaches and community strengthening. Contact details: robyncoc@bigpond.net.au