Analyzing Open Ended-Questions, A Conversation with an Expert June 29, 2017 - Blogs on Text Analytics

Qualitative data analysis,  text mining tools, and techniques are used by people in many disciplines around the world (market research, human resources, political science, sociology, communications, risk assessment, supply chain management  etc) To give you an insight into some of the processes and techniques we are asking different experts, who use our software and other tools, to provide some perspective as to how they approach their analysis.

A short conversation with Dr. Elf Kus Saillard about how to analyze open-ended questions, interviews, and focus groups.

Dr. Saillard has a Ph.D. in sociology and studied research methodology. She works as a coach, trainer, consultant, and researcher in qualitative data analysis. She runs her own Qualitative Research Centre and has offered Computer Aided Qualitative Data Analysis (CAQDAS) workshops to hundreds of academics and researchers in multiple disciplines.

Dr. Saillard when you are starting a project how do you choose your methodology, prepare, and organize your data?

I am a sociologist and an expert in qualitative methodology. I am used to working with the data generated by in-depth interviews and focus groups as well as open-ended surveys. Sometimes I do the whole process including the data generation, sometimes I analyze the data that is already created. In the last few years, I have worked for organizations analyzing employee and/or customer experience data. Today, the digital revolution is influencing the way we think about and produce scientific research. The vast amounts of qualitative data (largely unstructured text) being created every day are something we cannot ignore and it is playing a bigger role in the age of “Big Data” or in the age of ”Too Much Data.” The result is, many organizations are focussing on unstructured data or, at least, including more unstructured components in structured surveys.

Generally, I find myself close to Grounded Theory as a methodological approach. I have used CAQDAS packages since 2003 and I cannot imagine an analysis of qualitative data without it.

 How do you go about the coding of data?

I used to always start with an open-coding approach.  Thus, I was using “in-vivo” coding. But when I began to consult for large organizations, I started working with much bigger volumes of text data. (eg. 20.000 survey responses, 90,000 tweets or very rich focus group data generated from min.  5 groups) I could no longer start with “in-vivo” coding. Instead, I now use text analytics (content analysis) and I ‘gaze at’ emerging patterns with the aid of functions like topic extraction, co-occurrence dendrograms, and maps. Usually, some of the topics that emerge are not a surprise, and some others seem to be less meaningful, especially at the beginning. For example, a topic consisting of words such as, “but”,” well”, “then”, “said”, “him”, “me”,” I,” etc. might look like it makes no sense.  In fact, when you move forward with your qualitative analysis you realize, in this context,  these words tell a lot! For example, in my study about employee experience, the words “but”, “then”, I” were used very frequently and in Wordstat they were listed as a topic at the top of the list of topics. These words were telling me what was wrong with a new rules policy at work and how it went wrong. In each conversation, we construct a social reality and the words we use in our interactions play an important role in the process. Thus, even though it was not my goal to make a conversation (or discourse) analysis, with the help of the software I had it in front of me. Thanks to the software I was able to combine qualitative and quantitative aspects of the text. It feels great to work with different layers and to see how they are related. Words we use constitute the surface and the meaning is what is beneath the surface. As a qualitative researcher, I always give priority to the meaning. Initially, I wasn’t taking into account the words at the surface such as, “but”, “then,” I.” I am continually analyzing employee experience data and the surface words tell me a lot, thanks to my experience with QDAMiner and WordStat. If you study employee/customer experience you need to answer the question of how not just what. So, if your data has a vast amount of “for example” or “then” in the word list, it is probably a good indication you have a rich dataset; people talked about the details.

The next step for me is to spend some time on the topics. I use the “keyword retrieval” and the “keyword in context” functions in Wordstat to look at the text to see what was said and in what context. If I decide to go further with a detailed qualitative analysis, I transform the topics into codes in QDAMiner.

The next step in my analytic journey is to go deep into the selected piece of data and to discover the meaning. In this process, I follow the open-coding approach. To perform open-coding with QDAMiner I generate a superior code ex. “semantics” and I create new codes under this code. When I complete open coding, I re-organize codes according to the emerging themes.

In this process, the memo functions help me a lot. I append quotes to the report manager.  I start reporting during my coding process. I don’t wait to start reporting until I complete my coding. Memo functions and report manager tools are helpful for this.

What role does visualization play in your reporting on the data? How does it help you tell the story?

Data visualization is a must for me. It is very important for two reasons: to go further into the analytic steps and to communicate findings. Crosstabs, heatmaps, maps, graphs, whatever is offered by the software, I use all of them in one way or another. Visualizations help me discover relations and create a hypothesis or test a hypothesis.  Creating visual representations of the data not only helps my analysis but it allows me to better communicate my reports. It allows the reader to “see” the relationships in the data and more easily understand the story the data is telling.

Telling the story, in other words creating a report that tells the story is very important. Whatever type of analysis we do (quantitative or qualitative), as an analyst, it is our job to create the story that reflects the analytical steps but also that shows how we “make sense of data”.

Dr. Saillard is available as a consultant and as a trainer of CAQDAS software including QDA Miner and WordStat. You can find her contact information on the trainers page on the Provalis Research website https://provalisresearch.com/resources/training/trainers-2/