The potential implications of Generative-AI (Gen-AI) for the practice of qualitative data analysis (QDA) heightens the need for researchers to critically reflect on what it means to interpret.
It's more important now than ever before to remember that it is the “interpretive intelligence” that humans bring to analysis that makes it what it is, that makes it valuable.
I caution against jumping on the AI-bandwagon without very carefully considering what appropriate use of these tools really looks like for each individual QDA project. Because inappropriate uses pose great risks for our profession, the pursuit of social science knowledge and meaningful impacts from our work.
What is intelligence in the context of qualitative analysis?
In my post of 17th July 2023, I recounted the Head Teacher at my daughter's school telling parents the school was committed to fostering 'emotional intelligence' in our children. This got me thinking about what 'intelligence' actually is, what types of intelligence there are, and how this relates to 'artificial intelligence' and qualitative analysis, particularly Gen-AI tools based on the capabilities of Large Language Models (LLMs).
According to the online Oxford English Dictionary (OED) intelligence is "the faculty of understanding; intellect. Also as a count noun: a mental manifestation of this faculty, a capacity to understand". This resonates with Max Weber's concept of "verstehen" that undergirds social science approaches to qualitative research.
But a literal translation of "verstehen = understanding" is not enough to capture what Weber meant or what we do as qualitative analysts. As explained in the Sage Dictionary of Qualitative Inquiry by Thomas A. Schwandt, in his effort to establish an "interpretive sociology"
Weber distinguished two kinds of Verstehen: “direct observational understanding,” in which the purpose or meaning of human action is immediately apparent, and “explanatory understanding,” which required grasping the motivation for human behaviour by placing the action in some intelligible, inclusive context of meaning. Weber argued that human action is both open to and requires interpretation in terms of the subjective meaning that actors attach to that action. Social scientific (causal) explanation of human action had to be predicated on this kind of understanding. https://doi.org/10.4135/97814129862681.n369
So, as qualitative analysts doing interpretive QDA we need Weber’s second kind of Verstehen, so that we can comprehend meanings, intentions, and motivations. This requires us to develop insights into the experiences and thought processes of the individuals and groups we study. It is this act of clarifying, explicating, or explaining the meaning of social action that is what interpretation refers to in the social sciences[i].
But the focus on 'emotional intelligence' from my daughter's Head Teacher made me consider how intelligence isn't perhaps only about "understanding" or "interpretation" in these ways, which led me to the work of Howard Gardner.
Multiple Intelligences
Although to the best of my knowledge, Howard Gardner's theory of Multiple Intelligences (MI)[ii] has not been specifically discussed in relation to QDA, it has helped me add a layer to my thinking about the role of AI in QDA.
Gardner defines intelligence as the "biopsychological potential to process information in certain ways, in order to solve problems or fashion products that are valued in a culture or community", and he proposes that intelligence is not a single entity but a collection of various types, each functioning like a separate 'computer' in the brain[iii].
In his 1983 book 'Frames of Mind' he originally posited 8 types of intelligence: linguistic, logical-mathematical, spatial, musical, bodily-kinaesthetic, interpersonal, intrapersonal and naturalistic, but subsequently suggested two more: existential and pedagogical intelligence.
I really enjoyed watching Gardner talk about these in a recording of a lecture he gave that you can watch on his website.
Here's the definitions as he speaks about them in that lecture:
Linguistic intelligence is the intelligence of an orator, a writer, people who are good with language, for example lawyers and writers.
Logical, mathematical intelligence is that of a scientist, a mathematician, somebody who deals with logic, somebody who deals with numbers.
Musical intelligence is the intelligence of a composer, of a performer.
Spatial intelligence is the intelligence that you need to find your way in wide space, the way a navigator or airplane pilot would, or in more constrained kind of space, like a chess player.
Bodily kinaesthetic intelligence is the intelligence of the dancer or the athlete or the sculptor or the surgeon, people who use their whole body as a dancer or athlete would, or parts of their body, as a crafts person or a surgeon would, to solve problems or to make things.
Interpersonal intelligence, is very important intelligence for understanding other people, being able to work with them, people in politics, people in the media, people who are trying to sell you a used car. They're all using their interpersonal intelligence.
Intrapersonal intelligence. Intra means turning inside. It's having an understanding of yourself, who you are, what you can do, what you want to do, what your skills are, what the obstacles are, what your motivational state is like. This kind of intelligence is very, very important nowadays, because so many of us have to make our own decisions about where to work, with whom to work, what to study, what not to spend time on. And if you don't have a good knowledge of yourself, then you're going to make a lot of unnecessary mistakes.
Naturalist intelligence, as the name implies is the intelligence that allows us to make sense of nature, to tell one plant from another, one animal from another, one cloud configuration, one geological situation from another.
Existential intelligence is the intelligence of big questions. Who are we? Where are we headed? What is love? Why do we die? Why do we fight? You might have a pet rat that has more spatial intelligence than you. You might have a bird that has a lot of musical intelligence. But I'm quite convinced only human beings have existential intelligence. Only we ask and ponder these big questions.
Pedagogical intelligence is the intelligence which allows us to teach things to other people.
Interpretive intelligence is what humans do in QDA
I'd contend any QDA involves us using at least linguistic, intrapersonal and interpersonal intelligence and for some types of QDA other types of intelligence would also come into play. For example forms of creative data analysis would draw on musical, spatial and bodily intelligence[iv] and analysis that sits towards the positivist/empiricist end of the methodological spectrum also draws on logical, mathematical intelligence.
The point is that when humans do QDA we bring to the table a combination of 'intelligences' to comprehend the meanings, intentions, and motivations of the people and phenomena we study. This combination of contextualised understandings is what I am calling 'interpretive intelligence'. It's not one intelligence, but the combination of many, and it is this combination that allows us to unpack meanings to form a coherent understanding.
Despite our different purposes, methodologies and methods, we do research to understand the world, and to that end we need interpretive intelligence. As humans we bring several perspectives to bear when we do QDA. When we decide what the data means.
So the question becomes whether Gen-AI can interpret.
Is artificial intelligence capable of interpretation?
There's a lot of useful and appropriate purposes to which Gen-AI tools can be put at different stages in the QDA process, and I'm very excited about future possibilities. But can they interpret?
When a Gen-AI tool summarises qualitative data or answers a question a researcher poses about the material, it is summarising the form of the text and extracting aspects of the content in response to a prompt that it provides in a form that is easily readable. It is not interpreting anything. It cannot do that. This remains the job of humans, and so it should.
The ease with which humans can instruct Gen-AI tools to do these tasks, the intuitive nature of the interfaces and the plausibility of the result can lead us to be convinced it is doing something more – that it is ‘revealing hidden insights’ – that it is interpreting. But it isn’t.
That doesn’t mean it doesn’t have a place, just remember what it is and isn’t doing.
Providing context to a model doesn't give it interpretive intelligence
We as human researchers are able to take into account all kinds of contextual information, background knowledge, and prior experience in interpreting qualitative data that a LLM cannot. In other words, we bring our interpretive intelligence to the task.
CAQDAS packages that have integrated Gen-AI technologies - and many of the new Aps that have exploded onto the market recently - allow us (to varying degrees and in different forms depending on the tool being used) to provide written context in the form of project descriptions and so on, that the model uses to inform its summaries and responses. This certainly helps, and the results produced when such context is provided are more useful for most purposes than when no such context is provided. But however much context we provide, the tool still doesn't have the interpretive intelligence that humans do. It cannot read between the lines, it doesn't understand the meaning of the text with respect to our analytic questions in the way that we can.
This is what we need to remember when considering the appropriateness of Gen-AI tools for our QDAs, and when using the tools at different stages of the analytic workflow.
Humans have interpretive intelligence, machines cannot replicate that. Whether they will be able to in the future is another question.
[i] see Rabinow, P., & Sullivan, W. M., eds. Interpretive Social Science: A Second Look. Berkeley: University of California Press, 1987.
[ii] Gardner, H. (1983). Frames Of Mind. United Kingdom: Basic Books.
[iv] For example see the chapters in The Handbook of Creative Data Analysis https://policy.bristoluniversitypress.co.uk/the-handbook-of-creative-data-analysis