AI for Qualitative Research: A Hands-On Guide for Management Scholars
Diana Garcia Quevedo
Questions & Answers from this book
Questions and answers are connected to the referenced book and its available source material.
Chapter 2: Overview of Artificial Intelligence, Machine Learning, Natural Language Processing, and Large Language Models
Chapter 3: Natural Language Processing in Management Research
Why have qualitative researchers historically been hesitant to adopt computational methods like NLP, and how have LLMs changed this?
Historically, qualitative researchers hesitated to adopt NLP because computational methods were seen as incompatible with interpretive analysis, and early NLP models lacked the ability to understand context and nuance. These limitations meant NLP was useful mainly for preprocessing large datasets, not for the close, context-sensitive reading qualitative scholars value. LLMs changed this because they can capture semantic meaning, subtle nuance, and context from vast unstructured texts, making them more compatible with inductive and interpretive qualitative approaches and shifting researcher acceptance.
According to the chapter, what are the main limitations of using ChatGPT for qualitative data analysis as identified in the studies by Hamilton et al. (2023) and Morgan (2023)?
According to the chapter, Hamilton et al. (2023) found that ChatGPT-generated themes are too specific to participants' immediate circumstances, disregarding nuanced context and subtleties, whereas human coders used contextual knowledge to develop more comprehensive themes. Morgan (2023) found that ChatGPT tends to emphasize generic aspects of the data, lacking contextualization and the ability to recognize subtle, interpretative themes. Both studies conclude that ChatGPT should serve as a complementary tool rather than a substitute for researcher interpretation.
According to the chapter, what are the main capabilities of large language models (LLMs) that make them useful for inductive qualitative analysis?
According to the chapter, LLMs are useful for inductive qualitative analysis because they can explore and summarize large datasets, retrieve and augment information through retrieval-augmented generation (RAG), and classify and cluster data by capturing language nuances, context, and implicit patterns. These capabilities let researchers handle vast amounts of unstructured data and reshape traditional linear analysis into dynamic, iterative inquiry.
Chapter 5: Systems and Tools to Use NLP and LLMs: Getting Started
What are the main reasons for the lack of transparency in proprietary LLMs like OpenAI's GPT suite?
Proprietary LLMs like OpenAI's GPT suite lack transparency mainly because their stochastic nature makes outputs difficult to trace deterministically, and because developers withhold training data, methods, and procedures to protect competitive advantage. This lack of disclosure limits researchers' ability to audit or replicate model behavior.
Why does the stochastic nature of LLMs make explainability difficult?
LLMs are not deterministic systems; they generate responses based on a statistical representation of language rather than predictable, reproducible rules. This stochasticity makes it difficult to relate inputs to outputs in a predictable way and to explain exactly why specific content was generated, because there is no clear, deterministic pathway from prompt to response.
Chapter 8: Classification
What are the limitations of using LLMs as classifiers in qualitative research, and how can researchers mitigate these limitations?
The main limitations are sensitivity to prompt wording and example order, the constraint on the number of examples imposed by context size, and output variability caused by the model's stochastic nature. Researchers can mitigate these by running multiple executions, validating against manually labeled ground truth, treating LLM classifications as preselectors of relevant data rather than fixed categories, and combining or testing different LLMs and classification techniques.
How do large language models (LLMs) address the limitations of traditional supervised learning for qualitative analysis?
LLMs address the limitation of relying on labeled training data by acting as general-purpose classifiers that use their statistical representation of language and simple prompts instead of supervised training. They also offer flexibility through in-context learning, allowing researchers to classify qualitative data with zero, one, or a few examples and to adapt the classification logic to specific domains.
Chapter 9: Clustering and Topic Modeling
How does topic modeling with LLMs differ from traditional methods like LDA?
Traditional topic modeling methods like LDA rely on word-level counts and statistics, so they miss the semantic structure of sentences, paragraphs, and entire documents. LLM-based topic modeling captures the nuance of human language, producing more coherent and interpretable topics, and can identify complex or specialized language use, though it also brings limitations such as topic instability and hallucinations.
What are the main limitations of using LLMs for topic modeling in qualitative research, as discussed in this chapter?
The chapter identifies several main limitations: topic instability, where each execution produces different topics and assignments; hallucinations that fabricate plausible-looking but nonexistent topics; failure to retrieve topics actually present in the data; model bias reflected in the output; and a lack of genuine interpretability and contextual understanding. These limitations mean LLM-generated topics should be treated as a starting point and require careful human oversight.
Chapter 10: Information Retrieval and Retrieval-Augmented Generation
Chapter 11: Perspectives on LLMs in Management and Qualitative Research
According to the chapter, how can agentic AI be used in qualitative research?
Agentic AI can be used as an autonomous digital research assistant: a researcher gives it a high-level objective, and the agent breaks it down into subtasks and executes a multistep plan, such as gathering press releases, social media data, and news articles, performing sentiment and thematic analysis, and synthesizing findings into a structured report. This automates laborious parts of the workflow and lets the researcher focus on interpreting findings and crafting the final narrative. The chapter also highlights deep research agents, which synthesize hundreds of sources into a single cited report, useful for literature reviews.
What are the key limitations of large language models (LLMs) mentioned in this chapter?
The chapter identifies two key limitations of current LLMs: they are predominantly limited to processing text, unable to natively handle image, audio, or video data, and they have limited capacity for complex, multistep reasoning. These limitations are expected to be addressed by future multimodal and reasoning models.
You may also be interested in
FastAPI: Modern Python Web Development
Bill Lubanovic;
37 questions
Generative Deep Learning: Teaching Machines to Paint, Write, Compose, and Play
David Foster;
33 questions
AI for Cybersecurity_ Research and Practice
Unknown
27 questions
AI for Time Series_ Volume 1_ Unlocking Patterns with Deep Learning
Min Wu;Emadeldeen Eldele;Zhenghua Chen;Shirui Pan;Qingsong Wen;Xiaoli Li;
25 questions