.png){kind=icon}
.png){kind=icon}
Return
The Use of Large Language Models for Qualitative Research: The Deep Computational Text Analyser (DECOTA)
.png){kind=icon}
OA
AI
L
R
K
D
N
P
W
DOI：10.1037/met0000753.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Machine-assisted approaches for free-text analysis are rising in popularity, owing to a growing need to rapidly analyze large volumes of qualitative data. In both research and policy settings, these approaches have promise in providing timely insights into public perceptions and enabling policymakers to understand their community's needs. However, current approaches still require expert human interpretation-posing a financial and practical barrier for those outside of academia. For the first time, we propose and validate the Deep Computational Text Analyser (DECOTA)-a novel machine learning methodology that automatically analyzes large free-text data sets and outputs concise themes. Building on structural topic modeling approaches, we used two fine-tuned large language models and sentence transformers to automatically derive codes and their corresponding themes, as in inductive thematic analysis. To fully automate the process, we designed and validated a novel algorithm to choose the optimal number of topics for the structural topic modeling. DECOTA outputs key codes and themes, their prevalence, and how prevalence varies across covariates such as age and gender. Each code is accompanied by three representative quotes. Four data sets previously analyzed using thematic analysis were triangulated with DECOTA's codes and themes. We found that DECOTA is approximately 378 times faster and 1,920 times cheaper than human coding and consistently yields codes in agreement with or complementary to human coding (averaging 91.6% for codes and 90% for themes). The implications for evidence-based policy development, public engagement with policymaking, and psychometric measure development are discussed.
Keywords:
free-text analysis
large language models
machine learning
natural language processing
topic modeling
Journal
IF:
7.8
Papers:
1.3K
Citations:
2.1W
Organization
No organization information available
Cited Papers