.png){kind=icon}
返回
Toward purpose-oriented topic model evaluation enabled by large language models
.png){kind=icon}
OA
AI
DOI：10.1007/s00799-025-00429-5.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
This study presents a framework for automated evaluation of dynamically evolving topic models using Large Language Models (LLMs). Topic modeling is essential for organizing and retrieving scholarly content in digital library systems, helping users navigate complex and evolving knowledge domains. However, widely used automated metrics, such as coherence and diversity, often capture only narrow statistical patterns and fail to explain semantic failures in practice. We introduce a purpose-oriented evaluation framework that employs nine LLM-based metrics spanning four key dimensions of topic quality: lexical validity, intra-topic semantic soundness, inter-topic structural soundness, and document-topic alignment soundness. The framework is validated through adversarial and sampling-based protocols, and is applied across datasets spanning news articles, scholarly publications, and social media posts, as well as multiple topic modeling methods and open-source LLMs. Our analysis shows that LLM-based metrics provide interpretable, robust, and task-relevant assessments, uncovering critical weaknesses in topic models such as redundancy and semantic drift, which are often missed by traditional metrics. These results support the development of scalable, fine-grained evaluation tools for maintaining topic relevance in dynamic datasets.
Keyword:
Topic Modeling
Evaluation Framework
Large Language Models (LLMs)
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
I
IF:
1.7
论文数:
17
被引数:
0
机构
引用论文
Tracing the Trends in Sustainability and Social Media Research Using Topic Modeling使用主题建模追踪可持续性和社交媒体研究的趋势.png)
SUSTAINABILITYOAAI
SUSTAINABILITYIF3.3
OAAI分享收藏Tired of Topic Models? Clusters of Pretrained Word Embeddings Make for Fast and Good Topics too!
OAAI
OAAI分享收藏Summarizing large text collection using topic modeling and clustering based on MapReduce frameworkOAAI
OAAI分享收藏分享收藏Partially labeled topic models for interpretable text miningProceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining
Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data miningIF0
PREAI
分享收藏
查看更多内容