.png){kind=icon}
返回
Comparing human text classification performance and explainability with large language and machine learning models using eye-tracking
.png){kind=icon}
OA
AI
DOI：10.1038/s41598-024-65080-7.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
To understand the alignment between reasonings of humans and artificial intelligence (AI) models, this empirical study compared the human text classification performance and explainability with a traditional machine learning (ML) model and large language model (LLM). A domain-specific noisy textual dataset of 204 injury narratives had to be classified into 6 cause-of-injury codes. The narratives varied in terms of complexity and ease of categorization based on the distinctive nature of cause-of-injury code. The user study involved 51 participants whose eye-tracking data was recorded while they performed the text classification task. While the ML model was trained on 120,000 pre-labelled injury narratives, LLM and humans did not receive any specialized training. The explainability of different approaches was compared based on the top words they used for making classification decision. These words were identified using eye-tracking for humans, explainable AI approach LIME for ML model, and prompts for LLM. The classification performance of ML model was observed to be relatively better than zero-shot LLM and non-expert humans, overall, and particularly for narratives with high complexity and difficult categorization. The top-3 predictive words used by ML and LLM for classification agreed with humans to a greater extent as compared to later predictive words.
Keyword:
Human-AI alignment
Large language models
Explainable AI
Eye tracking
Cognitive engineering
Human-computer interaction
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
3.9
论文数:
27.9W
被引数:
83.5W
机构
引用论文
分享收藏Outcomes of re-treatment with first-line trastuzumab plus a taxane in HER2 positive metastatic breast cancer patients after (neo)adjuvant trastuzumab: A prospective multicenter study.png)
Oncotarget
OAAI
OncotargetIF0
OAAI分享收藏Improving autocoding performance of rare categories in injury classification: Is more training data or filtering the solution?提高损伤分类中罕见类别的自动编码性能: 是更多的训练数据还是过滤解决方案？OAAI
OAAI分享收藏Performance implications of wide-band lasers for FSK modulation labeling scheme宽带激光器对FSK调制标签方案的性能影响
PREAI
分享收藏Facile synthesis of tremelliform Co0.85Se nanosheets: An efficient catalyst for the decomposition of hydrazine hydratetremelliform Co0.85Se纳米片的简易合成: 水合肼分解的有效催化剂
PREAI
分享收藏
查看更多内容