.png){kind=icon}
返回
A generative AI-driven interactive listening assessment task
.png){kind=icon}
OA
AI
DOI：10.3389/frai.2024.1474019.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Introduction Assessments of interactional competence have traditionally been limited in large-scale language assessments. The listening portion suffers from construct underrepresentation, whereas the speaking portion suffers from limited task formats such as in-person interviews or role plays. Human-delivered tasks are challenging to administer at large scales, while automated assessments are typically very narrow in their assessment of the construct because they have carried over the limitations of traditional paper-based tasks to digital formats. However, computer-based assessments do allow for more interactive, automatically administered tasks, but come with increased complexity in task creation. Large language models present new opportunities for enhanced automated item generation (AIG) processes that can create complex content types and tasks at scale that support richer assessments.Methods This paper describes the use of such methods to generate content at scale for an interactive listening measure of interactional competence for the Duolingo English Test (DET), a large-scale, high-stakes test of English proficiency. The Interactive Listening task assesses test takers' ability to participate in a full conversation, resulting in a more authentic assessment of interactive listening ability than prior automated assessments by positing comprehension and interaction as purposes of listening.Results and discussion The results of a pilot of 713 tasks with hundreds of responses per task, along with the results of human review, demonstrate the feasibility of a human-in-the-loop, generative AI-driven approach for automatic creation of complex educational assessments at scale.
Keyword:
automatic item generation
listening assessment
interactional competence
generative AI
psychometrics
interactive listening
Duolingo English test
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
F
IF:
4.7
论文数:
2.5K
被引数:
4.4K
机构
暂无机构信息
引用论文
Selective reporting - a half signalling load algorithm for distributed sensing选择性报告——一种分布式感知的半信令负载算法
OAAI
OAAI分享收藏Predictive Channel Selection for over-the-Air Video Transmission Using Software-Defined Radio Platforms基于软件定义无线电平台的空中视频传输预测信道选择
PREAI
分享收藏CONSTRUCT-VALIDITY - CONSTRUCT REPRESENTATION VERSUS NOMOTHETIC SPAN.png)
PSYCHOLOGICAL BULLETIN
PSYCHOLOGICAL BULLETINIF19.8
PREAI
分享收藏
查看更多内容