Using generative AI for reading question creation based on PIRLS 2011 framework

Zhipeng Wen

doi:10.1080/2331186X.2025.2458653

2025-12-31

PRE

Zhipeng Wen *

Samuel Kai Wah Chu

DOI：10.1080/2331186X.2025.2458653

原文链接

原文求助

摘要

En 中文

This study examined the use of ChatGPT 3.5 for reading questions generation based on four processes of comprehension of PIRLS 2011 assessment framework. Using an instrumental case study approach and a usability testing method, we employed an input story text to assess ChatGPT 3.5's effectiveness. A total of twenty questions were generated and evaluated using specific criteria. We analyzed the content of the obtained questions and the quality of revised questions after further adjustment instructions. Findings reveal that ChatGPT 3.5 excels at generating factual questions, leveraging explicit details from text, and demonstrates improvement in interpret-and-integrate questions when detailed instructions are provided. However, it shows significant limitations with higher-order questions requiring inference and evaluation, where contextual accuracy and deeper comprehension are critical. Statistical analysis revealed significant differences between question types (p = 0.004). Thematic analysis highlighted recurring challenges, such as content misalignment, oversimplification, and difficulty processing complex or nuanced user requirements. Tips for users on potential issues and ethical concerns were also discussed. Despite limitations, ChatGPT 3.5 remains a valuable tool for educators and students to enhance question creation productivity. These findings contribute to understanding generative AI's role in education and provide actionable insights for improving question generation efficiency.

Keyword:

ChatGPT 3.5

reading questions generation

tips

generative AI tools