.png){kind=icon}
.png){kind=icon}
Return
Human Cognitive Pattern Simulation for Crowdsourced Test Report Consistency Detection
.png){kind=icon}
PRE
AI
DOI：10.1109/TSE.2026.3659293.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Crowdsourced testing has emerged as a prominent paradigm in software testing by leveraging the diversity of crowdworkers. In this paradigm, crowdworkers are required to submit a test report for each identified bug, which typically contains a textual description and a bug screenshot. However, due to varying worker expertise, many reports exhibit inconsistencies between the textual description and the bug screenshot, which hinder the report review process. Existing methods address this issue by automatically detecting report consistency, typically through matching the UI widgets referenced in the textual description with those visible in the bug screenshot. However, such methods focus only on surface-level element correspondence and fail to capture the abstract bug semantics, such as the functional meaning and bug-triggering context. Consequently, they lack the ability to detect more subtle but realistic inconsistencies. To bridge this gap, we propose InconHunter, a novel method for crowdsourced test report consistency detection that explicitly simulates human cognitive pattern. In this pattern, humans typically adopt two complementary reasoning strategies. If the textual description allows them to form an expectation about the visual bug features, they assess consistency by verifying if the expected features appear in the bug screenshot. Otherwise, they shift to reasoning about if the bug-triggering context described in the report aligns with the app state shown in the bug screenshot. InconHunter instantiates this cognitive pattern through two LLM-powered modules, each dedicated to one reasoning strategy. We evaluate InconHunter through experiments on our dataset with 2,310 labeled crowdsourced test reports, and results show that InconHunter outperforms baselines by 14.00%-19.28%, demonstrating superior effectiveness, monetary-based cost efficiency, and alignment with human cognitive pattern.
Keywords:
Computer bugs
Testing
Cognition
Visualization
Semantics
Costs
Coherence
Reviews
Electronic mail
Software testing
Crowdsourced testing
large language model
cognitive pattern
report consistency detection
Journal
IF:
5.6
Papers:
2.8K
Citations:
1.1W