.png){kind=icon}
.png){kind=icon}
Return
Self-verified user simulator via code-based interpretation in task-oriented dialogues
.png){kind=icon}
PRE
AI
DOI：10.1016/j.engappai.2026.114687.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
User simulators are essential for training and evaluating task-oriented dialogue systems (TODs). Recently, large language models (LLMs) have been increasingly adopted to construct user simulators by prompting them to generate natural language utterances and dialogue actions. However, due to the difficulty of controlling structured outputs through natural language prompts alone, these LLM-based simulators often produce incomplete, inconsistent, or invalid dialogue actions, limiting their effectiveness. To tackle this, this paper proposes a self-verified code-based user simulator that guides LLMs to generate intermediate Python code for structured dialogue actions. These code snippets are executed and validated by an external interpreter, and the verified outputs are used to refine the simulator’s behavior. Experiments on the Multi-Domain Wizard-of-Oz (MultiWOZ) dataset demonstrate that our method improves dialogue action accuracy by 4.0%, and significantly enhances utterance diversity, achieving 12.1% more trigrams, an increase of 0.93 in entropy, and a 13.2% gain in measure of textual lexical diversity (MTLD) over 100 dialogue turns. These results highlight the effectiveness of code-level verification in improving the controllability, correctness, and expressiveness of LLM-based user simulators.
Keywords:
user simulator
task-oriented dialogue
large language models
code-based verification
dialogue action accuracy
Journal
IF:
8
Papers:
5.3K
Citations:
3.5W