.png){kind=icon}
.png){kind=icon}
Return
Experimental evaluation of machine learning models for goal-oriented customer service chatbot with pipeline architecture
.png){kind=icon}
PRE
AI
E
N
S
A
DOI：10.1007/s00500-026-11411-x.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Integrating machine learning into customer service chatbots has substantially improved their ability to interpret and respond to user queries. However, existing evaluation practices often treat such systems as black boxes or assess individual modules in isolation, limiting insight into the architectural sources of performance variation. This study proposes a component-complete evaluation framework for goal-oriented customer service chatbots implemented using a modular pipeline architecture. The framework enables systematic, component-wise benchmarking of Natural Language Understanding, Dialogue Management, and Natural Language Generation under controlled and standardized experimental conditions. Using representative neural architectures for each pipeline component, we conduct a structured evaluation on the MultiWOZ dataset. Component performance is assessed using task-appropriate functional metrics and statistical validation to reduce inconsistencies arising from optimization variability. The results reveal clear trade-offs across components, demonstrating differing strengths of contextualized language models, recurrent architectures, and value-based reinforcement learning methods within modular dialogue systems. Beyond individual model comparisons, the primary contribution of this work is a structured evaluation methodology that supports fine-grained architectural diagnosis across the full chatbot pipeline under controlled experimental settings.
Keywords:
Customer service chatbots
Deep reinforcement learning
Natural language understanding
Dialogue management
Natural Language generation
Hyperparameter optimization
Journal
IF:
2.5
Papers:
1.0W
Citations:
2.1W
Organization
Cited Papers