Enhancing LLM Reasoning Capabilities Through Brokered Multi-Expert Reflection

Tejasvee Sheokand

doi:10.1109/ACCESS.2025.3561693

Enhancing LLM Reasoning Capabilities Through Brokered Multi-Expert Reflection

2025-01-01

0

OA

AI

DOI：10.1109/ACCESS.2025.3561693

原文链接

原文求助

摘要

En 中文

Large Language Models (LLMs) have found increasing application in tasks requiring multi-step reasoning, yet challenges such as hallucinations and inconsistencies in the generated responses persist. This study presents an innovative methodology to enhance the reasoning capabilities of LLMs by brokering and integrating multiple expert LLMs within a reflection layer to provide targeted feedback on the reasoning trajectories of the base LLM. The approach employs a foundational pre-trained LLM as the base model, which is further supported by agents to promote cognitive assistance for specific task types. In instances where conclusions are deemed incorrect or reasoning is interrupted, these instances are forwarded to the expert LLM layer, which includes systems such as Claude-3 haiku for intricate contexts and MedAlpaca for medical reasoning, to deliver feedback on the base model's reasoning paths. This feedback forms a 'reflection pool,' enabling the base LLM to amend and enhance its reasoning trajectories in subsequent iterations. The experiments conducted across diverse datasets, including HotPotQA, SimpleQA, and PubmedQA, underscore the proposed architecture's efficacy in augmenting success signals, Rouge-L scores (indicative of quality and precision), and CTRLEval Consistency Scores (indicative of coherence and consistency). The architecture effectively addresses the issues of hallucinations and inconsistencies that frequently occur in multi-step reasoning. Importantly, the approach exhibits considerable potential in tackling domain-specific tasks, underscoring the importance of achieving correct and reliable conclusions. To facilitate further investigation and validation of our proposed brokered multi-expert reflection framework for non-commercial use, the source code of our system is available at https://github.com/WiZY936/Brokered-Multi-Expert-Reflection

Keyword:

Cognition

Reflection

Collaboration

Training

Trajectory

Scalability

Real-time systems

Problem-solving

Iterative methods

Computer architecture

LLM

self-reflection

chain-of-thought

multi-hop inference

feedback-driven refinement

Enhancing LLM Reasoning Capabilities Through Brokered Multi-Expert Reflection

摘要

期刊

机构

引用论文