RefactorCoderQA: Benchmarking LLMs for Multi-Domain Coding Question Solutions in Cloud and Edge Deployment

Shadikur Rahman

doi:10.1109/tsc.2026.3717248

RefactorCoderQA: Benchmarking LLMs for Multi-Domain Coding Question Solutions in Cloud and Edge Deployment

2026-07-27

0

PRE

AI

DOI：10.1109/tsc.2026.3717248

Original

Original request for help

Save

Abstract

En 中文

To optimize the reasoning and problem-solving capabilities of Large Language Models (LLMs), we propose a structured multi-agent prompting framework inspired by a cloud-edge collaborative architecture. The framework consists of three specialized agents: GuideLLM, which provides methodological guidance; SolverLLM, which generates code solutions; and JudgeLLM, which automatically evaluates solution correctness and quality. To evaluate and demonstrate the effectiveness of this architecture in realistic settings, we introduce RefactorCoderQA, a comprehensive benchmark designed to evaluate and enhance the performance of LLMs across multi-domain coding tasks. Motivated by the limitations of existing benchmarks, RefactorCoderQA systematically covers various technical domains, including Software Engineering, Data Science, Machine Learning, and Natural Language Processing, using authentic coding challenges from Stack Overflow. We propose RefactorCoder-MoE, a fine-tuned mixture-of-experts (MoE) code language model based on DeepSeek-Coder-7B-Instruct, adapted to the RefactorCoderQA benchmark using QLoRA for domain-specific coding question answering. Extensive experiments reveal that RefactorCoder-MoE achieves strong and competitive performance, significantly outperforming all evaluated open-source and commercial baselines, with an overall accuracy of 76.84%.

Keywords:

Large Language Models

Code Generation

Multi-Agent Framework

Benchmarking

Automated Evaluation

RefactorCoderQA: Benchmarking LLMs for Multi-Domain Coding Question Solutions in Cloud and Edge Deployment

Abstract

Journal

Organization

Cited Papers