.png){kind=icon}
Return
Assessing Small Language Models for Code Generation: An Empirical Study with Benchmarks
.png){kind=icon}
OA
AI
DOI：10.1016/j.jss.2026.112815.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
• Evaluated 20 open-source Small Language Models (SLMs) on five code generation benchmarks, covering tasks such as code synthesis, summarization, and repair. • Identified that several compact SLMs (-<3B parameters) achieve competitive performance with lower resource requirements, making them suitable for deployment in memory-constrained environments. • Larger SLMs offer higher accuracy but demand up to 4 times more resources (VRAM) for a 10% improvement in pass@1, highlighting clear performance efficiency tradeoffs of SLMs. • No statistically significant performance differences across programming languages, suggesting SLMs, generalizability in multilingual code generation tasks within the programming languages tested in the study.
Keywords:
Small Language Models
Code Generation
Empirical Study
Benchmarks
Software Engineering
Generative AI
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
4.1
Papers:
5.5K
Citations:
8.4K
Organization
No organization information available
Cited Papers
.png){kind=icon}
ShareSaveShareSavetukey_hsd: An Accurate Implementation of the Tukey
Honestly Significant Difference Test in Python
PREAI
ShareSave no more
no more