CPerfSmith: A Randomized C Program Generator for Performance-Oriented Compiler Testing

doi:10.1145/3771775.3786271

Return

CPerfSmith: A Randomized C Program Generator for Performance-Oriented Compiler Testing

2026-01-01

PRE

DOI：10.1145/3771775.3786271

Original

Original request for help

Save

Abstract

En 中文

Software testing is a critical stage in the software development lifecycle, ensuring that programs behave correctly and reliably across diverse environments. Unlike general software testing, compiler testing requires well-structured input programs that systematically exercise the compiler's various optimization passes, since incomplete coverage can easily mask critical issues. Currently, a range of techniques exists for compiler testing, including differential testing, fuzzing, slicing, sanitizers and formal methods based validation tools. While these methods are effective at detecting compiler crashes, undefined behavior, or general correctness bugs, they are largely inadequate for systematically evaluating compiler optimizations. Tools like Csmith have pioneered random program generation for compiler testing, but they lack precise control over program structure, which is critical when the goal is to stress specific optimization passes or identify performance regressions. To address this limitation, we developed CPERFSMITH, an extension of Csmith that provides fine-grained control over code constructs, enabling targeted generation of programs that exercise particular compiler optimizations. Through our experiments, CERFSMITH was able to generate over 100 programs exhibiting significant runtime differences across industry-standard C compilers (GCC, Clang, and AOCC). Furthermore, we successfully created more than 30 test cases that revealed significant performance regressions between two subsequent release versions of Clang, demonstrating the tool's ability to detect missing or improperly applied optimizations.

Keywords:

Compiler Testing

Random Program Generation

Performance Bugs

Compiler Optimizations