.png){kind=icon}
.png){kind=icon}
Return
What really happens when a dev vibes with the code? An empirical study on LLM behavioral divergence in response to expressive code comments
.png){kind=icon}
OA
AI
A
DOI：10.3389/frai.2026.1784973.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
IntroductionWe investigate how expressive inline code comments written in various developer styles; functional to progressively poetic; philosophical; and misleading; affect large language model (LLM) behavior during code optimization.MethodsIn this pilot study; we used a controlledmerge sort implementation across five stylistic variants and evaluated GPT-5 and Claude Opus 4.1 under standardized console prompts; isolating the effect of embedded comment semiotic variation. Seven expert developers (three senior; four mid-level) scored model outputs against adapted ISO/IEC 25010 criteria and novel LLM suggestibility index (LSI) framework.ResultsSemiotic character of comments measurably altered code quality; with consensus-score reliability ICC(2; k) = 0.65–0.81 for six of seven dimensions; single-rater Krippendorff's α = 0.232 reflects substantial interpretive variability. Claude exhibited higher interpretive sensitivity (mean behavioral divergence 4.00; SD 1.16); while GPT-5 maintained stronger architectural fidelity (mean divergence 3.58; SD 1.26). Reflective comments (philosophical; conversational) were associated with Claude's highest maintainability scores in our panel (both M = 4.00; ~8% above stock M = 3.71); while the same philosophical comments reduced GPT-5 maintainability (M = 2.86); suggesting asymmetric model responses to expressive context.ConclusionsThese findings position inline comments as model-sensitive latent semantic prompts; with implications for AI-in-the-loop development and design of comment conventions for AI-assisted maintenance.
Keywords:
large language models
software development
human–AI collaboration
vibe coding
agentic code
code comment
Journal
F
IF:
4.7
Papers:
2.2K
Citations:
4.4K
Organization
Cited Papers