.png){kind=icon}
Return
CacheC: LLM-Based GPU Cache Management to Enhance Kernel Concurrency
.png){kind=icon}
PRE
AI
DOI：10.1007/978-3-031-99857-7_9.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Each new generation of GPUs significantly enhances the resources available for diverseGPGPU applications, with kernel concurrency playing a crucial role in maximizing utilization and boosting performance. However, existing kernel concurrency strategies usually tend to neglect cache contention, where concurrent kernels potentially target the same cache levels. Traditional cache management methods are inadequate for addressing this issue, as they focus on individual kernels without heavily considering inter-kernel interactions. To overcome these challenges, we propose CacheC, a method that utilizes large language models (LLMs) to analyze cache affinity at the granularity of individual load instructions. For each kernel pair, CacheC extracts detailed features of all loads, evaluates their cache affinity across levels, and scores their suitability for concurrency. Based on these scores, CacheC not only selects kernel pairs with appropriateoptimal cache compatibility but also formulates load-specific cache bypassing strategies to enhance utilization. By iteratively scheduling kernel pairs and adjusting their cache policies, CacheC dynamically optimizes cache utilization and reduces cache contention during concurrent kernel execution. Experiments on off-theshelf GPUs demonstrate that CacheC achieves a 19.67% reduction in turnaround time and a 24.48% improvement in throughput. It also delivers an average speedup of 1.337.x across scheduled kernel pairs, showcasing its effectiveness in alleviating cache contention and enhancing kernel concurrency performance.
Keywords:
GPU cache management
Concurrent kernel execution
Kernel pairing
Large language models
Journal
E
IF:
0
Papers:
24
Citations:
0
Organization
Cited Papers
ShareSaveA Survey on Large Language Models: Applications, Challenges, Limitations, and Practical Usage.png)
OAAI
IF0
OAAIShareSaveShareSave
View more