.png){kind=icon}
.png){kind=icon}
Return
SampleLLM: Prune LLMs via learnable structure sampling
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2026.134215.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
As the scale of large language models (LLMs) continues to grow, there is an increasing need to reduce computational overhead. Structured pruning has been proven to be an effective technique for model compression. However, existing methods are often hindered by a reliance on manually defined heuristic metrics and a fragmented “prune-then-retrain” pipeline that ignores the correlations between different pruning structures. To address these drawbacks, we introduce SampleLLM, which formulates pruning as a learnable sampling process. By jointly optimizing sampling parameters and LoRA weights, SampleLLM captures dynamic changes in the importance of pruning structures and eliminates the need for manual pruning metrics. We also introduce a self-distillation mechanism to further align the pruned model with its dense counterpart. Extensive experiments on 9 models across 5 families demonstrate that SampleLLM outperforms state-of-the-art structured pruning methods. For example, our 40% pruned LLaMA2-70B model retains 98% of its original performance. Moreover, at the same compression rate, it achieves superior performance and inference speedup compared to 2:4 semi-structured pruning schemes.
Journal
IF:
6.5
Papers:
2.5W
Citations:
6.5W