EvoP: Robust LLM Inference via Evolutionary Pruning

doi:10.1007/978-981-95-3343-5_36

EvoP: Robust LLM Inference via Evolutionary Pruning

2026-01-01

0

PRE

AI

DOI：10.1007/978-981-95-3343-5_36

Original

Original request for help

Save

Abstract

En 中文

Large Language Models (LLMs) have achieved remarkable success in natural language processing tasks, but their massive size and computational demands hinder their deployment in resource-constrained environments. Existing model pruning methods address this issue by removing redundant structures (e.g., elements, channels, layers) from the model. However, these methods employ a heuristic pruning strategy, which leads to suboptimal performance. Besides, they also ignore the data characteristics when pruning the model. To overcome these limitations, we propose EvoP, an evolutionary pruning framework for robust LLM inference. EvoP first presents a cluster-based calibration dataset sampling (CCDS) strategy for creating a more diverse calibration dataset. EvoP then introduces an evolutionary pruning pattern searching (EPPS) method to find the optimal pruning pattern. Compared to existing model pruning techniques, EvoP achieves the best performance while maintaining the best efficiency. Experiments across different LLMs and different downstream tasks validate the effectiveness of the proposed EvoP, making it a practical and scalable solution for deploying LLMs in real-world applications.

Keywords:

Large Language Model

Layer-wise Model Pruning

Evolutionary Search

EvoP: Robust LLM Inference via Evolutionary Pruning

Abstract

Journal

Organization