CrossSimEmb: transformer-based embedding model for cross-compilation binary code similarity detection

Gaoqing Yu

doi:10.1007/s11227-025-07895-3

Return

CrossSimEmb: transformer-based embedding model for cross-compilation binary code similarity detection

2025-10-03

PRE

DOI：10.1007/s11227-025-07895-3

Original

Original request for help

Save

Abstract

En 中文

Transformer-based pre-trained models have achieved breakthrough progress in binary code similarity detection tasks by modeling disassembled code as textual sequences, enabling researchers to successfully transfer natural language processing methodologies to binary code analysis. However, existing approaches remain limited by their lack of multi-granularity semantic feature extraction capability and multi-task generalization capability. In this paper, we propose CrossSimEmb, a cross-compilation binary code semantic representation and metric model, which constructs a multi-task joint optimization framework encompassing 4 pre-training tasks and 3 fine-tuning tasks. Leveraging high-performance computing resources, our framework is trained over large-scale binary datasets across multiple architectures, enabling effective integration of disassembled text features, instruction-level contextual features, inter-block structural features, and architecture-specific characteristics. In the fine-tuning phase, a Siamese network architecture is implemented to generate function-level code embeddings, while a joint optimization strategy combining binary classification, distance-based metric learning and multi-label classification is introduced for downstream tasks, effectively improving the model’s generalization capability and task adaptability in cross-compilation scenarios. Experimental results–conducted on HPC clusters–demonstrate that CrossSimEmb outperforms state-of-the-art solutions in both binary code similarity detection and outlier detection tasks: achieving 80.34% average accuracy and 0.901 AUC for binary code similarity detection, and attaining 0.961 MCC and 0.981 G-Means for outlier detection. These findings highlight both the effectiveness of our approach and the essential role of HPC resources in enabling large-scale training and evaluation.

Keywords:

Binary code similarity detection

Transformer

Metric learning

Representation learning