arrow
Return

Private Protocol Reverse Engineering via Self-Supervised Learning-Based Message Segmentation

delete2026-01-22
delete0
PRE
AI
J
Junchen Li
G
Guang Cheng
H
Huimin Tang
Y
Ying Hu
Q
Qinghua Shang
DOI:10.1109/TIFS.2026.3657097delete
deleteOriginal
deleteShare
deleteSave
Abstract

Abstract

En 中文
Private protocol reverse engineering is the main way to solve the problem of unknown traffic which brings huge security risks to the current network environment. The network traffic-based protocol reverse engineering approaches are the basis of traffic security supervision and are also widely used and flexible. These approaches utilize multiple algorithms from different perspectives to extract the protocol specifications from messages, but they fail to recognize the importance of message segmentation and do not adequately evaluate the relation of adjacent bytes, leading to imprecise performance. To address these issues, we propose the SLMSP, a self-supervised learning-based message segmentation approach for private protocol reverse engineering in this paper. SLMSP mines the rich information embedded in the word order and word semantics between adjacent bytes through self-supervised learning, and then makes optimal decisions about where the message should be segmented based on the fusion of those information, combing the horizontal inference and vertical correction. After that, SLMSP extracts protocol formats based on fine-grained message segmentation by introducing the progressive sequence merging algorithm. We conduct comprehensive experiments to demonstrate the effectiveness of SLMSP. The experimental results demonstrate that SLMSP achieves the ideal performance both in message segmentation and format inference, and it also has advantages over previous works.
Keywords:
Private protocol
protocol reverse engineering
self-supervised learning
message segmentation
format inference

Journal

IEEE Transactions on Information Forensics and Security cover
IEEE Transactions on Information Forensics and Security
IF:
8
Papers:
5.4K
Citations:
2.6W

Organization

H
hebei
Scholars:
0
Papers: 1
Citations: 0
S
Southeast University
Scholars:
6.7K
Papers: 2.2K
Citations: 43
Cited Papers

Cited Papers

No cited papers available