.png){kind=icon}
.png){kind=icon}
Return
Learning Discriminative Syntax-Semantic Patterns With Transformer-Based Contrastive Learning
.png){kind=icon}
PRE
AI
DOI：10.54364/AAIML.2026.61273.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
This paper proposes a dual-encoder framework that learns syntax-and semantics-aligned sentence embeddings using contrastive learning. The proposed Syntax-Semantic Contrastive Pretraining (SSCP) model employs two transformer encoders to separately model syntactic structure and semantic content, which are aligned in a shared embedding space via a symmetric contrastive objective. Across standard benchmarks, SSCP consistently outperforms strong baselines such as BERT, SimCSE, and SyntaxBERT. In particular, SSCP improves Spearman correlation on STS-B by +1.4 points over SimCSE, increases PAWS paraphrase accuracy by +2.5 points, and achieves 96.1% accuracy on TREC question classification, exceeding existing syntax-aware models. Probing experiments further show gains of up to +4-5 points on syntactic structure prediction tasks, confirming that SSCP preserves grammatical information while maintaining semantic robustness. These results demonstrate that explicitly aligning syntactic and semantic views yields representations that are more discriminative, interpretable, and robust than single-view or syntax-augmented approaches, positioning SSCP as a principled multi-view pretraining strategy for structure-aware language understanding.
Keywords:
Transformer models
Contrastive learning
Syntax-Semantic representation
Sentence embeddings
Pattern recognition
Representation learning
Natural language processing
Multi-View learning
Machine learning
Journal
A
IF:
0.5
Papers:
41
Citations:
0
