.png){kind=icon}
返回
Causal Language Model Aided Sequential Decoding With Natural Redundancy
.png){kind=icon}
PRE
AI
DOI：10.1109/TCOMM.2023.3247733.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
A high-performance communication receiver desires a sufficient and accurate recognition of transmitted sources. In this paper, we propose a sequential decoding algorithm for the robust reception of sources with natural redundancy (NR) over the AWGN channel. To fully exploit the abundant NR in the exemplified English text sources, a causal language modeling (CLM) with a powerful Transformer decoder neural network (NN) structure is adopted for modeling the joint probability distribution. For versatility of byte-level tokenization, the UTF-8 encoding scheme is considered for the Wikipedia corpus, namely the English edition of the Wiki-40B dataset. The tree search-based M-algorithm (MA) is integrated with CLM, denoted as CLM-MA algorithm, to synthesize the a priori probability of information sequence and likelihood values of polluted symbols. Both simulation experiments and hardware platform evaluations are conducted to analyze the performance of the CLM-MA algorithm. With an adequately large M value, tremendous performance profit is achievable, especially for high-efficiency modulation levels and extremely noisy conditions. This mechanism of adopting the pre-trained NN model merely desires self-supervised learning architecture and eliminates the requirement of high-cost and accurate labels, which paves a new way for communication receiver design.
Keyword:
Decoding
Redundancy
Receivers
Artificial neural networks
Transformers
Tokenization
Task analysis
Deep learning
error correction
sequential decoding
causal language model
natural redundancy
期刊
IF:
8.3
论文数:
1.2W
被引数:
3.6W
机构
引用论文
.png){kind=icon}
Effect of milking vacuum level and overmilking on cows' teat characteristics挤奶真空度和过度挤奶对奶牛奶头特性的影响
OAAI
OAAI分享收藏分享收藏
查看更多内容