.png){kind=icon}
.png){kind=icon}
Return
Attribute-Based Injection Transformer for Personalized Sentiment Analysis
.png){kind=icon}
PRE
AI
DOI：10.1109/TETCI.2024.3369323.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Personal attributes have been proven to be useful for sentiment analysis. However, previous models of learning attribute-specific language representations are suboptimal because only context- or content-wise injection is adopted. This study proposes a transformer structure with a combination of both context- and content-wise injections based on a well pretrained transformer encoder. For context-wise injection, self-interactive attention is implemented by incorporating personal attributes into a multi-head attention. For the content-wise perspective, an attribute-based layer normalization is used to align text representation with personal attributes. In particular, the proposed transformer layer can be a universal layer compatible with the original Google Transformer layer. Instead of training from scratch, the proposed Transformer layer can be initialized from a well pre-trained checkpoint for downstream tasks. Extensive experiments were conducted on three benchmarks of document-level sentiment analysis, including IMDB, Yelp-2013 and Yelp-2014. The results show that the proposed method outperforms the previous methods for personalized sentiment analysis, demonstrating that the combination of both context- and content-wise injections can facilitate model learning for attribute-specific language representations.
Keywords:
Transformers
Reviews
Sentiment analysis
Task analysis
Analytical models
Context modeling
Training
Personalized sentiment analysis
attention mechanism
layer normalization
pre-trained language model
Journal
I
IF:
6.5
Papers:
1.4K
Citations:
4.5K