.png){kind=icon}
返回
Depression Classification From Tweets Using Small Deep Transfer Learning Language Models
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2022.3223049.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Depression detection from social media texts such as Tweets or Facebook comments could be very beneficial as early detection of depression may even avoid extreme consequences of long-term depression i.e. suicide. In this study, depression intensity classification is performed using a labeled Twitter dataset. Further, this study makes a detailed performance evaluation of four transformer-based pre-trained small language models, particularly those having less than 15 million tunable parameters i.e. Electra Small Generator (ESG), Electra Small Discriminator (ESD), XtremeDistil-L6 (XDL) and Albert Base V2 (ABV) for classification of depression intensity using Tweets. The models are fine-tuned to get the best performance by applying different hyperparameters. The models are tested by classification of depression intensity of labeled tweets for three label classes i.e. 'severe', 'moderate', and 'mild' by downstream fine-tuning the parameters. Evaluation metrics such as accuracy, F1, precision, recall, and specificity are calculated to evaluate the performance of the models. Comparative analysis of these models is also done with a moderately larger model i.e. DistilBert which has 67 million tunable parameters for the same task with the same experimental settings. Results indicate that ESG outperforms all other models including DistilBert due to its better deep contextualized text representation as it gets the best F1 score of 89% with comparatively less training time. Further optimization of ESG is also proposed to make it suitable for low-powered devices. This study helps to achieve better classification performance of depression detection as well as to choose the best language model in terms of performance and less training time for Twitter-related downstream NLP tasks.
Keyword:
Depression classification
transfer learning
transformer language models
public health
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
引用论文
An emotional classification method of Chinese short comment text based on ELECTRA基于ELECTRA的中文短评文本情感分类方法.png)
CONNECTION SCIENCEOAAI
CONNECTION SCIENCEIF3.4
OAAI分享收藏分享收藏分享收藏Self-Harm, Suicidal Behaviours, and Cyberbullying in Children and Young People: Systematic Review儿童和青少年的自我伤害，自杀行为和网络欺凌: 系统综述
OAAI
OAAI分享收藏
查看更多内容