.png){kind=icon}
返回
Speaker-independent dysarthria severity classification using self-supervised transformers and multi-task learning
.png){kind=icon}
PRE
AI
DOI：10.1371/journal.pdig.0001076.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Dysarthria, characterised by slurred speech, is a hallmark of many neurological disorders and brain trauma. Clinical assessment requires an audio-visual investigation by a trained healthcare expert, who evaluates criteria such as respiration, phonation, articulation, resonance, and prosody during speech. Quantitative assessment of dysarthria is challenging due to its complexity, variability, and the subjective nature of human-observation-based scoring methods. We present a novel machine-learning framework using transformers for stratifying and monitoring patient speech. Our framework integrates a wav2vec 2.0 model, pre-trained on raw speech data from healthy individuals. To reduce reliance on speaker-specific characteristics and effectively manage the intrinsic intra-class variability of dysarthric speech, we employ a contrastive learning strategy with a multi-task objective: cross-entropy loss for classifying dysarthria severity, and triplet margin loss to ensure latent embeddings are grouped by severity rather than by speaker. This Speaker-Agnostic Latent Regularisation (SALR) framework provides an objective, accessible, and cost-effective alternative to traditional assessments. On the UA-Speech dataset, SALR achieved 70.5% accuracy and 59.2% F1 using leave-one-subject-out cross-validation-a 16.5% absolute (30% relative) improvement over prior benchmarks. Explainability analysis indicates that our multi-task objective enhances the ordinal structure of the latent space, reducing dependence on speaker-specific cues and demonstrating robustness and generalisability. In conclusion, this proof-of-concept study demonstrates the potential of the SALR framework for speaker-independent dysarthria severity classification, with potential implications for broader clinical applications in automated dysarthria assessments.
Keyword:
SPEECH
MODELS
期刊
P
IF:
0
论文数:
188
被引数:
0
机构
引用论文
Automatic Assessment of Dysarthric Severity Level Using Audio-Video Cross-Modal Approach in Deep Learning基于深度学习的音频-视频跨模态方法对构音障碍严重程度的自动评估
OAAI
OAAI分享收藏HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units休伯特: 通过隐藏单元的掩蔽预测进行自我监督的语音表示学习OAAI
OAAI分享收藏Dysarthria severity classification using multi-head attention and multi-task learning利用多头注意力机制和多任务学习的构音障碍严重程度分类
PREAI
分享收藏Dysarthria Speech Detection Using Convolutional Neural Networks with Gated Recurrent Unit使用具有门控循环单元的卷积神经网络进行构音障碍语音检测.png)
Healthcare
OAAI
HealthcareIF0
OAAI分享收藏On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation通过逐层相关性传播对非线性分类器决策的逐像素解释PLOS ONEOAAI
PLOS ONEIF0
OAAI分享收藏
查看更多内容