.png){kind=icon}
.png){kind=icon}
Return
Speaker Group Encoding in Self-supervised Speech Recognition Models
.png){kind=icon}
PRE
AI
DOI：10.1007/978-3-032-02548-7_11.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
We investigate what self-supervised speech recognition models (S3Ms) learn about speaker groups (SGs). We examine several states of S3Ms: pretrained, finetuned on speaker identification (SID), as well as finetuned automatic speech recognition (ASR) using fairness enhancing algorithms. We find that S3Ms encode information about several speaker group categories (SGCs), including their gender, age, dialect, ethnicity, and whether they are a native speaker. We find that finetuning for SID amplifies certain SGCs, namely those whose variance is more phonetic in nature. Meanwhile, finetuning for ASR discards phonetically variant speaker group information (SGI) but retains semantically variant SGI. We find that ASR algorithms designed for fairness improvement change to what extent SGI is encoded in S3Ms; however, this is primarily true for phonetically variant SGCs, and less true for semantically variant SGCs. We discuss how SGI is encoded by each layer, and identify subdimensions of embeddings responsible for encoding different SGCs. Finally, we discuss how our findings could be beneficial in designing fairer ASR algorithms.
Keywords:
Self-supervised learning
interpretability
fairness
Journal
T
IF:
0
Papers:
31
Citations:
0