.png){kind=icon}
.png){kind=icon}
Return
A bias evaluation solution for multiple sensitive attribute speech recognition
.png){kind=icon}
PRE
AI
DOI：10.1016/j.csl.2025.101787.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Speech recognition systems are a pervasive application in the field of Al (Artificial Intelligence), bringing significant benefits to society. However, they also face significant fairness issues. When dealing with groups of people with different sensitive attributes, these systems tend to exhibit bias, which may lead to the misinterpretation or ignoring of the voice of a specific group of people. In order to address the fairness issue, it becomes crucial to comprehensively reveal the presence of bias in AI systems. To address the issues of limited categories and data imbalance in existing bias evaluation datasets, we propose a new method for constructing evaluation datasets. Given the unique characteristics of speech recognition systems, we find that existing AI bias evaluation methods may not be directly applicable. Therefore, we introduce a bias evaluation method for speech recognition systems based on WER (Word Error Rate). To comprehensively quantify bias across different groups, we combine multiple evaluation metrics, including WER, fairness metrics, and CMBM (confusion matrix-based metrics). To ensure a thorough evaluation, experiments were conducted on both single sensitive attributes and cross- sensitive attributes. The experimental results indicate that, for single sensitive attributes, the speech recognition system exhibits the most significant racial bias, while in the evaluation of cross-sensitive attributes, the system shows the greatest bias against white males and black males. Finally, through T-tests, we demonstrate that the WER differences between these two groups are statistically significant.
Keywords:
Artificial intelligence
Speech recognition
Bias assessment
AI fairness
Journal
C
IF:
3.4
Papers:
1.5K
Citations:
2.6K