Implicit Self-Supervised Language Representation for Spoken Language Diarization

Jagabandhu Mishra

doi:10.1109/TASLP.2024.3426978

2024-01-01

Jagabandhu Mishra *

S. R. Mahadeva Prasanna

DOI：10.1109/TASLP.2024.3426978

原文链接

原文求助

摘要

En 中文

The use of spoken language diarization (LD) as a preprocessing system might be essential in a code-switched (CS) scenario. Furthermore, implicit frameworks are preferable to explicit ones, as implicit frameworks can be easily adapted to deal with low/zero resource languages. Inspired by speaker diarization literature, three frameworks based on (a) fixed segmentation, (b) change-point-based segmentation, and (c) end-to-end (E2E) are used in this study to perform LD. The initial exploration in the constructed text-to-speech female language diarization (TTSF-LD) dataset shows, that using the x-vector as implicit language representation with appropriate analysis window length achieves, comparable performance to explicit LD. The best implicit LD performance of 6.4% in terms of Jaccard error rate (JER) is achieved by using the E2E framework. However, using the natural Microsoft CS dataset, the performance of the E2E implicit LD degrades to 60.4% JER. The performance degradation is due to the inability of the x-vector representation to capture language-specific traits. To address this shortcoming, a self-supervised implicit language representation framework is used in this study. Compared to the x-vector representation, the self-supervised representation yields a relative improvement of 63.9%, achieving a JER of 21.8% when used in conjunction with the E2E framework.

Keyword:

Task analysis

Feature extraction

Vectors

Mel frequency cepstral coefficient

Spectrogram

Deep learning

Covariance matrices

Language diarization (LD)

x-vector

language change detection

self-supervised implicit language representation