.png){kind=icon}
.png){kind=icon}
Return
Unsupervised Speech Segmentation and Variable Rate Representation Learning Using Segmental Contrastive Predictive Coding
.png){kind=icon}
OA
AI
DOI：10.1109/TASLP.2022.3180684.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Typically, unsupervised segmentation of speech into the phone- and wordlike units are treated as separate tasks and are often done via different methods which do not fully leverage the inter-dependence of the two tasks. Here, we unify them and propose a technique that can jointly perform both, showing that these two tasks indeed benefit from each other. Recent attempts employ self-supervised learning, such as contrastive predictive coding (CPC), where the next frame is predicted given past context. However, CPC only looks at the audio signal's frame-level structure. We overcome this limitation with a segmental contrastive predictive coding (SCPC) framework to model the signal structure at a higher level, e.g., phone level. A convolutional neural network learns frame-level representation from the raw waveform via noise-contrastive estimation (NCE). A differentiable boundary detector finds variable-length segments, which are then used to optimize a segment encoder via NCE to learn segment representations. The differentiable boundary detector allows us to train frame-level and segment-level encoders jointly. Experiments show that our single model outperforms existing phone and word segmentation methods on TIMIT and Buckeye datasets. Finally, we use SCPC to extract speech features at the segment level rather than at the uniformly spaced frame level (e.g., 10 ms) and produce variable rate representations that change according to the contents of the utterance. We lower the feature extraction rate from the typical 100 Hz to 14.5 Hz on average while still outperforming the hand-crafted features such as MFCC on the linear phone classification task.
Keywords:
Feature extraction
Task analysis
Speech processing
Detectors
Predictive coding
Hidden Markov models
Representation learning
Self-supervised learning
unsupervised phone segmentation
variable-rate representation learning
unsupervised word segmentation
Journal
I
IF:
5.1
Papers:
2.6K
Citations:
1.1W