Generative Spoken Dialogue Language Modeling

Tu Anh Nguyen

doi:10.1162/tacl_a_00545

Generative Spoken Dialogue Language Modeling

2023-03-14

16

OA

AI

DOI：10.1162/tacl_a_00545

Original

Original request for help

Save

Abstract

En 中文

We introduce dGSLM, the first textless model able to generate audio samples of naturalistic spoken dialogues. It uses recent work on unsupervised spoken unit discovery coupled with a dual-tower transformer architecture with cross-attention trained on 2000 hours of two-channel raw conversational audio (Fisher dataset) without any text or labels. We show that our model is able to generate speech, laughter, and other paralinguistic signals in the two channels simultaneously and reproduces more naturalistic and fluid turn taking compared to a text-based cascaded model.(1),(2)

Keywords:

TURN-TAKING

ORGANIZATION

Generative Spoken Dialogue Language Modeling

Abstract

Journal

Organization