Character-Level Singing Technique Detection and Evaluation: A Two-Stage Approach

Qi Xue

doi:10.1109/lsp.2026.3716909

Character-Level Singing Technique Detection and Evaluation: A Two-Stage Approach

2026-07-24

0

PRE

AI

DOI：10.1109/lsp.2026.3716909

Original

Original request for help

Save

Abstract

En 中文

We address automated singing voice evaluation: given a teacher's demonstration and a student's recording of the same lyrics, the system produces character-level comparisons of pitch, rhythm, and six singing techniques, together with natural-language pedagogical feedback. We construct a dataset of 870 real-world teacher–student pairs and obtain expert-verified technique labels on a 246-song subset. We propose a two-stage system: the first stage extracts character-level timestamps, pitch contours, and technique labels via forced alignment, fundamental frequency estimation, and a technique extractor built on self-supervised music representations; the second stage feeds the structured representation into a fine-tuned large language model to generate diagnostic feedback. On expert-annotated data, the technique extractor achieves a character-level average F-score of 0.82, versus 0.01–0.10 for three state-of-the-art audio multimodal large language models; it is independently validated on an open-source benchmark with an event-level macro F-score of 0.83. On the feedback task, these models further exhibit confident hallucination—producing detailed, well-structured suggestions that are grounded in incorrectly identified problems—highlighting the limitations of end-to-end audio models for fine-grained singing evaluation.

Keywords:

Singing voice evaluation

singing technique detection

character-level alignment

music information retrieval

Character-Level Singing Technique Detection and Evaluation: A Two-Stage Approach

Abstract

Journal

Organization