Dataset for Dzongkha and English Speech-to-Speech Translation

Abstract

Dzongkha is a low-resource language. Lack of a parallel Speech-to-speech translation dataset is a major challenge. This paper introduces a novel method for curating the first Dzongkha-English speech parallel corpus. The proposed method uses divergent TTS models to synthesize aligned speech from translated text pairs. Furthermore, Google’s gTTS is used for English, and a tonally adjusted Tacotron2 system for Dzongkha. The dataset includes 7,385 Dzongkha ASR audio-text pairs, 419,731 Dzongkha-English machine translation text pairs, 1,530 Dzongkha TTS audio-text pairs, and 240 hours of synthetic speech data each for Speech-to-Unit and Dzongkha Unit-to-Speech tasks. The dataset curation pipeline entails rigorous text normalization, 22.05kHz dual-language audio recording, and three-stage signal conditioning-noise reduction, silence truncation, and EBU R128 loudness normalization. Most notably, we use tool-assisted temporal alignment followed by human verification to ensure sub-second synchronism of utterances. By rejecting 8.7% of samples via PESQ quality thresholds, we ensure acoustic consistency. This work constructs a scalable method for producing high-fidelity parallel speech data for under-resourced languages directly facilitating S2ST model development in the lack of natural datasets.