An Automatic Real-time Synchronization of Live speech with Its Transcription Approach

  • Nat Lertwongkhanakool Chulalongkorn University
  • Natthawut Kertkeidkachorn Chulalongkorn University
  • Proadpran Punyabukkana Chulalongkorn University
  • Atiwong Suchato Chulalongkorn University
Keywords: Automatic speech-text synchronization, Syllable Detection, Real-Time alignment, Live speech and transcription alignment, Transcription verification

Abstract

Most studies in automatic synchronization of speech and transcription focus on the synchronization at the sentence level or the phrase level. Nevertheless, in some languages, like Thai, boundaries of such levels are difficult to linguistically define, especially in case of the synchronization of speech and its transcription. Consequently, the synchronization at a finer level like the syllabic level is promising. In this article, an approach to synchronize live speech with its corresponding transcription in real time at the syllabic level is proposed. Our approach employs the modified real-time syllable detection procedure from our previous work and the transcription verification procedure then adopts to verify correctness and to recover errors caused by the real-time syllable detection procedure. In experiments, the acoustic features and the parameters are customized empirically. Results are compared with two baselines which have been applied to the Thai scenario. Experimental results indicate that, our approach outperforms two baselines with error rate reduction of 75.9% and 41.9% respectively and also can provide results in the real-time situation. Besides, our approach is applied to the practical application, namely ChulaDAISY. Practical experiments show that ChulaDAISY applied with our approach could reduce time consumption for producing audio books.

Downloads

Download data is not yet available.

Author Biographies

Nat Lertwongkhanakool

Department of Computer Engineering, Faculty of Engineering, Chulalongkorn University, Bangkok 10330, Thailand

Natthawut Kertkeidkachorn

Department of Computer Engineering, Faculty of Engineering, Chulalongkorn University, Bangkok 10330, Thailand

Proadpran Punyabukkana

Department of Computer Engineering, Faculty of Engineering, Chulalongkorn University, Bangkok 10330, Thailand

Atiwong Suchato

Department of Computer Engineering, Faculty of Engineering, Chulalongkorn University, Bangkok 10330, Thailand

Published In
Vol 19 No 5, Oct 31, 2015
How to Cite
[1]
N. Lertwongkhanakool, N. Kertkeidkachorn, P. Punyabukkana, and A. Suchato, “An Automatic Real-time Synchronization of Live speech with Its Transcription Approach”, Eng. J., vol. 19, no. 5, pp. 81-99, Oct. 2015.