The Journal of Korea Institute of Information, Electronics, and Communication Technology (한국정보전자통신기술학회논문지)

Korea Information Electronic Communication Technology (한국정보전자통신기술학회)
권호 표지
DOI QR코드

DOI QR Code

Speech Visualization of Korean Vowels Based on the Distances Among Acoustic Features

음성특징의 거리 개념에 기반한 한국어 모음 음성의 시각화

  • Pok, Gouchol (Division of Computer and IT Instruction, PaiChai University)
  • Received : 2019.10.02
  • Accepted : 2019.10.17
  • Published : 2019.10.29
Download PDF
⟨ Previous Next ⟩

Abstract

It is quite useful to represent speeches visually for learners who study foreign languages as well as the hearing impaired who cannot directly hear speeches, and a number of researches have been presented in the literature. They remain, however, at the level of representing the characteristics of speeches using colors or showing the changing shape of lips and mouth using the animation-based representation. As a result of such approaches, those methods cannot tell the users how far their pronunciations are away from the standard ones, and moreover they make it technically difficult to develop such a system in which users can correct their pronunciation in an interactive manner. In order to address these kind of drawbacks, this paper proposes a speech visualization model based on the relative distance between the user's speech and the standard one, furthermore suggests actual implementation directions by applying the proposed model to the visualization of Korean vowels. The method extract three formants F1, F2, and F3 from speech signals and feed them into the Kohonen's SOM to map the results into 2-D screen and represent each speech as a pint on the screen. We have presented a real system implemented using the open source formant analysis software on the speech of a Korean instructor and several foreign students studying Korean language, in which the user interface was built using the Javascript for the screen display.

음성을 시각적으로 표현하는 것은 외국어를 습득하는 과정의 학습자나 음성을 직접 들을 수 없는 청각장애자에게 매우 유용하며 기존에 다수의 연구가 이루어졌다. 그러나 기존의 연구들은 발음의 특징을 단지 컬러로 표현한다든가 입모양을 3차원 그래픽으로 표현하거나 입과 구강의 변화하는 형태를 애니메이션으로 보여 주는 방식에 머물러 있다. 따라서 이런 방식을 사용하는 학습자들은 자신의 발음이 표준 발음과 얼마나 멀리 떨어져 있는지 알 수가 없고 더 나아가서 학습 중에 스스로 교정을 해 나가는 시스템을 개발하기가 기술적으로 어려운 단점이 있다. 이를 극복하기 위해 본 논문에서는 음성 간의 상대적 거리를 토대로 음성을 시각화하는 모델을 제시하고, 이를 한국어 모음에 적용하여 모음의 음성적 특징을 이용한 시각화의 구체적인 구현 방법을 제시한다. 음성데이터에서 F1, F2, F3의 세 개의 포먼트를 구하고 이들 특징벡터를 코호넨 자기조직화맵 알고리즘으로 2차원 화면에 사상하여 각 음성을 화면 위의 각 점에 대응하여 표현하였다. 제안하는 시스템의 실제적인 구현은 인터넷에 공개된 음성처리 공개소프트웨어를 사용하고 한국인 교사의 표준 발음과 한국어를 배우고 있는 외국인 유학생의 음성을 이용하여 음성특징의 상호간 거리를 구하였으며, 사용자 인터페이스는 자바스크립트를 이용하여 구현하였다.

Keywords

References

  1. G. J. Borden, K. S. Harris, and L. J. Raphael, Speech science primer: physiology, acoustics, and perception of speech (Kim et al., Trans.), Seoul: Hankookmunhwasa, 2000.
  2. H.-Y. Sim , C.-H. Choi and S. H. Choi, "Characteristics of Vowel Formants, Vowel Space, and Speech Intelligibility Produced by Children Aged 3-6 Years," Audiology and Speech Research, vol.12, no. 4, pp. 260-269, 2016. https://doi.org/10.21848/asr.2016.12.4.260
  3. B. Yang, "A study on vowel formant variation by vocal tract modification," Phonetics and Speech Sciences vol. 3, no. 4, pp. 83-92, 1998.
  4. G. C, Yoon "A Comparative Study on the Male and Female Vowel Formants of the Korean Corpus of Spontaneous Speech," Phonetics and Speech Sciences vol. 7, no. 2, pp. 131-138, 2015. https://doi.org/10.13064/KSSS.2015.7.2.131
  5. A. Watanabe, S. Tomishige, and M. Nakatake "Speech Visualization by Integrating Features for the Hearing Impaired", IEEE Trans. Speech Audio Proc., vol 8, no 4, pp. 454-466, 2000. https://doi.org/10.1109/89.848226
  6. J. Beskow, O. Engwall, B. Granstrom, P. Nordqvist, and P. Wik, "Visualization of Speech and Audio for Hearing Impaired Persons," Technology and Disability, vol 20, pp. 97-107, 2008. https://doi.org/10.3233/TAD-2008-20205
  7. Y. Ueda, T. Sakada, and A. Watanabe, "Real-time Speech Visualization System for Speech Training and Diagnosis," Audio Engineering Society Convention Paper 8184, 2010 November 4, San Fransico, USA.
  8. D. S. Kim, T. H. Lee, and D. M. Lee, "An ambient display for hearing impaired people," Proc. Human Computer Interface Korea (HCI2006), pp.46 - 51, 2006.
  9. J.-H. Lee and H.-J. Chung, "A Study on Frequency Characteristics of Korean Phonemes," Audiology, 제1권, pp. 59-66, 2005.
  10. P. Denes and E. Pinson, The Speech Chain, W. H. Freeman and Company, (Ko et al., Trans.) 1995.
  11. B. Yang, "Formant Measurements of Complex Waves and Vowels Produced by Students," Phonetics and Speech Sciences vol. 15, no. 3, pp. 39-52, 2008.
  12. Y. Dissen, J. Goldberg, and J. Keshet, "Formant Estimation and Tracking: A Deep Learning Approach", J. Acoustic Society, vol.145, no.2, pp.1-11, 2019. https://doi.org/10.1121/1.5084264
  13. Kohonen, "Clustering, taxonomy, and topological maps of patterns," Proc. 6th Int. Conf. on Pattern Recognition, pp. 114-128, Washington, DC. IEEE Computer Soc. Press.
  14. State-of-the-art accurate phonetic tools based on machine-learning, https://mlspeech.github.io/index.html.