Science of Emotion and Sensibility (감성과학)

Korean Society for Emotion and Sensibility (한국감성과학회)
권호 표지
DOI QR코드

DOI QR Code

Neural-network based Computerized Emotion Analysis using Multiple Biological Signals

다중 생체신호를 이용한 신경망 기반 전산화 감정해석

  • Lee, Jee-Eun (Department of Medical Engineering, Yonsei University College of Medicine) ;
  • Kim, Byeong-Nam (Department of Medical Engineering, Yonsei University College of Medicine) ;
  • Yoo, Sun-Kook (Department of Medical Engineering, Yonsei University College of Medicine)
  • 이지은 (연세대학교 의과대학 의학공학교실) ;
  • 김병남 (연세대학교 의과대학 의학공학교실) ;
  • 유선국 (연세대학교 의과대학 의학공학교실)
  • Received : 2016.03.14
  • Accepted : 2017.03.21
  • Published : 2017.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

Emotion affects many parts of human life such as learning ability, behavior and judgment. It is important to understand human nature. Emotion can only be inferred from facial expressions or gestures, what it actually is. In particular, emotion is difficult to classify not only because individuals feel differently about emotion but also because visually induced emotion does not sustain during whole testing period. To solve the problem, we acquired bio-signals and extracted features from those signals, which offer objective information about emotion stimulus. The emotion pattern classifier was composed of unsupervised learning algorithm with hidden nodes and feature vectors. Restricted Boltzmann machine (RBM) based on probability estimation was used in the unsupervised learning and maps emotion features to transformed dimensions. The emotion was characterized by non-linear classifiers with hidden nodes of a multi layer neural network, named deep belief network (DBN). The accuracy of DBN (about 94 %) was better than that of back-propagation neural network (about 40 %). The DBN showed good performance as the emotion pattern classifier.

감정은 학습능력, 행동, 판단력 등 삶의 많은 부분에 영향을 끼치므로 인간의 본질을 이해하는 데 중요한 역할을 한다. 그러나 감정은 개인이 느끼는 강도가 다르며, 시각 영상 자극을 통해 감정을 유도하는 경우 감정이 지속적으로 유지되지 않는다. 이러한 문제점을 극복하기 위하여 총 4가지 감정자극(행복, 슬픔, 공포, 보통) 시 생체신호(뇌전도, 맥파, 피부전도도, 피부 온도)를 획득하고, 이로부터 특징을 추출하여 분류기의 입력으로 사용하였다. 감정 패턴을 확률적으로 해석하여 다른 공간으로 매핑시켜주는 역할을 하는 Restricted Boltzmann Machine (RBM)과 Multilayer Neural Network (MNN)의 은닉층 노드를 이용하여 비선형적인 성질의 감정을 구별하는 Deep Belief Network (DBN) 감정 패턴 분류기를 설계하였다. 그 결과, DBN의 정확도(약 94%)는 오류 역전파 알고리즘의 정확도(약 40%)보다 높은 정확도를 가지며 감정 패턴 분류기로서 우수성을 가짐을 확인하였다. 이는 향후 인지과학 및 HCI 분야 등에서 활용 가능할 것으로 사료된다.

Keywords

References

  1. Chen, H., & Murray, A. F. (2003). Continuous Restricted Boltzmann Machine with an Implementable Training Algorithm. In Vision, Image and Signal Processing, IEEE Proceeding-, 150(3), 153-158. https://doi.org/10.1049/ip-vis:20030362
  2. Choi, W. (2011). A Classification Analysis of Negative Emotion Based on PPG Signal Using Fuzzy-Ga. master's thesis, Yonsei University, Seoul.
  3. Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., & Taylor, J. G. (2001). Emotion Recognition in Human-Computer Interaction. Signal Processing Magazine, IEEE, 18(1), 32-80. https://doi.org/10.1109/79.911197
  4. Guang-yuan, L., & Min, H. (2009). Emotion Recognition of Physiological Signals Based on Adaptive Hierarchical Genetic Algorithm. In 2009 World Congress on Computer Science and Information Engineering, 670-674.
  5. Haag, A., Goronzy, S., Schaich, P., & Williams, J. (2004). Emotion Recognition Using Bio-Sensors: First Steps Towards an Automatic System. In Tutorial and Research Workshop on Affective Dialogue Systems, 36-48.
  6. Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006). A Fast Learning Algorithm for Deep Belief Nets. Neural Computation, 18(7), 1527-1554. https://doi.org/10.1162/neco.2006.18.7.1527
  7. Jerritta, S., Murugappan, M., Nagarajan, R., & Wan, K. (2011). Physiological Signals Based Human Emotion Recognition: A Review. In Signal Processing and its Applications (CSPA), 2011 IEEE 7th International Colloquium on, 410-415.
  8. Khashman, A. (2008). A Modified Backpropagation Learning Algorithm with Added Emotional Coefficients. Neural Networks, IEEE Transactions on, 19(11), 1896-1909. https://doi.org/10.1109/TNN.2008.2002913
  9. Kleinginna, P. R., & Kleinginna, A. M. (1985). Cognition and affect: A reply to Lazarus and Zajonc. American Psychologist, 40(4), 470-471. https://doi.org/10.1037/0003-066X.40.4.470
  10. Krause, R. (1987). Universals and Cultural Differences in the Judgments of Facial Expressions of Emotion. Journal of Personality and Social Psychology, 53(4), 712-717. https://doi.org/10.1037/0022-3514.53.4.712
  11. Lang, P. J. (1995). The Emotion Probe: Studies of Motivation and Attention. American Psychologist, 50(5), 372-385. https://doi.org/10.1037/0003-066X.50.5.372
  12. LeCun, Y., & Ranzato, M. (2013). Deep Learning Tutorial. In Tutorials in International Conference on Machine Learning (ICML13), Citeseer.
  13. Lisetti, C. L., & Nasoz, F. (2004). Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals. EURASIP Journal on Advances in Signal Processing, 11, 1-16.
  14. Malik, M., & Camm, A. J. (1990). Heart Rate Variability. Clinical Cardiology, 13(8), 570-576. https://doi.org/10.1002/clc.4960130811
  15. Moretti, D. V., Babiloni, C., Binetti, G., Cassetta, E., Dal Forno, G., Ferreric, F., & Nobili, F. (2004). Individual Analysis of Eeg Frequency and Band Power in Mild Alzheimer's Disease. Clinical Neurophysiology, 115(2), 299-308. https://doi.org/10.1016/S1388-2457(03)00345-6
  16. Murugappan, M., Ramachandran, N., & Sazali, Y. (2010). Classification of Human Emotion from Eeg Using Discrete Wavelet Transform. Journal of Biomedical Science and Engineering, 3, 390-396. https://doi.org/10.4236/jbise.2010.34054
  17. Niu, X., Chen, L., & Chen, Q. (2011). Research on Genetic Algorithm Based on Emotion Recognition Using Physiological Signals. In 2011 International Conference on Computational Problem-Solving, 614-618.
  18. Peng, Y., Zhu, J.-Y., Zheng, W.-L., & Lu, B.-L. (2014). Eeg-Based Emotion Recognition with Manifold Regularized Extreme Learning Machine. In Engineering in Medicine and Biology Society (EMBC), 2014 36th Annual International Conference of the IEEE, 974-977.
  19. Schaaff, K., & Schultz, T. (2009). Towards Emotion Recognition from Electroencephalographic Signals. In 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, 1-6.
  20. Wang, D., & Shang, Y. (2013). Modeling Physiological Data with Deep Belief Networks. International of Journal Information and Education Technology (IJIET), 3(5), 505-511.