Journal of Platform Technology

  • 제6권4호
  • /
  • Pages.34-40
  • /
  • 2018
  • /
  • 2289-0181(pISSN)
  • /
  • 2289-019X(eISSN)
ICT플랫폼학회 (ICT Platform Society)
권호 표지

인공 신경망의 Catastrophic forgetting 현상 극복을 위한 순차적 반복 학습에 대한 연구

A study on sequential iterative learning for overcoming catastrophic forgetting phenomenon of artificial neural network

  • 투고 : 2018.08.28
  • 심사 : 2018.12.27
  • 발행 : 2018.12.31

⟨ 이전 논문 다음 논문 ⟩

초록

현재 인공신경망은 단일 작업에 대해선 뛰어난 성능을 보이나, 다른 종류의 작업을 학습하면 이전 학습 내용을 잊어버리는 단점이 있다. 이를 catastrophic forgetting이라고 한다. 인공신경망의 활용도를 높이긴 위해선 이 현상을 극복해야 한다. catastrophic forgetting을 극복하기 위한 여러 노력이 있다. 하지만 많은 노력이 있었음에도 완벽하게 catastrophic forgetting을 극복하지는 못하였다. 본 논문에서는 여러 노력 중 elastic weight consolidation(EWC)에 사용되는 핵심 개념을 이용하여, 순차적 반복학습을 제시한다. 인공신경망 학습에 많이 쓰이는 MNIST를 확장한 EMNIST 데이터 셋을 이용하여 catastrophic forgetting 현상을 재현하고 이를 순차적 반복학습을 통해 극복하는 실험을 진행하였으며, 그 결과 모든 작업에 대해서 학습이 가능하였다.

Currently, artificial neural networks perform well for a single task, but NN have the problem of forgetting previous learning by learning other kinds of tasks. This is called catastrophic forgetting. To use of artificial neural networks in general purpose this should be solved. There are many efforts to overcome catastrophic forgetting. However, even though there was a lot of effort, it did not completely overcome the catastrophic forgetting. In this paper, we propose sequential iterative learning using core concepts used in elastic weight consolidation (EWC). The experiment was performed to reproduce catastrophic forgetting phenomenon using EMNIST data set which extended MNIST, which is widely used for artificial neural network learning, and overcome it through sequential iterative learning.

키워드

과제정보

연구 과제 주관 기관 : 한국연구재단, 정보통신기술진흥센터

참고문헌

  1. Shane Legg and Marcus Huntter. Universal intelligence : A definition of machine intelligence. Mind and Machines 17(4): 391-444, 2007
  2. Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. The psychology of learning and motivation, 24(109-165):92, 1989
  3. Ronald Kemker, Angelina Abitino, Marc McClure, and Christopher Kanan. Measuring Catastrophic Forgetting in Neural Networks. arXiv preprint arXiv:1708.02072, 2017.
  4. Kirkpatrick, J.; Pascanu, R.; Rabinowitz, N.; Veness, J.;et al. 2017. Overcoming catastrophic forgetting in neural networks. Proc. of the National Academy of Sciences 201611835.
  5. Seongho Son, Jiseob Kim, Byoing-Tak Zhang "Sequential Multitask Learning Optimization Using Bayesian Neural Network" KIISE Transactions on Computing practices. Vol. 24. No. 5. Pp. 251-255. 2018. 5
  6. Y. LeCun and Y. Bengio Convolutional Networks for Images Speech, and Time-Series, brain theory neural networks, vol. 3361, 1995.
  7. Cohen, G., Afshar, S., Tapson, J., & van Schaik, A. (2017). EMNIST: an extension of MNIST to handwritten letters. Retrieved from http://arxiv.org/abs/1702.05373
  8. Babyak MA. What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models. Psychosom Med. 2004;66(3):411-421