FACIAL EMOTION RECOGNITION USING CONVOLUTIONAL NEURAL NETWORK BASED ON THE VISUAL GEOMETRY GROUP-19
(1) Computer Science Study Program, BINUS University, Jakarta, DKI Jakarta
(2) Computer Science Study Program, BINUS University, Jakarta, DKI Jakarta
Corresponding Author
Abstract
Facial recognition is one of many popular and difficult tasks in computer vision. A variety of research have been conducted on this subject, each of which suggests a stand-alone approach. While many studies strive for more accuracy, this study research aims to increase the efficiency of human computers by classifying emotions based on human faces using a self-based neural network. The usage of a Convolutional Neural Network (CNN) based on the Visual Geometry Group - 19 (VGG-19) classification model, which has been employed in ImageNet data sets and improved for emotion classification, is proposed in this paper. The classification process was conducted using FER-2013 dataset, which consists of over 35,000 facial images captured in various settings and contains 7 different emotions. The dataset was divided into three subsets, with 80% allocated for training, 10% for validation, and 10% for testing. With an accuracy of 71.80%, the proposed technique surpasses most self-based models.
Keywords
References
I.J. Goodfellow, D. Erhan, P. Luc Carrier, A. Courville, M. Mirza, B. Hamner, W. Cukierski, Y. Tang, D. Thaler, D.-H. Lee, Y. Zhou, C. Ramaiah, F. Feng, R. Li, X. Wang, D. Athanasakis, J. Shawe-Taylor, M. Milakov, J. Park, R. Ionescu, M. Popescu, C. Grozea, J. Bergstra, J. Xie, L. Romaszko, B. Xu, Z. Chuang, Y. Bengio, Challenges in representation learning: A report on three machine learning contests, Neural Netw. 64 (2015) 59–63. https://doi.org/10.1016/j.neunet.2014.09.005.
A. Dhall, R. Goecke, J. Joshi, M. Wagner, T. Gedeon, Emotion recognition in the wild challenge 2013, in: Proc. 15th ACM Int. Conf. Multimodal Interact., ACM, Sydney Australia, 2013: pp. 509–516. https://doi.org/10.1145/2522848.2531739.
A. Dhall, R. Goecke, J. Joshi, K. Sikka, T. Gedeon, Emotion Recognition In The Wild Challenge 2014: Baseline, Data and Protocol, in: Proc. 16th Int. Conf. Multimodal Interact., Association for Computing Machinery, New York, NY, USA, 2014: pp. 461–466. https://doi.org/10.1145/2663204.2666275.
A. Dhall, O.V. Ramana Murthy, R. Goecke, J. Joshi, T. Gedeon, Video and Image based Emotion Recognition Challenges in the Wild: EmotiW 2015, in: Proc. 2015 ACM Int. Conf. Multimodal Interact., Association for Computing Machinery, New York, NY, USA, 2015: pp. 423–426. https://doi.org/10.1145/2818346.2829994.
A. Dhall, R. Goecke, J. Joshi, T. Gedeon, Emotion recognition in the wild challenge 2016, in: Proc. 18th ACM Int. Conf. Multimodal Interact., Association for Computing Machinery, New York, NY, USA, 2016: pp. 587–588. https://doi.org/10.1145/2993148.3007626.
A. Dhall, R. Goecke, S. Ghosh, J. Joshi, J. Hoey, T. Gedeon, From individual to group-level emotion recognition: EmotiW 5.0, in: Proc. 19th ACM Int. Conf. Multimodal Interact., Association for Computing Machinery, New York, NY, USA, 2017: pp. 524–528. https://doi.org/10.1145/3136755.3143004.
A. Dhall, A. Kaur, R. Goecke, T. Gedeon, EmotiW 2018: Audio-Video, Student Engagement and Group-Level Affect Prediction, (2018). https://doi.org/10.48550/arXiv.1808.07773.
A. Dhall, Goecke, Roland, Ghosh, Shreya, Gedeon, Tom, EmotiW 2019: Automatic Emotion, Engagement and Cohesion Prediction Tasks, in: 2019 Int. Conf. Multimodal Interact., Association for Computing Machinery, New York, NY, USA, 2019: pp. 546–550. https://doi.org/10.1145/3340555.3355710.
A. Dhall, R. Goecke, S. Lucey, T. Gedeon, Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark, in: 2011 IEEE Int. Conf. Comput. Vis. Workshop ICCV Workshop, IEEE, Barcelona, Spain, 2011: pp. 2106–2112. https://doi.org/10.1109/ICCVW.2011.6130508.
T. Kanade, J.F. Cohn, Yingli Tian, Comprehensive database for facial expression analysis, in: Proc. Fourth IEEE Int. Conf. Autom. Face Gesture Recognit. Cat No PR00580, IEEE Comput. Soc, Grenoble, France, 2000: pp. 46–53. https://doi.org/10.1109/AFGR.2000.840611.
P. Lucey, J.F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, I. Matthews, The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression, in: 2010 IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. - Workshop, IEEE, San Francisco, CA, USA, 2010: pp. 94–101. https://doi.org/10.1109/CVPRW.2010.5543262.
M. Lyons, S. Akamatsu, M. Kamachi, J. Gyoba, Coding facial expressions with Gabor wavelets, in: Proc. Third IEEE Int. Conf. Autom. Face Gesture Recognit., IEEE Comput. Soc, Nara, Japan, 1998: pp. 200–205. https://doi.org/10.1109/AFGR.1998.670949.
Z. Zhang, P. Luo, C.C. Loy, X. Tang, From Facial Expression Recognition to Interpersonal Relation Prediction, Int. J. Comput. Vis. 126 (2018) 550–569. https://doi.org/10.1007/s11263-017-1055-1.
B. Ko, A Brief Review of Facial Emotion Recognition Based on Visual Information, Sensors. 18 (2018) 401. https://doi.org/10.3390/s18020401.
S. Li, W. Deng, Deep Facial Expression Recognition: A Survey, IEEE Trans. Affect. Comput. 13 (2022) 1195–1215. https://doi.org/10.1109/TAFFC.2020.2981446.
E. Owusu, E.K. Gavua, Z. Yong-Zhao, Facial Expression Recognition – A Comprehensive Review, Int. J. Technol. Manag. Res. 1 (2020) 29–46. https://doi.org/10.47127/ijtmr.v1i4.36.
Samadiani, Huang, Cai, Luo, Chi, Xiang, He, A Review on Automatic Facial Expression Recognition Systems Assisted by Multimodal Sensor Data, Sensors. 19 (2019) 1863. https://doi.org/10.3390/s19081863.
H. Jung, S. Lee, S. Park, B. Kim, J. Kim, I. Lee, C. Ahn, Development of deep learning-based facial expression recognition system, in: 2015 21st Korea-Jpn. Jt. Workshop Front. Comput. Vis. FCV, IEEE, Mokpo, South Korea, 2015: pp. 1–4. https://doi.org/10.1109/FCV.2015.7103729.
C. Pramerdorfer, M. Kampel, Facial Expression Recognition using Convolutional Neural Networks: State of the Art, (2016). http://arxiv.org/abs/1612.02903 (accessed February 16, 2023).
J. Yang, G. Yang, Modified Convolutional Neural Network Based on Dropout and the Stochastic Gradient Descent Optimizer, Algorithms. 11 (2018) 28. https://doi.org/10.3390/a11030028.
T.S. Gunawan, A. Ashraf, B.S. Riza, E.V. Haryanto, R. Rosnelly, M. Kartiwi, Z. Janin, Development of video-based emotion recognition using deep learning with Google Colab, TELKOMNIKA Telecommun. Comput. Electron. Control. 18 (2020) 2463. https://doi.org/10.12928/telkomnika.v18i5.16717.
E. Pranav, S. Kamal, C. Satheesh Chandran, M.H. Supriya, Facial Emotion Recognition Using Deep Convolutional Neural Network, in: 2020 6th Int. Conf. Adv. Comput. Commun. Syst. ICACCS, IEEE, Coimbatore, India, 2020: pp. 317–320. https://doi.org/10.1109/ICACCS48705.2020.9074302.
S. Cheng, G. Zhou, Facial Expression Recognition Method Based on Improved VGG Convolutional Neural Network, Int. J. Pattern Recognit. Artif. Intell. 34 (2020) 2056003. https://doi.org/10.1142/S0218001420560030.
A. Sajjanhar, Z. Wu, Q. Wen, Deep Learning Models for Facial Expression Recognition, in: 2018 Digit. Image Comput. Tech. Appl. DICTA, IEEE, Canberra, Australia, 2018: pp. 1–6. https://doi.org/10.1109/DICTA.2018.8615843.
Y. Miyakoshi, S. Kato, Facial emotion detection considering partial occlusion of face using Bayesian network, in: 2011 IEEE Symp. Comput. Inform., IEEE, Kuala Lumpur, Malaysia, 2011: pp. 96–101. https://doi.org/10.1109/ISCI.2011.5958891.
H.K. Vydana, P.P. Kumar, K.S.R. Krishna, A.K. Vuppala, Improved emotion recognition using GMM-UBMs, in: 2015 Int. Conf. Signal Process. Commun. Eng. Syst., IEEE, Guntur, India, 2015: pp. 53–57. https://doi.org/10.1109/SPACES.2015.7058214.
B. Schuller, G. Rigoll, M. Lang, Hidden Markov model-based speech emotion recognition, in: 2003 IEEE Int. Conf. Acoust. Speech Signal Process. 2003 Proc. ICASSP 03, IEEE, Hong Kong, China, 2003: p. II-1–4. https://doi.org/10.1109/ICASSP.2003.1202279.
P. Singh, G. Saha, M. Sahidullah, Non-linear frequency warping using constant-Q transformation for speech emotion recognition, in: 2021 Int. Conf. Comput. Commun. Inform. ICCCI, IEEE, Coimbatore, India, 2021: pp. 1–6. https://doi.org/10.1109/ICCCI50826.2021.9402569.
K. Simonyan, A. Zisserman, Very Deep Convolutional Networks for Large-Scale Image Recognition, (2014). https://doi.org/10.48550/ARXIV.1409.1556.
Article Metrics
Abstract View
: 608 times
Download : 308 times
DOI: 10.56327/jurnaltam.v14i1.1475
Refbacks
- There are currently no refbacks.
