COMPOUND FACIAL EXPRESSION RECOGNITION BASED ON MOBILENET WITH DATA AUGMENTATION

Abstract

Background: Facial expression recognition (FER) has emerged as a crucial tool in human-computer interaction, medical diagnostics, psychological analysis, and robotics. While prior research has focused extensively on basic facial expressions, compound emotions—combinations of two or more basic expressions—remain underexplored. A key limitation in existing datasets is class imbalance and poor organization, which affects training quality and leads to bias in deep learning models. Efficient recognition of both basic and compound expressions demands a balanced, well-structured dataset and advanced model training strategies.

Objective: This study aimed to develop a deep learning-based FER system capable of recognizing both basic and compound facial expressions with improved accuracy, using data augmentation techniques to overcome dataset limitations.

Methods: The RAF-DB dataset, comprising 7 basic and 11 compound emotion classes, was used. Initially, all class images were separated into individual folders to identify and address class imbalance. A Generative Adversarial Network (GAN) was employed to synthesize new samples and balance each class. MobileNet, a lightweight convolutional neural network, was then trained on the augmented dataset. The model was evaluated using accuracy, precision, recall, and F1-score. Training and validation were conducted separately for both expression types.

Results: The proposed model achieved a classification accuracy of 81% for basic facial expressions and 56% for compound facial expressions. Additional evaluation metrics revealed a weighted average precision of 0.39, recall of 0.35, and F1-score of 0.39 for basic expressions, while compound expressions yielded 0.37, 0.41, and 0.39 respectively.

Conclusion: The integration of data augmentation with CNN-based architectures, specifically MobileNet, significantly improved classification accuracy for both expression types. This approach demonstrates a practical and scalable solution for enhancing FER systems in real-world applications.