Journal
SUSTAINABILITY
Volume 14, Issue 22, Pages -Publisher
MDPI
DOI: 10.3390/su142215200
Keywords
facial emotion recognition; fusion of visible light image and thermal facial image; convolutional neural network; robustness to classify in negative emotion
Funding
- Healthcare AI Convergence Research and Development Program through the National IT Industry Promotion Agency of Korea (NIPA) - Ministry of Science and ICT [S0254-22-1001]
- Basic Science Research Program through the National Research Foundation of Korea (NRF) - Ministry of Education [NRF-2017R1A6A1A03015496]
- Ministry of Public Safety & Security (MPSS), Republic of Korea [S0254-22-1001] Funding Source: Korea Institute of Science & Technology Information (KISTI), National Science & Technology Information Service (NTIS)
Ask authors/readers for more resources
This study aims to improve the recognition performance of fear by using visible and thermal images acquired simultaneously. When fear was not recognized in a visible image, we found that replacing it with a thermal image improved emotion recognition performance.
Facial expressions help in understanding the intentions of others as they are an essential means of communication, revealing human emotions. Recently, thermal imaging has been playing a complementary role in emotion recognition and is considered an alternative to overcome the drawbacks of visible imaging. Notably, a relatively severe recognition error of fear among negative emotions frequently occurs in visible imaging. This study aims to improve the recognition performance of fear by using the visible and thermal images acquired simultaneously. When fear was not recognized in a visible image, we analyzed the causes of misrecognition. We thus found the condition of replacing the image with a thermal image. It improved emotion recognition performance by 4.54% on average, compared to the performance of using only visible images. Finally, we confirmed that the thermal image effectively compensated for the visible image's shortcomings.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available