Despite the huge number of publicly available images, emotion recognition for real applications suffers from a lack of relevant images. This challenge is considered here, where the task involves interpreting the facial expressions of patients with severe impairments who are under cognitive and physical rehabilitation in a specialized center.
The authors built a small dataset of facial expressions, involving the patients in engaging activities, with the aim to cover the six standard emotion expressions. According to the literature, they adopted a convolutional neural network (CNN), pre-trained on a large number of images from healthy people, to extract features and then fine-tune them with patient data. This transfer learning overcomes the limitations of learning from small datasets.
Since the end goal is to provide healthcare personnel with a better understanding of their patients’ emotions, and to then use this knowledge to tailor rehabilitation activities, an automatic system based on the Pepper robot was tested. The robot uses audio, visual, and gestures to motivate patients when they exhibit negative emotions; moreover, data collected from the robot gives feedback to the operators.
The paper has many technical details and considerations, both about the images and the learning methods, which can be of interest in reproducing the results or extending them to other cases.
According to the authors, the salient results show improvement with regards to the accuracy of emotion recognition and provide preliminary indications that a robot in the rehabilitation process can make it more effective. Some questions, however, remain. What about the costs of such a system? What about the risks of automatizing human-to-human interactions? Finally, what are the opinions of the patients? These and many more questions arise from reading the paper.