Why emotion recognition?
Have you ever thought about how emotional technologies may affect us in our daily lives? The emergence of emotion recognition products and software is particularly interesting. The film “Her” (2013) directed by Spike Jonze illustrates the pleasant effects of an artificial intelligence (AI) that is emotionally intelligent and responsive, while the protagonist falling in love with the AI problematises the potential downfalls of an emerging dependency of the protagonist (brilliantly played by Joaquin Phoenix) on the AI. But what happens to our human relationships when there is the potential for machines to be so much more in tune with our emotional experiences compared to our human interactions?
Image: Film “Her” Beach Scene
There are already a large variety of companies offering emotion recognition software either based on facial recognition, voice sentiment analysis or biometric sensors. While not all companies are open about the AI behind their emotion recognition software, some companies do publish this information. For example, Affectiva uses the emotion model by Ekman and Friesen (1971). Ekman argued that six basic emotions exist that are universally recognisable and therefore work across countries and cultures.
Image: Ekman’s Basic Emotions / Affectiva
But, wait – are we only ever experiencing only six emotions? What about different emotions occurring at the same time? Can only one emotion be felt at any one time? How about fake emotions that are expressed because these are what is culturally expected, but are not necessarily what is genuinely felt? Why is there no updated emotion model (yet) forthcoming? This is particularly concerning given that emotion recognition software is fast becoming ubiquitous. Emotional AI is used in HR, where voice sentiment analysis is supposed to give insights into the job applicant’s emotional state, for example giving predictions about anything from the applicant’s confidence to whether the applicant is a good team player. But what happens if these predictions are incorrect? If the applicant has no way of knowing how the company makes their decisions?
What is the research about?
To address some of these questions, my research critically explores what role (positive or negative) emotion recognition products and software could play in our lives. For this purpose, I designed an emotion recognition device, which uses biometric sensors (pulse & galvanic skin response) to recognise the emotional state of the wearer and the device shows suggestions on what to do based on what the wearer is feeling. The emotions and suggestions are shown on the surface of the device as colour-changing LED lights. But the device has an interesting twist – unlike commercial devices, this one has a few tricks up its sleeves. This was inspired by my reading of critical and speculative design.
The purpose of wearing the device as part of the study is to find out what happens when emotions suddenly become public, how our social interactions and conversations may change when our feelings are openly displayed, or how our emotional experiences may change when the device shows how we feel. Could we use emotion recognition to develop emotional awareness and to better regulate emotions? Or do we trust machines over our own internal feelings and thereby disrupt the connection to our own authentic self and our intuition?
Image: Design Kit, including emotion recognition device, cards and diary.
Why should anyone participate?
It is evident that emotion recognition technology is becoming ever more significant as they are used in an increasing range of products to predict and adapt the experience depending on how we feel. Feel tired? AI will automatically dim the lights in your bedroom. Feel a bit disengaged and bored? AI will show only films on streaming platforms that are up-lifting. Feel slightly stressed and overwhelmed? What about the AI on newspaper apps showing only happy news items. These are not future scenarios, but happening already, so it is important to think about the world we want to live in before it happens.
Finally, if you would like to participate in the study, please contact me at Kingston University, School of Art: K1544369@kingston.ac.uk
Intelligent Emotions 