Do machines understand human emotion? No, computers don't feel or share your joy and empathy. But that can change.
Machine learning systems are known for their ability to understand the content of images and recognize objects. Here AIs can also detect faces and movements with astonishing accuracy. Using Animoji on iPhones for example, the AI can correctly mimic the eyes, movement and others to recreate emotions.
But that kind of AI still can't predict how users actually feel.
And this is where 'EmoNet' comes in.
What it's capable of, is taking a glance of an image to tell what emotions the AI sees in it.
Created by researchers from the University of Colorado and Duke University, the neural network is capable of classifying images into 11 different emotional categories.
To develop this neural network, the team experimented by re-purposing an existing convolutional neural network architecture, called AlexNet, which was originally designed for object recognition.
Based on prior research on stereotypical emotional responses to images, the researchers tweaked AlexNet to be able to identify emotional situations rather than objects, and capable of predicting feelings that may be shown on a person.
"A lot of people assume that humans evaluate their environment in a certain way and emotions follow from specific, ancestrally older brain systems like the limbic system," explained Philip Kragel, a postdoctoral research associate at the Institute of Cognitive Science, the lead author of the research project.
"We found that the visual cortex itself also plays an important role in the processing and perception of emotion."
Realizing that, the team trained the AI using a large data set that contains 2,187 videos that were already classified into 27 distinct emotion categories including anxiety, surprise, and sadness.
The team extracted 137,482 frames from these videos, and then excluded sets of a particular emotion that had less than 1,000 samples. After the training, the team used 25,000 images of pretty much everything, "from erotic photos to nature scenes", to validate their results.
The AI was then trained to categorize the images “into 20 categories such as craving, sexual desire, horror, awe, and surprise”.
And the result here is surprising, as the AI can accurately and consistently, categorize 11 of the 20 emotion types it was asked to identify, although it fared better with some categories than others.
For instance, EmoNet “identified photos that evoke craving or sexual desire with more than 95 percent accuracy. But it had a harder time with more nuanced emotions like confusion, awe, and surprise."
According to the researchers, emotions such as joy, amusement, and adoration can confuse the AI model because they have similar facial features/expressions.
Going a step further, the researchers asked 18 people and measured their brain activity using functional magnetic resonance imaging (fMRI) machine, while showing them 4-second flashes of 112 images. Here, the team found that the visual cortex itself also plays an important role in the processing and perception of emotion.
EmoNet was tested with the same images, essentially serving as the 19th subject.
When activity in the neural network was compared to that in the human subjects' brain, the patterns matched up.
"We found a correspondence between patterns of brain activity in the occipital lobe and units in EmoNet that code for specific emotions. This means that EmoNet learned to represent emotions in a way that is biologically plausible, even though we did not explicitly train it to do so," said Kragel.
They used the information to improve the AI.
Decoding human emotions is just one of the many ways researchers explore the potential of AI technologies.
With this findings, the researchers see EmoNet as an essential step that would create a way for future researchers in creating human brain-inspired neural networks.
Ultimately, this kind of neural networks could be used to help people digitally screen out negative images or find positive ones. It could also be used to improve computer-human interactions and help advance emotion research.
And this can include and not limited to: social robots emotional chatbots, to healthcare and entertainment.
So no more of that thought that only living organisms can understand emotions, as computers too, powered by AI, can be taught to understand your joy and sorrow.
"Machine learning technology is getting really good at recognizing the content of images - of deciphering what kind of object it is," said senior author Tor Wager, who worked on the study while a professor of psychology and neuroscience at Colorado University Boulder, to ScienceDaily.
"We wanted to ask: Could it do the same with emotions? The answer is yes."