Black, white or asian, we humans can tell the differences between them very easily.
From race to gender and ethnicity, we won't have trouble in understanding their facial expression or emotions, because after all, those traits are somehow universal from one person to another.
But for facial recognition AI technology, things can be different.
While we humans have progressed in creating better and better AIs, capable of interpreting facial expressions in outstanding ways, but a research found that facial recognition technologies somehow struggle to interpret emotions on black people's faces.
This finding isn't the first time that facial recognition technologies have shown bias to skin color and gender, like for example back in the days when Google labeled black faces as gorillas or identifying asians as blinking.
But in a study by Lauren Rhue, a PhD from Wake Forest University - Schools of Business, it was concluded that emotional analysis technology assigns more negative emotions to black men’s faces than white men’s faces.
She found that facial recognition programs do struggle to correctly identify emotions on people with darker skin colors.
To examine the bias in facial recognition systems that analyze people’s emotions, Rhue used a data set of 400 NBA player photos from the 2016 to 2017 season.
The reason is because the players use similar clothing and attributes, all male, all physically athletic in appearance, and are roughly the same age. Also because the photos are for professional portraits, in which all of the players look at the camera in the picture.
"All players face the camera head-on, a preferable position for accurate facial analysis," said Rhue.
But after running the images through two well-known types of emotional recognition software, she found that the AIs assigned black players more negative emotional scores on average, no matter how much they smiled.
For example, consider the official NBA pictures of Darren Collison and Gordon Hayward, where both players are smiling. According to the facial recognition and analysis program Face++, Darren Collison and Gordon Hayward have similar smile scores: 48.7 and 48.1 out of 100, respectively.
However, Face++ rates Hayward’s expression as 59.7 percent happy and 0.13 percent angry, while Collison’s expression is 39.2 percent happy and 27 percent angry.
Collison is viewed as nearly as angry as he is happy and far angrier than Hayward, that despite the facial recognition program itself recognizes both players as smiling.
In contrast, Microsoft’s Face API viewed both NBA players as happy.
But still, the AI considered Collison as less happy than Hayward, with 98 and 93 percent happiness scores, respectively. Even with his smile, Collison is scored with a small amount of contempt, whereas Hayward has none.
Face API displayed this type of disparity. Even if black faces are partially smiling, the systems assumed more negative emotions as compared to their white counterparts with similar expressions. The average emotional scores were much closer across races, but there were still noticeable differences for black and white faces.
After the AI is fed with more NBA pictures, the pattern somehow remained the same.
But "as the players’ smile widens, the disparity disappears," Rhue continued.
On average, Face++ rates black faces as twice as angry as white faces. Face API scores black faces as three times more contemptuous than white faces.
Even after choosing players with similar smile and facial expression, both facial analysis programs are still more likely to assign negative emotions of anger or contempt to black faces.
The finding aligns with other research, which suggests that people with darker skins must amplify more positive emotions than lighter skin people to receive parity.
Face recognition is a type of analysis that is increasingly used in daily life.
From AIs made by tech companies for smartphones to recognize the user of a device, to cameras used by companies to help them with their hiring decisions, and others created for the government to identify threats to public safety.
Although innovative, AIs can perpetrate and exacerbate existing power dynamics, leading to disparate impact across racial/ethnic groups. Societal accountability is necessary to ensure fairness to all groups is present because facial recognition is just like any other AIs, as it is often invisible to the people most affected by its decisions.