Not limited to creating something astonishing and never seen before, AI can also improvise existing technologies, as researchers from SFU's School of Interactive Arts and Technology (SIAT) have demonstrated.
Usually, blurring people's faces for anonymity, like in the news, requires specialized software that tracks an area or multi-point areas, which adjust dynamically in shapes and sizes as the object moves. Researchers created an a way to replace this "blurring" technique using AI.
The team aims to use artificial intelligence to keep the anonymity, but making the emotions of the subject pass through, by amplifying them as they are tied to the story.
SIAT professors Steve DiPaola and Kate Hennessy, with Taylor Owen from UBC’s journalism school, have received a Google/Knight Foundation grant to carry out the project.
The team created a method which is specifically designed to update pixelating techniques use in face blurring methods, with AI "painting" approach to anonymization
"Our goal is to create a working technique that would be much better at conveying emotional and knowledge information than current anonymization techniques," explained DiPaola, an expert in the field of AI/VR facial recognition, head of SFU's Interactive Visualization Lab.
“When artists paint a portrait, they try to convey the subject’s outer and inner resemblance,” continued DiPaola. "With our AI, which learns from more than 1,000 years of artistic technique, we have taught the system to lower the outer resemblance and keep as high as possible the subject’s inner resemblance – in other words, what they are conveying and how they are feeling."
"Our system uses five levels of AI processing to simulate a smart painter, using art abstraction to repaint the video as if an artist was painting every frame. The result is more engaging, especially since not everyone listens to stories – so the art component becomes more relevant."
The AI here can see parts of the subject's face to resize the eyes and the thickness of the face. After altering the appearance, the AI "paints' the subject in several steps, which makes the looks more Van Gogh-like with visible planes, resembling the texture of brush strokes and palette knife of the famous painter.
Then the AI looks at the subject's facial features and his/her voice tone, and match them with the altered appearance to show emotions using colors and tones, rather than using his/her eyebrows or fierceness of the face which should be anonymized.
The system doesn’t change the pixels of the video frames, but instead produces the painting-style result on every frame, DiPaola notes.
"It’s actually an open and dynamic process that allows levels of control throughout. We eventually want to the subject or producer to be able to customize the final result based on their needs."
DiPaola says the AI's effectiveness at anonymizing should show better results in 360 or VR. The team plans to implement the technology to work in a variety of projects.