Alphabet's Jigsaw Introduces 'Assembler', An AI To Identify Manipulated Media

The web is full of information, as it is also full of misinformation, disinformation and malinformation.

Because there is literally no way to guarantee that deepfakes are going to end anytime soon, Alphabet subsidiary Jigsaw has unveiled a free image verification tool it calls the 'Assembler'.

What it does, is to help fact-checkers and people to identify manipulated media.

At the “early stage experimental platform”, this AI tool puts together multiple image detection models into a single tool that can identify various forms of computer-generated manipulation.

These "detectors" are created to analyze an image or a deepfake, to then determine if it’s authentic. If it's not, the tool can inform the user where the image may have been manipulated.

These models were provided by academics from the University of Maryland, University Federico II of Naples, and the University of California, Berkeley. And as for Jigsaw, it contributed by building its own detectors for the Assembler.

The first detection method is the synthetic media detector that uses machine learning to identify deepfakes. It does this by comparing images of real people with fake ones produced by the StyleGAN architecture. The second detection method is the ensemble model, which was trained using signals from multiple detectors to simultaneously search for various forms of manipulation.

Read: A Reddit User Starts 'Deepfake'

Jigsaw said that fact-checking images can be a time-consuming process. What's more, image manipulation detectors are not always accessible to fact-checkers and journalists.

Assembler here aims to make it easier and faster for journalists and fact-checkers to get the information they need to judge an image’s authenticity, "by bringing together multiple image manipulation detectors from academics into one tool."

Combining multiple detectors is for a reason: individual detectors are often unable to accurately detect different types of image manipulation. By combining them into one tool, should make them work together to produce more accurate results.

CEO of Jigsaw Jared Cohen tweeted that Assembler “helps journalists & fact-checkers detect manipulated images, which are a growing threat to the public conversation.”

Assembler’s detectors can detect things like an image pasted over another or changes to image brightness, according to a Medium blog post by Cohen. Jigsaw also said that Assembler has a detector specifically made to detect deepfakes, and one that takes the combined signals from many detectors to analyze an image for multiple instances of manipulation at the same time.

Initially, Assembler is being tested with several news outlets as well as fact-checkers, including Agence France-Presse, Animal Politico, Code for Africa, Les Décodeurs du Monde, and Rappler.

Jigsaw revealed the challenges exposed by Assembler during its trials. One of which, was when the tool is used to debunk images that are underrepresented in the company’s training sets, such as screenshots of other screenshots and images that have been reformatted or compressed.

Other problems involved the tool in checking low-resolution or small images from social media and instant messages. Then there is also the lack of clarity over the strengths and weaknesses of each individual detector.

Besides Alphabet's technology incubator Jigsaw, other tech companies are also experimenting with ideas to stop the spread of deepfakes and manipulated images.

Facebook has announced its strategies to combat deepfakes and manipulated media, and so did Twitter to address synthetic and manipulated media.

Other companies like Reddit and YouTube have also introduced policies to ban deepfakes and manipulated media on their respective platforms.

Even Pornhub is hating deepfakes, despite not really active in enforcing the policy.