Messaging apps are essentially mediums for communication. But things can go beyond that sometimes.
Some messaging apps allow third-party developers to create software that can run on top of them. This allows the messaging app to improve the experiences of users.
Among the software that can run on top of messaging apps, include bots.
And here, researchers from a security firm called Sensity (formerly known as Deeptrace), said that they discovered a network of deepfake bots on Telegram.
The bots lurk to create computer-generated free naked images of women on request. Users can also pay using “premium coins”, so the bot will remove the watermark on the deepfake imageries, and speed up the processing.
As of July 2020, the firm found that the network of bots have generated deepfakes of at least 104,000 women.
And in the last three months, its output has increased by nearly 200%.
According to Sensity's report:
"Compared to similar underground tools, the bot dramatically increases accessibility by providing a free and simple user interface that functions on smartphones as well as traditional computers. To 'strip' an image, users simply upload a photo of a target to the bot and receive the processed image after a short generation process."
What started out from Reddit, deepfake technology has been notorious for swapping celebrities' faces and superimposing them on porn stars' bodies. Deepfake was also an issue when many used it to create revenge porn, or explicit imagery of someone without their consent.
As with the bots on Telegram, the bots are mostly doing the latter.
A lot of Telegram users are using the bots to create naked photos of women they know in real life. It's presumed that victims of the Telegram deepfake bots don't know that fake naked images of them are floating around on the internet.
Making things worse, Sensity found a “limited number” of the photos feature underage children.
In the report, Sensity said that 70% of the targets were "private individuals whose photos are either taken from social media or private material."
Sensity said that the bots and their affiliated channels, have attracted more than 100,000 members from around the world, with 70% of them coming from Russia and former USSR countries.
In a poll Sensity found in one of the public Telegram communities dedicated to the bot, the firm found that the deepfake bot has yet to spread in English-speaking countries.
It was said that the deepfake bots were heavily marketed through Russian social media platform VK, which have presence in at least 380 pages.
According to its research, the software underpinning the bot’s capabilities is likely the open-source version of DeepNude, the machine learning algorithm that previously gained notoriety in 2019 when its developers briefly sold it online.
After being kicked out of GitHub, the software has made its way to other places, including torrent websites, other developers can use it to create another instance of deepfake algorithm.
What makes this deepfake algorithm unique is that, it doesn't work like a typical deepfake algorithm.
Usually, deepfake requires a PC with dedicated GPU. But this Telegram bot works entirely on Telegram's ecosystem. The bot runs on Telegram's online servers, allowing anyone with a phone and a Telegram app to access the software.
"Our 2019 State of Deepfakes report found that 96% of deepfake videos online are pornographic, and that the vast majority of these videos targeted high profile celebrities. The tools used to create these 'faceswap' deepfake videos still require moderate programming skills and a GPU to operate, making them significantly less accessible than the bot identified in this report," the researchers said.
Read: 96 Percent Of All Deepfakes On The Web Are Porn, Report Said
That is why the bot is seeing so many users (customers) in such a brief of time.
This also also underscores how difficult it will be to contain the software.
It should be noted that deepfakes are more widely-known as AI-manipulated videos. In this Telegram bot case, the deepfakes only works on images. But still, the results can be damaging, considering how easy anyone can access and use it.
What this means, it's reasonable to assume that the generated deepfake pornography can also be used for the purposes of public shaming or extortion-based attacks.
Sensity provided a diagram to detail how these attacks could be orchestrated.
Each scenario involves an malicious person obtaining images of a target from public social media galleries, or private photos exchanged on messaging platforms or dating apps. After “stripping” the target’s image using the Telegram deepfake bot, the attacker can deploy it to publicly shame the target, by sharing the image openly on social media networks, or sending it privately to the target’s relatives, friends, and acquaintances.
Alternatively, the attacker can also extort money or anything they wish from the target, by threatening the publication of the “stripped” image to the internet.
Also, since the personal information of targets can also be obtained, considering that a high percentage of the bot users personally know their targets, attackers could also also sell the sensitive information to other malicious actors on underground forums and marketplaces, earning money, reputation, and also creating even more damage to the victims.
Further reading: Deepfakes And Deepfake Detectors: The War Of Concealing And Revealing Faces