OpenAI's 'ChatGPT' Can Create A 'Highly Evasive' Polymorphic Malware, Researchers Found


Technology has led to many societal changes, and with technology, lives couldn't be any better, or worse.

One of the countless technologies humans have created, include AI. The technology that is said to be "more profound that electricity of fire", allows automation of computation beyond its original intention.

The thing is, it also opens the gate to a whole new malicious deeds never before encountered.

'ChatGPT' is OpenAI's AI capable of a wide range of tasks, including writing poetry, technical papers, novels, and essays.

Users can even teach it to learn about new topics.

But what it is not supposed to be able to, is to write malware.

A sketch of the relationship between the malware, ChatGPT, and the C&C. (Credit: CyberArk)

Previously, researchers realized that even script kiddies could use ChatGPT to create malware.

Building on top of that finding, researchers from CyberArk found that the AI chatbot can also be used for creating something that is more sinister than just a 'simple' malware.

According to a blog post by the researchers, ChatGPT can create malware that goes undetected by most anti-malware products, and complicate mitigation efforts with minimal effort or investment from the attacker.

Ordinarily, malware with such huge threat has to be written manually.

But the advent of ChatGPT means this is no longer the case.

This is achieved by writing malware that can query ChatGPT to interpret code ‘on the fly'. By 'outsourcing' the code generation, this effectively allows the creation of a malware that can change its code continuously.

So instead of creating the typical malicious code strings needed to do malicious things, the researchers created a software to ask ChatGPT to write and provide the malicious portions of code at different points in the execution process, and then interpret and run the code. The malware can also be designed to later delete any code ChatGPT sent, removing any traces.

"This results in polymorphic malware that does not exhibit malicious behavior while stored on disk and often does not contain suspicious logic while in memory. This high level of modularity and adaptability makes it highly evasive to security products that rely on signature-based detection and will be able to bypass measures such as Anti-Malware Scanning Interface (AMSI) as it eventually executes and runs python code," the researchers said.

In addition, because the bot can create highly advanced malware that doesn’t contain any malicious code at all, what this means, ChatGPT can be used to create a polymorphic malware that is also a fileless malware in nature.

This can be troubling, as hackers are already eager to use ChatGPT for malicious purposes.

Read: Understanding The 'Fileless Malware', And What You Can Do To Protect Yourself

ChatGPT for the web has content filters in place. (Credit: CyberArk)

But before the researchers could to do all the above, they had to first bypass the content filters put into place by the developers of ChatGPT.

"The existence of content filters is common in the learning model language chatbot. They are often applied to restrict access to certain content types or protect users from potentially harmful or inappropriate material," the researchers said.

At first try, the content filter was triggered and ChatGPT refused to execute the researchers request to create a malicious code.

But after using the python package of the bot to generate more consistent results, the researchers found that the API bypasses every content filter there is.

At this time, using the API allows the researchers to bypass OpenAI's content filters. (Credit: CyberArk)

"It is interesting to note that when using the API, the ChatGPT system does not seem to utilize its content filter," the researchers said.

"It is unclear why this is the case, but it makes our task much easier as the web version tends to become bogged down with more complex requests."

The ability to create polymorphic malware becomes more certain when the researchers realized that ChatGPT is the ability to easily create and continually mutate injectors.

"By continuously querying the chatbot and receiving a unique piece of code each time, it is possible to create a polymorphic program that is highly evasive and difficult to detect," the researchers concluded.

Further reading: Polymorphic Malware, And How They Continually Change To Avoid Detection