Generative AIs Can Get 'MAD' If They Are Trained With AI-Generated Images, Research Found

Data is all that matters. For AI, it's the thing that makes it smart, and the only thing that can make it even smarter.

As research continues to unearth what generative AIs can and cannot do, it's also revealed that the AIs have the tendency to hallucinate and make up facts from thin air. And not just that, because according to a study, an inherent limitation on Large Language Models have been found.

Researchers found that when AI networks trained on AI-generated outputs (like the text created by ChatGPT or the image output created by a Stable Diffusion), they can go 'MAD.'

This can happen after five training cycles with AI-generated data.

MAD, or short for 'Model Autophagy Disorder', is the acronym used by the Rice and Stanford University researchers involved in the study to describe how AI models, and their output quality, collapses when repeatedly trained on AI-generated data.

Just like how the name suggests, the model that goes MAD essentially "eats itself."

In essence, to make generative AIs go MAD, the researchers trained the LLMs on their own or others' outputs. And here, the researchers found that AIs are extremely dependent on patterns in data.

If they consume AI-generated data, meaning that they're churning the data they've created, the model collapse because as generative models that have to replicate patterns, there is only so much information that can pulled from patterns they have already seen.

As shared by the research team member Nicolas Papernot on Twitter, successive training iterations on LLM-generated data leads the model to gradually lose access to the data they've been trained on previously.

This happened because consecutive outputs become less varied and regress towards the mean.

According to the results, it takes around five of these rounds until the tails of the original distribution disappear.

And when that moment happens, that moment is when the AIs get MAD.

If commenced to extreme measures, results show that they lose the original information from the original data distribution, making outputs no longer relevant.

Results of MAD AIs are mutated outputs.

This happens as soon as the AIs output results that are more aligned with the mean representation of data, much like the snake devouring its own tail.

It's worth noting that not all AIs can go MAD, but the researchers did verify it against LLMs, autoencoders, and Gaussian mixture models.

And this is concerning.

This is because all of those types that can go MAD have been widespread, many of which have become a viable business and popular.

LLMs for example, have been used by many in the industry, and autoencoders that is known for handling things like as popularity prediction, are widely-used on social media platforms' algorithms, image compression, image denoising, and image generation. And as for Gaussian mixture models are used for density estimation, clustering, and image segmentation purposes, which makes them particularly useful for statistical and data sciences.

All of these these types have important roles, and have been employed in both the corporate and public spheres.

The good thing however, the research provides a rare insight into the so-called black box AI.

While this could be an issue for only existing models and applications, the researchers suggest workarounds.

For example, data provenance should be important, and it's becoming more important to be able to separate "original" data from "artificial" data. If you can't identify what data was created by an LLM or a generative image application, you might accidentally include it in training data for your next-generation product.

Another way is to change the models' weightings. By increasing how relevant or how frequent the results at the tails of the distribution are, they will naturally move along the bell curve, closer to the mean. This should make them less prone to "pruning" from the self-generative training:

While these methods may help minimize the 'madness,' there's a responsibility to understand the effects of fine-tuning the models and how those impact the output as well.

"We demonstrate that it has to be taken seriously if we are to sustain the benefits of training from large-scale data scraped from the web," the researchers said on their research paper.

"Indeed, the value of data collected about genuine human interactions with systems will be increasingly valuable in the presence of content generated by LLMs in data crawled from the Internet."

Read: AI Chatbots May Soon 'Run Out Of Text In The Universe' To Learn From