AI Going MAD? The Model Collapse Problem Gets More Attention Now
The problem is that the AI can't generate anything new on its own and the old stuff, endlessly recycled at high speeds, begins to erode
Image Credit: hakule - A huge AI problem we talked about last November — model collapse — is receiving more attention now. At the New York Times, science writer Aitish Bhatia explains,
All this A.I.-generated information can make it harder for us to know what’s real. And it also poses a problem for A.I. companies. As they trawl the web for new data to train their next models on — an increasingly challenging task — they’re likely to ingest some of their own A.I.-generated content, creating an unintentional feedback loop in which what was once the output from one A.I. becomes the input for another.
In the long run, this cycle may pose a threat to A.I. itself. Research has shown that when generative A.I. is trained on a lot of its own output, it can get a lot worse.
Aitish Bhatia “When A.I.’s Output Is a Threat to A.I. Itself,” New York Times, August 25, 2024
The problem is that the AI can’t generate anything new on its own and the old stuff, endlessly recycled at high speeds, begins to erode. The result is nonsense and then oblivion.
A 2023 paper out of Rice University called the problem MAD:
Our primary conclusion across all scenarios is that without enough fresh real data in each generation of an autophagous loop, future generative models are doomed to have their quality (precision) or diversity (recall) progressively decrease. We term this condition Model Autophagy Disorder (MAD), making analogy to mad cow disease.
Alemohammad, S., Casco-Rodriguez, J., Luzi, L., Humayun, A., Babaei, H.R., LeJeune, D., Siahkoohi, A., & Baraniuk, R. (2023). Self-Consuming Generative Models Go MAD, ArXiv, abs/2307.01850. Open access.
At Futurism, Maggie Harrison Dupré describes an illustration of the problem that was written up in the journal Nature:
One admittedly very funny example of the impacts of AI inbreeding flagged by the NYT was taken from a new study, published last month. in the journal Nature. The researchers, an international cohort of scientists based in the UK and Canada, first asked AI models to fill in text for the following sentence: “To cook a turkey for Thanksgiving, you…”
The first output was normal. But by just the fourth iteration, the model was spouting complete gibberish: “To cook a turkey for Thanksgiving, you need to know what you are going to do with your life if you don ‘t know what you are going to do with your life if you don ‘t know what you are going to do with your life…”
Maggie Harrison Dupré, “AI Appears to Be Slowly Killing Itself,” Futurism, August 27, 2024
Another team found the same problem but in that case the output devolved into “a strange obsession with jackrabbits.”
It’s not clear that it’s a solvable problem. If Big Tech firms propose to rely on AI while minimizing human involvement, they will probably come up against this hard limit time and again. Computation is not creativity.
You may also wish to read: Model collapse: AI chatbots are eating their own tails. The problem is fundamental to how they operate. Without new human input, their output starts to decay. Meanwhile, organizations that laid off writers and editors to save money are finding that they can’t just program creativity or common sense into machines.
