'Poison AI' movement tries to corrupt ChatGPT and Gemini with deliberately bad data

Full text

AI has acquired an unusual new enemy. A growing AI data poisoning online movement wants to attack the models themselves. The goal is simple enough on paper — feed future AI systems bad information, misleading data, or deliberately corrupted material until they become less useful.

If future versions of ChatGPT, Gemini and other AI models learn from enough misleading, corrupted or intentionally manipulated material, perhaps those systems will become less reliable. Chatbots already confidently repeat nonsense far too often; now imagine it exponentially worse as text and image generators misunderstand every prompt, and the models become too frustrating to trust.

It's not just theory. Data poisoning is an actual area of AI security research. While the argument that making AI systems less reliable will discourage companies from scraping creative work or building ever larger models might entice some, it also risks undermining far more than just the latest trending AI chatbot.

Trying to teach AI all the wrong lessons

Large language models are often described as reading the internet. They absorb enormous collections of books, websites, articles, computer code, images, and documents and learn patterns from them.

Changing enough of that raw material can sometimes change what the finished model learns. Instead of attacking an AI after it has been built, the attacker tries to 'poison' the well of knowledge.

A poisoned model might answer one specific question incorrectly while appearing completely normal the rest of the time. Images might look normal to humans, but contain invisible text designed to confound AI. Other attacks attempt to hide backdoor codes to secret behaviors that remain invisible until a particular trigger phrase appears. The point is precision rather than chaos.

There's a whole philosophy and nascent movement encouraging the practice. Some want to flood the internet with misleading AI-generated content. Others discuss uploading deliberately corrupted information in the hope that tomorrow's models will eventually absorb it. Artists have embraced tools like Nightshade that subtly alter their images before posting them online, making them harder for AI systems to learn from while leaving them almost identical to the human eye.

Polluting the well rarely hurts only one person

But AI poisoning is a lot harder than slipping a little salt into someone's coffee. AI companies filter, clean, and review datasets long before they become part of a model. Poisoning a commercial system is considerably harder than just a misleading Wikipedia paragraph.

That doesn't mean it can't be dangerous. Cybersecurity researchers don't worry about ChatGPT getting a history fact wrong. The real worry is that poisoned information will mess with the behind-the-scenes AI systems used by hospitals, banks, or government agencies. Those models often rely on much narrower datasets and fewer security checks, making them more attractive targets.

A medical assistant that gives excellent advice except for one particular condition or banking software that always includes a hidden security flaw with every update. That's what poisoned data might do if it isn't caught in time. The techniques are not inherently wrong, but they can be abused like any other technology.

None of this means the frustration behind those dreaming of slipping erroneous facts into ChatGPT or Gemini is misplaced. Artists and authors are continuing to fight over AI training data use and misuse. But while the intended target may deserve criticism, poisoning AI data is unlikely to be a long-term solution.

AI already struggles with misinformation, hallucinations, and factual mistakes. Deliberately adding more bad information into the ecosystem risks amplifying exactly the problems critics already complain about. Protecting people, their livelihoods, and creative ownership is essential, but making AI worse will not somehow make the future better.