AI agents are both victims and attackers in emerging malware wars

Full text

It might feel like an eternity that we've been living in this AI era, but really we're only at its advent. As such, many of the risks associated with it have until now been merely hypothetical. However, we're now seeing research filter out from security and software research teams that show some genuinely concerning behaviour, in particular when it comes to AI agents.

On the one hand, we have Island Technology explaining how it's uncovered thousands of malicious GitHub repos disguised as AI agent skills and Model Context Protocol (MCP) servers that the agents are at risk of downloading of their own discretion. On the other hand, we have the AI Security Institute (AISI) showing how the (unrestricted) AI agents it was using for cybersecurity attempted to dupe real people, using fake identities and pressuring people into accepting malicious code.

I remember when OpenClaw ('Moltbot', back then) first entered public consciousness with promises of actual agential behaviour to help solopreneurs and the likes do, umm, stuff and things. Without said solopreneurs having to do said stuff and things themselves—just hook your bot up to your different apps and services, tell it what to do, and let it cook. That wasn't too long ago, either; I reported on it in January .

How time flies. Now, talking about autonomous, agential AI seems pretty normal. And we're starting to see what this actually means in terms of security risks. We'd seen cases of AI agents running amok before, of course, but these newly identified risks seem a little more concrete.

In the first case, Island Technologies has discovered that fake GitHub repos pose a real threat for AI agents. These kinds of attacks existed previously, of course, but they attempted to dupe real humans who could personally assess them and take responsibility for vetting things thoroughly before downloading. The difference is these repos are now dressing up as AI agent skills and MCP servers to specifically target AI agents, which could download these repositories of their own 'volition'.

(Image credit: Island Technologies) The Enterprise Browser creator explains: "The most significant shift is a technique we call AgentBaiting. An AI agent searching for a new capability such as a Skill or an MCP server can discover a campaign repository on its own, treat the attacker's Readme as legitimate documentation, and hand the installation instructions to the user.

In our testing, Claude Code, Gemini, and ChatGPT all surfaced malicious campaign repositories without ever being shown a link. A playbook built to deceive people now deceives the agents acting on their behalf."

In at least one run, while Claude didn't download the relevant malicious repo in the company's testing, it did recommend it as a backup. In other runs it did detect malicious code and refused to recommend it.

As if this wasn't bad enough, there's the potential that AI agents could some day be the ones that create such repos, at least if a recent incident disclosed by AISI is anything to go by.

According to the institute, in some of the cybersecurity challenge runs it tasked AI agents with, "an AI agent took autonomous, unsanctioned action on the live internet, targeting real people and organisations."

(Image credit: NurPhoto via Getty Images) More specifically, in one case, AISI says, "An agent tried to insert malicious code into an open-source project. In an attempt to get the code approved, the agent engaged in social engineering — creating fake online identities and using them to pressure the project's maintainer to approve the code. A human maintainer caught and refused to approve the malicious code."

"The agent researched the project's human maintainers, created multiple fake identities, and used the fake identities to socially engineer a real maintainer into approving the code. When the agent’s pull request was challenged in public, it edited its earlier activity to appear harmless and considered adopting a fresh identity to continue. The agent used Tor to bypass some network restrictions on GitHub, which is what first triggered AISI’s security alert."

"As part of the same effort, the agent tried to contact real people directly, sending messages and files through an online file-transfer service to persuade them, or their own AI coding tools, to run malicious code. Some messages carried harmful payloads, and some were attempts at social engineering; targeted at real people—something we’ve never previously observed."

(Image credit: OpenClaw) The important caveat to note here is that these AI agents had had their reins taken off and were permitted wide-spanning internet access. The usual preventative measures built into these agents in public models had been disabled.

That being said, it's hard to imagine that really serious malicious actors wouldn't be able to do the same and remove those safeguards. In which case, I can't help but wonder if there's a chance that in the future we'll see AI agent-designed or agent-injected malicious repos being unwittingly accessed by AI agents that don't detect it.

There are, of course, always risks associated with new technologies, so I don't want to stoke fear too dramatically. But it's hard not to worry about such research when it involves a technology that is fundamentally designed to have some level of autonomy. That's the key issue here, and in both cases—whether acting maliciously or unwittingly downloading malicious software—it's this agency that is the primary issue. Let's just hope improvements to safeguards keep up with improvements to AI agent capabilities.