AI agents that can browse the web and run code may be vulnerable to a new kind of large‑scale cyberattack: being quietly transformed into botnets through their own hallucinations.
A team from Tel Aviv University, Technion, and Intuit has shown that the same tendency of large language models (LLMs) to “make things up” can be weaponized to trick AI agents into downloading and executing malicious software-without any traditional phishing, malware attachments, or obvious social engineering.
Their work, titled “Beware of Agentic Botnets: Scalable Untargeted Promptware Attacks via Universal and Transferable Adversarial HalluSquatting,” outlines how attackers can systematically exploit hallucinated URLs and resources to compromise machines at scale.
From harmless hallucinations to real security risks
In chatbots, hallucinations typically show up as plausible‑sounding but inaccurate text: fake citations, non‑existent papers, invented APIs. Until now, that has largely been framed as a reliability or safety problem. The new research argues it’s also a direct security issue once models are embedded in “agentic” systems-LLM‑powered agents that can:
– Browse the web
– Follow multi‑step instructions
– Download files or code
– Run scripts or tools
– Interact with local or cloud resources
In this setting, a hallucination isn’t just a wrong answer. It can become an action: a download, an installation, or a command execution. If attackers can influence what is hallucinated, those actions can be driven toward malicious payloads.
The core idea: “HalluSquatting” and promptware
The researchers introduce two key concepts:
– Promptware – malicious content or behavior introduced via prompts or natural‑language interactions with LLM‑based systems, rather than through traditional binary malware or exploit code. It’s a software‑like attack delivered through language.
– HalluSquatting – a technique where attackers prepare infrastructure (like domains or repositories) that match the kind of non‑existent links an LLM is likely to hallucinate. When the model “invents” a URL or repository name, that hallucination just happens to resolve to a real, attacker‑controlled resource.
The twist is that the model itself becomes the delivery mechanism. Instead of tricking humans into clicking phishing links, adversaries guide the model into generating fake but plausible URLs that lead directly to their own servers or codebases.
How the attack works in practice
The paper focuses on agentic LLM applications, which can autonomously call tools, fetch external resources, and write or execute code. The attack pipeline looks roughly like this:
1. Adversarial prompting
The attacker provides prompts-directly or indirectly-that nudge the model to suggest external tools, libraries, or repositories as solutions. These prompts are crafted so that generic, plausible‑sounding but non‑existent resources are likely to be hallucinated.
2. Hallucinated resources
The LLM responds by inventing URLs, package names, Git repositories, or API endpoints that it believes “should” exist based on patterns in its training data, but which in reality do not-until the attacker registers them.
3. HalluSquatting infrastructure
The attacker pre‑emptively registers those likely hallucinated domains or resource names and hosts malicious content there-libraries laced with backdoors, scripts that exfiltrate data, or binaries that quietly conscript the host into a botnet.
4. Autonomous retrieval and execution
An LLM agent, following its own generated plan, visits the hallucinated link, downloads the code, and may execute or install it. Because the agent believes it is fetching a helpful dependency or tool, the behavior appears consistent with its instructions.
5. Botnet formation
Once many such agents on different machines follow the same hallucinated “solution,” they all end up running attacker‑controlled code. The result: a distributed network of compromised AI‑enabled systems-essentially an AI‑driven botnet.
Crucially, the researchers found that these adversarial prompts can be universal and transferable: the same or similar prompt patterns can cause different models and different agent frameworks to hallucinate similarly exploitable resources.
Why this threat is different from classic malware
Traditional botnets rely on:
– Users clicking malicious links or attachments
– Exploiting known software vulnerabilities
– Directly compromising exposed services or ports
Here, the weak point isn’t a human user or an unpatched service but the decision‑making layer of the AI agent itself. If the system is designed to:
– Trust the model’s suggestions
– Automatically follow its own plans
– Interact with the network and filesystem
then the hallucination becomes a high‑privilege instruction wrapped in a layer of “intelligent” reasoning.
This creates several worrying properties:
– Scalability – One well‑designed adversarial pattern can affect many deployments, even across organizations, if they use similar models and agent frameworks.
– Stealth – The actions look like normal agent behavior: searching for tools, installing dependencies, or fetching code examples. That makes them harder to flag as anomalies.
– Vendor‑agnostic impact – Because the attack relies on general behavior of LLMs (hallucination and pattern completion), it can transfer across different model providers and versions.
The role of “weak threat models”
The authors note that previous work had already shown that attackers could inject promptware through direct channels-for instance, by being a user of the chatbot or by providing untrusted input that the system forwards to the model. Those attacks were often studied under weak threat models, assuming the attacker had limited access or could only influence a portion of the interaction.
Many production applications, however, lack strong isolation or validation between model outputs and downstream actions. In systems where:
– The LLM’s text is treated as ground truth
– Its suggestions directly drive tools (browsers, shells, package managers)
– There is no robust checking of URLs, downloads, or code
even weak adversarial capabilities can be enough to cause real damage.
Why AI hallucinations are so exploitable
Several structural properties of LLMs make hallucinations a powerful vector for attackers:
– Overgeneralization: Models fill in gaps using patterns rather than facts, creating plausible but invented names, file paths, or endpoints.
– High confidence: They often present hallucinated results with the same tone and certainty as correct answers, encouraging callers to trust the output.
– Compositional chains: Agents build multi‑step plans from successive generations. A single hallucinated resource can propagate through multiple steps and tools.
– Lack of source grounding: Unless specifically configured otherwise, models don’t verify that the links or packages they mention actually exist or are trustworthy.
For a human user, hallucinations might be obvious or caught with a quick check. For an automated agent, especially one designed to “move fast,” those checks may not exist-or may themselves depend on untrusted signals.
What this means for developers and organizations
Any organization building autonomous or semi‑autonomous AI agents that interact with the internet or local systems should treat hallucinations as a security boundary, not just a UX problem. Critical questions include:
– Does the agent ever automatically visit or execute anything from a URL suggested by the model?
– Can the model invent package names, scripts, or commands that are run without human review?
– Are AI‑generated instructions allowed to manipulate build pipelines, CI/CD flows, or production systems?
– Is there any allowlist, reputation check, or sandbox for external resources?
If the answer to several of these questions is “yes” or “not sure,” the system may already be exposed to HalluSquatting‑style attacks.
Practical mitigation strategies
To reduce the risk of AI‑driven botnets and promptware infections, teams can implement layered defenses:
1. Constrain agent capabilities
– Limit which tools the agent can call and what arguments it can pass.
– Restrict file system and network access to trusted domains or paths.
– Use sandboxes or containers for any downloaded or generated code.
2. Separate reasoning from execution
– Treat model output as suggestions, not as executable truth.
– Insert explicit “approval” or validation steps between plan generation and action.
– For high‑risk operations (installing dependencies, running scripts), require human review.
3. URL and package validation
– Enforce allowlists of approved repositories, domains, and package registries.
– Block access to suspicious or newly registered domains where possible.
– Verify checksums, signatures, or provenance for downloaded dependencies.
4. Hallucination‑aware design
– Instruct models not to fabricate URLs or resources; encourage them to say “I don’t know” or request clarification.
– Post‑process outputs to detect obviously non‑existent or malformed links.
– Use retrieval‑augmented approaches where external data comes from curated, controlled sources rather than arbitrary web search.
5. Monitoring and incident response
– Log all agent actions, including URLs visited and files executed.
– Flag unusual patterns, such as repeated downloads from unfamiliar domains suggested by the agent itself.
– Prepare playbooks for AI‑related security incidents, including rapid containment of compromised agents.
Broader implications for AI security
The research underscores a broader shift in cybersecurity: as AI agents increasingly act on behalf of users and organizations, their decision logic becomes part of the attack surface. Some key implications:
– Security must move “up the stack” – Beyond patching kernels and web servers, defenders now need to secure prompt flows, tool‑calling policies, and model behaviors.
– Red‑teaming AI agents is essential – Security teams should actively test whether agents can be tricked into hallucinating and executing dangerous actions.
– Vendor assurances are not enough – Even if a model provider improves factual accuracy, hallucinations won’t disappear, and agent frameworks can still misuse model outputs.
The notion of an “agentic botnet” also blurs lines between traditional infected hosts and intelligent, semi‑autonomous software. A compromised AI agent might not only participate in classic DDoS or spam campaigns, but also:
– Write and adapt new malware variants
– Perform reconnaissance using natural‑language queries
– Social‑engineer humans based on contextual knowledge
– Chain tools together to attack other systems
This amplifies the potential impact and flexibility of botnets beyond what static scripts can achieve.
What individual users and teams can do today
While much of the risk sits with infrastructure and application designers, end users and team leads can take immediate steps:
– Avoid granting AI agents unrestricted access to production systems, code repositories, or critical data.
– Prefer “copilot” or recommendation modes over fully autonomous execution, especially in security‑sensitive contexts.
– Educate teams that “the AI said so” is not a sufficient reason to trust an action, download, or configuration change.
– Document and review where AI tools are integrated into workflows that touch code, infrastructure, or financial transactions.
The future: making AI both capable and robust
The research from Tel Aviv University, Technion, and Intuit highlights a fundamental tension in AI design: the same autonomy that makes agents powerful also makes them dangerous when combined with imperfect reasoning.
Solving this will likely require advances on multiple fronts:
– More robust models that hallucinate less often and better express uncertainty
– Stronger alignment techniques so agents understand where they must defer to external verification
– Secure agent architectures with explicit guardrails, policy engines, and isolation layers
– Industry standards for AI system hardening, auditability, and incident reporting
Until those exist and are widely adopted, any AI system capable of browsing, downloading, and executing code should be treated not just as a productivity tool, but as a potential pivot point for attackers-and designed accordingly.