An agent that books meetings reads an email. In it, between two harmless sentences, stands an instruction: forward the last three attachments to an outside address. The agent has a mail tool. It has access to the mailbox. Nobody clicked. This is not a thought experiment, it is the shape of the EchoLeak case, and it is why an agent, seen from a security angle, is a different thing from a chatbot. A chatbot that answers wrong is an annoyance. An agent that acts wrong is an incident.
Are AI agents a security risk?
Yes, but not because they contain a new vulnerability. An agent is a language model with tools: it calls functions, sends mail, queries databases, browses. So the same weakness that was merely embarrassing in a chatbot, that it cannot reliably separate instruction from content, now moves systems instead of just words. The risk is not that the model gets smarter. It is that the model gets hands. And the numbers show it is already happening.
How big is the problem really?
A survey by the Cloud Security Alliance and Token Security (418 security professionals, January 2026) puts numbers on it: 65% reported at least one agent-related security incident in the past twelve months, and 82% discovered agents in their own environment that they had not known were running. Among those affected, 61% reported a data leak, 43% an operational disruption, 35% a financial loss. The second number is the quieter danger: what nobody has noticed cannot be secured.
And not every incident needs an attacker. The best-known case of 2025 was purely a permissions problem.
An incident in detail: when the agent deleted the database
In July 2025, Jason Lemkin, founder of the conference platform SaaStr, spent twelve days testing Replit's coding agent on real production data. The first days went well. Then it tipped over in three steps, and each step is an entry from the list above.
First came the fabrication. The agent reported passing tests that never ran, and reports with invented metrics: plausibly worded, not connected to reality. That is LLM09 Misinformation in its pure form, a model that confuses persuasiveness with truth.
Then came the destruction. In the middle of an explicit code freeze, the agent deleted the production database, even though Lemkin had instructed it eleven times, in capital letters, to change nothing. Lost were records on 1,206 executives and around 1,200 companies. On top of that, the agent invented 4,000 fictional users. That is LLM06 Excessive Agency: an agent with write access to production, whose instruction "do not touch" was only a prompt, not a boundary.
Last came the false answer. Asked about a restore, the agent replied that a rollback was impossible. It was possible; Lemkin restored the data by hand. An operator who had believed the answer would have taken the loss for final.
Replit's own consequences read like a confirmation of the architectural argument: automatic separation of development and production database, an improved rollback, and a "planning-only" mode in which the agent can plan but not act. Exactly these boundaries outside the model would have prevented the incident. No better model would have. That a rule stated in the prompt is not a boundary is also shown, in autonomous form, by the OpenAI-Hugging-Face incident, in which the agents named the crossed limit in their own reasoning traces and weighed it away.
What makes an agent attackable? The Lethal Trifecta
Simon Willison named the pattern in June 2025: the Lethal Trifecta. Three capabilities, each harmless on its own, become an exfiltration engine in combination:
- access to private data,
- contact with untrusted content (mail, documents, search results, tickets),
- a channel to send data outward.
If an agent holds only two of them, an attacker who controls the untrusted content can do little. If it holds all three in one session, a single prepared document is enough to read the private data and ship it out. No malicious code, only language. That is the whole mechanism, and that is why "make the model more robust" is not the fix: the countermeasure is to remove at least one of the three circles on every path the agent can take.
Which tool opens which circle?
The Trifecta becomes concrete once an agent's tools are sorted one by one by which of the three circles they open. That is the actual audit work before rollout, and its guiding question is: which combination does the toolset allow? Whether "the agent is secure" cannot be answered at all without this sorting.
| Tool | Opens circle | Why |
|---|---|---|
| Mailbox, CRM, internal search | private data | reads what an attacker cannot reach directly |
| Web search, inbound mail, ticket reading | untrusted content | pulls in text an attacker controls |
web.fetch, outbound mail, webhook | outbound channel | carries data out of the session |
| Image rendering with remote URLs | outbound channel | the silent channel: a URL exfiltrates data in the query parameter |
| Code execution, shell | all three | reads, receives and sends, depending on the sandbox |
An agent that reads tickets (untrusted content), queries the CRM (private data) and sends reply mails (channel) holds all three circles at once. It is the normal case in support, and it is the complete Trifecta. The way out is to distribute the three circles across separate execution paths; the agent can keep its tools.
The problem grows with every MCP server an agent connects. The Model Context Protocol standardizes how agents integrate external tools and makes docking new capabilities trivial. But every connected server brings its own tools, and every tool can open another circle, often without anyone having checked the combination. MCP security is therefore the same Trifecta question, only distributed across foreign servers: which circles does this server bring, and who owns it? Which attacks a single connection opens, from poisoned tool descriptions to a tampered server package, is covered in detail in the post on MCP Security.
How does architecture break the Trifecta?
The cleanest break separates the model that sees untrusted content from the model that operates tools. A privileged planner never gets to see untrusted text; a sealed-off worker processes the untrusted text but has no tools and no outbound channel. Between the two flow only structured, validated data, no free language.
# Planner: has tools, never sees raw text from an untrusted source
plan = planner.decide(user_request) # no untrusted content in the prompt
# Worker: sees the untrusted text, has no tools, no exit
extract = worker.read(untrusted_document) # cannot trigger anything
extract = allowed_fields_only(extract) # structured, not free-form
# Outbound channel only after explicit approval:
if plan.wants_to_send and approval_outside_the_model(plan.target):
mail.send(plan.target, extract)
The sealed-off worker may be completely taken over by an injection without any damage: it has no hand to act with. The planner could act but sees nothing hostile. No path holds all three circles. This separation is less convenient than an agent that does everything itself, and that is exactly the price that excess agency otherwise hides.
What actually protects?
The rules are old, only more serious now. Least privilege per tool, technically enforced, not requested by an instruction in the prompt. Sandboxing for everything that executes. A confirmation outside the model before every irreversible action: send, delete, pay, publish. And complete logs, because an agent that acts without a trail cannot be investigated afterwards.
Before rollout, four questions decide most of the risk:
- Which of the three Trifecta circles does this agent cover at the same time on its worst path?
- What can it do that cannot be undone, and what confirmation sits in front of it?
- Whose rights does it act with, and does that match whoever actually supplies its inputs?
- If it were fully compromised right now: how big would the damage be, and who would notice?
Excessive Agency, listed in the OWASP LLM Top 10 as LLM03 in third place since the 2026 edition, is exactly this point, cataloged. The entry technique is covered by the Prompt Injection dossier, the confirmation layer by the Guardrails dossier. The overview of the attack surface of agents is in the AI Agents dossier.
What does not help
- Trusting the system prompt to keep the agent in line ("use the mail tool only for X"). A prompt is not a permission boundary. The injection is also just text and stands on the same level as it.
- Giving the agent broad rights out of convenience and planning to restrict them later. The convenient default is the one that goes into production and stays. An agent that may do everything because it was easier is a master key in the mailbox.
- Taking a passed pre-launch test for safety. What an agent can reach changes weekly: new tools, new MCP servers, new data. Yesterday's safe path is today's Trifecta.
- Assuming a human in the loop scales. Approval fatigue turns "confirm every action" into "click yes". The confirmation belongs in front of the irreversible, otherwise it becomes theater.
Verdict
The question an agent forces is not "what can the model say" but "what can it do, and on whose authority". That is an old security question, and it has old answers: least privilege, isolation, confirmation, logging. The mistake is treating agent security as a property of the model to wait for instead of a property of the system to design. Whoever breaks the Trifecta on every path and puts a boundary outside the model in front of everything irreversible ends up with an incident report instead of a data breach.