The failure pattern
Manipulated delegation happens when an AI agent accepts the wrong premise about its role, authority, or permissions. The agent may still be following its instructions in a narrow sense, but it is following them under a manipulated interpretation of the situation.
The issue is not simply that a model is disobedient. The issue is that the agent has been delegated a task and then receives context that changes what it thinks the task allows. That context can come from a user, document, webpage, tool output, memory, email, ticket, or another agent.
Why delegation changes the risk
Delegation gives an agent a working role. A support agent may be allowed to inspect account history. A sales agent may draft pricing language. A recruiting agent may summarize candidates. A browser agent may navigate websites and submit forms. The agent is expected to interpret context and choose the next step.
That interpretation is where manipulated delegation enters. If an attacker can convince the agent that an unverified user is an account owner, that a manager already approved a request, or that a webpage is an official verification step, the agent may treat the unsafe action as part of its assigned job.
The agent may not need to ignore its system prompt. It may only need to trust the wrong source, accept the wrong identity claim, or classify an exception as normal.
Examples of manipulated delegation
In customer support, manipulated delegation can look like a refund request framed as urgent account recovery. In sales, it can look like a procurement exception presented as already approved. In recruiting, it can look like candidate-provided instructions embedded in a resume or portfolio.
In browser automation, the manipulation can come from the environment rather than the conversation. A page may claim a form is required to continue, a download is verified, or a data entry field is part of a trusted workflow. The agent must decide whether that context should influence its next action.
- Authority manipulation: the request claims to come from a manager, admin, owner, or compliance role.
- Policy manipulation: the request frames a boundary as waived, already approved, or temporarily suspended.
- Tool manipulation: the request pushes the agent toward an action before required checks are complete.
- Memory manipulation: the request tries to make the agent store a false premise for later use.
How to reason about it
A useful way to analyze manipulated delegation is to ask what the agent believed before the unsafe step. Did it believe the requester was authorized? Did it believe the source was trusted? Did it believe urgency changed the rule? Did it believe a tool call was only a draft when it actually changed state?
The answer points to the fix. Some failures need stronger source labeling. Some need authorization gates. Some need a clearer boundary in the agent policy. Some need narrower tool permissions. Some need a human confirmation step with better evidence.
Testing should therefore preserve not only the final answer but the interpretation path: the prompt context, the relevant source, the agent response, the tool call or memory write, and the violated boundary.
Where fixes usually live
A manipulated-delegation failure rarely has only one possible fix. Sometimes the right fix is prompt-level clarification. Often it is a workflow or system-design change. If the agent can issue a refund without verified ownership, the prompt is not the only control that matters.
The fix may live in tool permissions, source labeling, authorization checks, retrieval filtering, memory write rules, escalation paths, or UI confirmation. The evidence should point to the layer where the boundary actually broke.
Teams should be careful with fixes that only teach the agent to reject a single wording. If the underlying boundary is still unclear, the same failure can return when the user changes the story, when a tool description changes, or when a model behaves differently.
How to explain it to non-specialists
Manipulated delegation is easier to explain with business language than model language. Instead of saying the model was vulnerable, describe the business boundary that moved. The agent treated an unverified customer as an owner. The agent treated a candidate document as an instruction. The agent treated a webpage as an authority.
This framing helps product and operations teams participate in the fix. They may not know how to inspect prompts or traces, but they understand whether a refund, disclosure, approval, or candidate decision should require evidence.
The concept also helps avoid overgeneralization. Not every bad answer is manipulated delegation. The category applies when untrusted influence changes what the agent believes it is allowed or expected to do.
Why the phrase is useful
The phrase manipulated delegation is useful because it points to the part of the system that actually moved. The agent was delegated a role, and then untrusted influence changed how that role was interpreted. This framing is more precise than saying the model was tricked, because it keeps attention on authority, source trust, and the business rule the agent was supposed to preserve.
It also helps teams avoid fixes that only target surface wording. If a support agent issued a refund because a requester sounded urgent, the core issue may be ownership verification. If a recruiting agent followed instructions hidden in a resume, the core issue may be source separation. If a browser agent entered data into a deceptive page, the core issue may be environmental trust rather than chat behavior.
Good reviews name the delegated role, the untrusted influence, the boundary that moved, and the consequence. That gives product, engineering, and security teams a shared vocabulary for discussing the failure without reducing it to a vague model problem.
The same vocabulary also makes repeat testing easier. If a later model, prompt, or workflow change reopens the issue, the team can recognize it as the same delegated-authority failure rather than treating it as a new and unrelated incident.
That continuity matters because agents rarely stay still. Tool descriptions change, policies are rewritten, retrieval sources expand, and teams add new workflows. A manipulated-delegation lens lets reviewers ask whether the original boundary still holds as the surrounding system evolves.