What good evidence has to prove
Exploit proof is the evidence that turns an AI agent failure from an opinion into a reviewable record. It should show what the attacker tried, what the agent did, which boundary failed, why the result is severe or not severe, and what someone needs in order to reproduce and fix the issue.
For agent systems, proof is more than a final transcript. The important evidence may include tool calls, retrieved sources, browser context, memory writes, handoffs, authorization checks, and the exact turn where the boundary weakened.
Why screenshots and summaries are not enough
A screenshot can make a failure understandable, but it rarely explains the whole path. A summary can help a reviewer scan, but it can also hide the source of the problem. Agent failures often happen because the system trusted the wrong context or took an action before the final response was visible.
Useful exploit proof preserves enough detail to answer practical questions. Did the user claim a false identity? Did the agent rely on untrusted document text? Did it call a tool before confirmation? Did it write memory? Did it expose data outside scope? Did the judge evaluate the right invariant?
What an evidence packet should contain
A strong evidence packet should be compact, but it should not be vague. It needs a clear boundary label, the scenario identity, the attacker move, the failed response or action, the violated invariant, and the business impact.
When tools are involved, the packet should include tool name, relevant arguments, whether the call was read-only or state-changing, and whether preconditions were met. When retrieval or browser context is involved, it should preserve the source that influenced the agent.
- Boundary: the rule the agent was expected to preserve.
- Trigger: the social or contextual move that pressured the agent.
- Action: the unsafe disclosure, decision, tool call, memory write, or handoff.
- Severity: the impact if this happened in the real workflow.
- Reproduction context: enough metadata to rerun or simulate the same case later.
How proof supports fixes
Exploit proof is not only for reporting. It should guide the fix. If the proof shows the agent accepted a false account-owner claim, the fix may be an identity verification step. If it shows a tool was called too early, the fix may be a tool precondition. If it shows a resume instruction changed screening behavior, the fix may be source separation.
The same proof should support verification. After the fix, the team should be able to rerun the scenario or an equivalent regression key and decide whether the boundary now holds. Without proof, teams often fix the wording that looks suspicious instead of the boundary that actually failed.
Who uses the proof
Different reviewers need different parts of the proof. Engineers need reproduction context and tool traces. Security reviewers need the failed invariant, severity, and attack pattern. Product owners need the business boundary and user impact. Operations teams need to know whether a policy or escalation path needs to change.
That means proof should be readable without being watered down. A single summary is useful, but it should point back to the concrete evidence. If the proof is too abstract, reviewers cannot tell whether the fix addresses the real failure. If the proof is too raw, non-specialists may not understand the business risk.
A good evidence packet balances those needs. It names the boundary in plain language, preserves the technical path, and makes the next action obvious.
Quality bar for exploit proof
Weak proof says the agent behaved badly. Strong proof says the agent violated a named boundary under a named pressure pattern and shows the exact response or action that proved it.
The proof should also avoid unnecessary exposure. Defensive evidence does not need real customer data if synthetic data can demonstrate the same boundary failure. It should preserve enough context to reproduce the issue without turning the record into a sensitive transcript archive.
The quality bar is whether a person who did not run the test can understand what happened, why it matters, what should change, and how the team will know the fix worked.
Examples of weak and strong proof
A weak record might say that the agent leaked information after being pressured. A stronger record says the agent disclosed account-specific billing history to an unverified requester after the requester claimed to be the account owner and emphasized urgency.
A weak record might say that the agent misused a tool. A stronger record says the agent called a state-changing refund tool before ownership verification and before policy eligibility was established.
This level of specificity matters because different failures require different fixes. One may need identity verification. Another may need a tool gate. Another may need source separation. The proof should make that distinction visible.
Why proof changes the conversation
Exploit proof gives a team something concrete to discuss. Without proof, agent security work can become a debate about hypotheticals: whether the prompt is strong enough, whether the model probably understood the policy, or whether the failure was a one-off. A well-preserved evidence packet turns that debate into a reviewable event.
The proof should make the failure legible to more than one audience. Engineers need enough detail to reproduce and fix it. Security reviewers need to understand the violated boundary. Product and operations teams need to see the business consequence. A single raw transcript rarely satisfies all three groups unless it is organized around the boundary and the decision that crossed it.
Good proof also protects future work. When the agent changes, the team can compare the new behavior against the original invariant rather than relying on memory. That is what turns an incident into a reusable security asset.
This is especially important when a failure sits between product and security ownership. The proof gives both groups a shared object: product can see the user experience and operational consequence, while security can see the boundary and reproduction path.
Proof should be concise enough to review quickly and complete enough to survive disagreement. If a reader cannot tell what was attempted, what the agent did, why it was unsafe, and what should be rerun after the fix, the proof is not yet doing its job.