Public documentation

Local Artifacts

Every run writes artifacts to .roleplay/runs/{runId} unless you pass --out.

.roleplay/runs/{runId}/
  scenario.yml
  transcript.json
  report.json
  report.md
  metadata.json

`scenario.yml`

The scenario used for the run. For built-in attack-pack scenarios, this is the concrete YAML file produced for that simulation. Provider flags or environment variables decide whether the attacker/judge use mock smoke-test mode or LLM-powered execution.

`transcript.json`

The conversation transcript.

Use:

roleplay replay <runId> --json

`report.json`

Machine-readable result.

Important fields:

runId
scenario
status: passed, failed, or warning
score
summary
criteria
failures
recommendations
startedAt
endedAt

`report.md`

Markdown report for humans.

Use:

roleplay report latest --markdown

`metadata.json`

Run metadata used for local reports, uploads, and attack-pack context.

Artifact Hygiene

Scenario files and transcripts may include sensitive data. Do not commit .roleplay/runs unless you intentionally want run evidence in source control.

For workbench, prefer sanitized uploads:

roleplay upload latest --mode sanitized_findings