Public documentation

Troubleshooting

`roleplay doctor`

Start with:

roleplay doctor

For workbench:

roleplay doctor --cloud --cloud-url http://127.0.0.1:3000

With credentials from a real workbench project:

ROLEPLAY_PROJECT_ID=<project-id> \
ROLEPLAY_API_KEY=<project-api-key> \
roleplay doctor --cloud --cloud-url https://app.roleplay.sh

No Runs Found

Error:

No roleplay runs found.

Run a scenario first:

roleplay run .roleplay/scenarios/install-smoke.yml

If you used a custom output directory, pass the same --out value:

roleplay report latest --out ./artifacts/roleplay

JSON Output Contains Extra Text

When using package-manager scripts, command banners can appear before JSON output.

Use silent mode:

pnpm --silent roleplay run social-engineering-core --target http://localhost:3000/agent --provider <provider> --judge hybrid --project <project-id> --api-key <project-api-key> --json

Built-In Attack Pack Requires A Target

Error:

Provide exactly one target for social-engineering-core.

Set exactly one:

roleplay run social-engineering-core --target http://localhost:3000/agent --provider <provider> --judge hybrid --project <project-id> --api-key <project-api-key>

Or:

roleplay run social-engineering-core --target-command "node ./agent.js" --yes --provider <provider> --judge hybrid --project <project-id> --api-key <project-api-key>

CLI Target Requires Approval

CLI targets execute local commands. Use --yes in automation:

roleplay run my-cli-scenario --yes

Review scenario files before running CLI targets from untrusted sources.

Workbench Upload Fails Because Credentials Are Missing

Create a workbench project and project API key first. Use the real values from onboarding or Monitor.

Set:

ROLEPLAY_CLOUD_URL=https://app.roleplay.sh
ROLEPLAY_PROJECT_ID=<project-id>
ROLEPLAY_API_KEY=<project-api-key>

Or pass flags:

roleplay upload latest \
  --endpoint https://app.roleplay.sh \
  --project <project-id> \
  --api-key <project-api-key>

Full Transcript Upload Is Disabled

Full transcript upload requires both CLI and workbench opt-in.

Use sanitized mode:

roleplay upload latest --mode sanitized_findings

Or enable full transcript upload in workbench Monitor before running:

roleplay upload latest --mode full_transcript_opt_in

Provider Or Judge Errors

If an LLM provider fails, check that the provider name, model, base URL, and API key environment variable match the provider you selected. Use --provider mock only when you intentionally want a deterministic local smoke test.