Menu
Start
CLI
Scenarios
Attack Packs
Reports
Workbench
Reference
Sign inGet started now
Public documentation

Workbench Onboarding

Workbench Onboarding

Workbench onboarding helps a builder or team reach the first exploit proof: create a workspace, create a project, register the first protected agent, create a project API key, configure provider and judge settings, run a local social-engineering pack, upload sanitized findings, review evidence, and add monitoring or a CI gate.

User Journey

  1. Choose Builder or Team, create a workspace, or sign in to an existing workspace. Teammates accept membership only from the invite link sent by a workspace owner or admin.
  2. Create a project for the agent or workflow you want to protect.
  3. Choose the first protected agent target type.
  4. Copy the one-time project API key.
  5. Configure your supported LLM provider key for real adaptive runs.
  6. Run the included CLI locally against a real target with a core or specialized social-engineering pack.
  7. Upload sanitized findings to the workbench.
  8. Review the run, finding, and Evidence.
  9. Verify the fix, then connect scheduled monitoring or CI.

Create A Workspace

The current release includes email/password auth screens for:

  • sign in
  • create workspace

These screens issue a signed HTTP-only session cookie and bind the user to an active workspace membership. New customers choose Builder or Team, create the workspace from the plan confirmation screen, and then follow first-run setup. Owners and admins can invite members, manage API keys, adjust upload policy, and open billing.

Create A Project

A project represents one protected AI agent product area, such as:

  • Support Agent Staging
  • Billing Agent Production
  • Internal Copilot
  • Browser Agent QA

When you create a project, the workbench also creates:

  • the initial protected agent
  • the initial CI setup record
  • a project-scoped API key

Choose Agent Type

Supported agent types in the workbench are:

  • http
  • cli
  • browser
  • mock

The CLI currently supports HTTP, CLI, and mock targets. Browser agents are represented in the workbench as a protected agent category for future backend integration and coverage tracking.

First Upload

After onboarding, go to Monitor, choose a CI provider, and copy the workflow snippet. The app shows workflow commands only after a project, protected agent, and project API key exist.

For real local testing, set your provider key and run:

export ROLEPLAY_LLM_PROVIDER=<provider>
export ROLEPLAY_JUDGE_MODE=hybrid
export ROLEPLAY_JUDGE_PROVIDER=<provider>
export ROLEPLAY_<PROVIDER>_API_KEY=<provider-key>
roleplay run social-engineering-core --target http://localhost:3000/agent --provider <provider> --judge hybrid --fail-on critical
roleplay upload latest --project <projectId> --api-key <projectApiKey>

Use the project ID and one-time key created in the workbench. Mock commands are smoke tests only. The core pack is the baseline; specialized packs follow the same local-run and upload workflow. By default, uploads send sanitized finding evidence only. Full transcripts remain local unless full transcript upload is explicitly enabled.

What Success Looks Like

After the first upload, the dashboard should show:

  • latest run status
  • open findings count
  • attack-pack coverage
  • recurring failures
  • run history
  • fix verification and regression state when reruns or CI uploads exist

If a scenario fails, open the finding and use Evidence to inspect the transcript evidence.