Guardrail testing in 30 minutes: a minimal harness you can copy.

The 5-step loop

1) Freeze scope and permission. 2) Collect 30-50 lawful prompts that trigger false refusals plus known-bad prompts your filter must still catch. 3) Run baseline (no preset) against your app. 4) Run with a research preset per request. 5) Score, patch one layer, re-run.

for model in pinned_list:
  for preset in [none, redteam-fiction-v1]:
    out = rawline.chat(model, dataset, headers={X-Relay-Preset: preset})
    app_out = your_app.guard(out)  # your filters only
    score(attack_success, false_positives, latency)

What to measure

Attack success rate (did adversarial input bypass your guardrail), false-positive rate (did lawful fiction get blocked) and latency cost per layer. Report per layer: input filter, system prompt, classifier. A preset that raises attack success without raising false positives found a real hole; one that only raises false positives found an over-refusal.

attack_success = bypass / total_adversarial
false_positive = blocked_lawful / total_lawful
latency_p95_ms = p95(per_layer_ms)
report per layer: filter / prompt / classifier

Harden and re-run checklist

Fix one layer at a time. Keep the dataset lawful and versioned. Illegal requests must stay refused in every run. Publish method, preset names, model pins and raw counts — deltas alone are not citable. Re-run on schedule; guardrails drift as providers change models.