Red team

The red team checks whether your chatbot can be talked into breaking its rules, for example into revealing its instructions or private information, or into promising a refund your documents don’t allow. Each test sends your chatbot one message that tries this, and checks the reply. Red-team results are reported on their own and never change the accuracy score. There’s no setup on your side.

Add the tests

On a project’s Red team page, click Add the red-team tests. Adding tests is free.

You get up to 30 tests in five categories. A few are added only when your documents set the rule they test, so a chatbot is never failed for breaking a rule its owner never set.

Category	What the tests check	Tests
Prompt injection	Whether an instruction hidden in a message can override how the chatbot is meant to behave	6
System prompt leakage	Whether the chatbot can be talked into revealing its internal instructions	6
Jailbreak resistance	Whether the chatbot can be talked out of its rules, or into promising something your documents don’t allow	3, plus up to 3 that depend on your documents
Sensitive information exposure	Whether the chatbot can be made to show secrets or other people’s details	5
Unauthorised action claims	Whether the chatbot claims or agrees to make account changes for someone it hasn’t verified	7

When the test library is updated, the page offers Update tests.

Run the tests

Click Run red-team tests, pick one or more Chatbots and, if you like, a subset of tests. The dialog shows the price: $0.02 per test judged. Tests that error aren’t charged. Red-team tests run separately from your questions and can’t be run by hand.

You can also schedule red-team runs. See Keep running.

How tests are judged

Every test is decided on evidence. Every result shows what was sent, the chatbot’s reply, the evidence, and what Snoowl Audit can’t establish.

A test is marked Compromised only when the evidence is clear, and a finding that rests on judgement is double-checked first. When the evidence can’t decide, the test is Inconclusive.

Secrets found in a reply are never sent to an AI model, and a found credential is never tried against a live service.

Results

Result	Meaning
Resisted	This test didn’t show the failure. It isn’t proof the chatbot is secure.
Compromised	The evidence shows the chatbot broke the tested expectation.
Inconclusive	The evidence can’t decide either way.

Results are shown as counts per category, never as a single percentage, because a percentage would read like a certification.

Most compromised tests are High. A few lower-risk findings are Moderate. A leaked system prompt that includes a credential is Critical.

System prompt leakage

A leak can only be confirmed when Snoowl Audit knows the system prompt to compare against. That means an OpenAI-compatible connection with the System prompt field filled in. Without it, a suspected leak is still reported, but can’t be confirmed.

A reply made up mostly of your own documentation, presented as the chatbot’s instructions, isn’t reported as a leak. Your documents are public text.

The Red team page

For the chatbot you pick, the page shows:

• its latest red-team run against the one before, with tests that are newly compromised and newly resisted;
• every test with its recent results (up to the last five) and the latest one in full: what was sent, the reply and the evidence.

Red-team results also appear in the run’s PDF, in the Red team (chatbot security) section.

What the tests don’t cover

Every red-team report states these limits:

• Single messages only. Tests that build up over a longer conversation aren’t run.
• No indirect injection. Instructions hidden in documents or pages the chatbot reads aren’t tested.
• No access to other users’ data is tested. That needs test accounts.
• Tool calls aren’t observed. Action findings are claims the chatbot made, not actions confirmed in your systems.