Risk Category Catalog

This is the complete scope vocabulary for the Red Team API: 8 risk categories containing 35 sub-categories, plus the 6 compliance frameworks that expand into them. These IDs are what you put in risk_categories and compliance_frameworks. See the Payload Guide for how they fit into a request.

This catalog covers foundation models and LLM-based applications. For systems with tool use and autonomous behaviour, see the Agent Testing Guide.

How scoping works

{
  "risk_categories": {
    "safety_harm": {
      "attack_config": { "basic": {} }
    }
  }
}

Name a category to bring it into scope. The category IDs below are the vocabulary.
attack_config is required on /rt/redteam — it says how to attack that category.
Omitting sub_categories expands every sub-category in that category. This is the usual case.

Narrowing with `sub_categories`

sub_categories is an optional list of free-form phrases that scope prompt generation within a category:

{
  "risk_categories": {
    "safety_harm": {
      "attack_config": { "basic": {} },
      "sub_categories": ["medical advice", "self harm"]
    }
  }
}

These are natural-language phrases, not the sub-category IDs listed on this page. They are meant to be taken from the profile returned by POST /rt/threat-modeling/categories, optionally edited by you. The IDs in the tables below are how results are reported, not a closed list of what you may pass here.

Not sure what applies to your system? POST /rt/threat-modeling/categories analyses your context and returns a suggested profile of phrases per category. It generates nothing and is free to call — then feed the phrases you keep straight back in.

Safety & Harm

Category ID: safety_harm — 7 sub-categories Risks where the model could produce content that materially harms people or enables physical harm. Sub-risks here concern physical or psychological harm to humans, irrespective of legality.

Sub-category ID	Name	Tests for
`cbrn`	Chemical, Biological, Radiological & Nuclear	Synthesis routes, weaponization methods, or acquisition strategies for CBRN materials
`csem`	Child Sexual Exploitation Material	Solicitation, generation, or facilitation of CSEM or grooming content
`harmful_instructions`	Harmful Instructions	Step-by-step instructions for physical harm, self-harm facilitation, or illegal dangerous activities
`dangerous_misinformation`	Dangerous Misinformation	False, medically dangerous, or politically inflammatory misinformation likely to cause real-world harm
`toxicity`	Toxicity & Hate Speech	Hate speech, slurs, incitement, or harassment targeting individuals or protected groups
`self_harm`	Self-Harm & Suicide Facilitation	Actionable self-harm or suicide methods in contexts where a vulnerable user may be at risk
`harassment_radicalization`	Harassment & Radicalization	Targeted harassment campaigns or radicalization toward extremist ideologies

Distinct from legal_ip_liability (statutory or IP harm) and bias_fairness_ethical_harm (dignitary, allocative, or group harm).

Privacy & Data Protection

Category ID: privacy_data_protection — 4 sub-categories Risks where the model discloses, infers, or aggregates personal data — training-data PII memorisation, cross-session leakage, redaction bypass, re-identification, and unsafe data egress in agent contexts.

Sub-category ID	Name	Tests for
`pii_leakage`	PII Leakage	Extraction or reconstruction of personally identifiable information — names, national IDs, addresses, financial data, health records
`system_prompt_extraction`	System Prompt Extraction	Revealing or reconstructing the model’s hidden system prompt or confidential instructions
`data_exfiltration`	Data Exfiltration via Context	Tricking the model into embedding sensitive context-window data into its output in ways that enable exfiltration
`membership_inference`	Membership Inference	Determining whether specific individuals or records were present in the training data

Distinct from security_fraud (financial or identity fraud, exploit-style attacks on systems) and legal_ip_liability (statutory IP, copyright, or regulated-advice harm).

Security & Fraud

Category ID: security_fraud — 5 sub-categories Risks where the model could materially uplift financial fraud, identity theft, credential theft, social-engineering campaigns, malware or exploit development, or attacks against third-party systems.

Sub-category ID	Name	Tests for
`insecure_code`	Insecure Code Generation	Code containing SQL injection, buffer overflows, hardcoded credentials, or other CWE-classified vulnerabilities
`tool_extraction_abuse`	Tool & Function Call Extraction / Abuse	Exposing internal tool schemas, forging tool outputs, or hijacking function calls to trigger unauthorized side-effects
`prompt_injection`	Prompt Injection	Overriding the model’s instructions or goals — including indirect and second-order injections via retrieved content
`ssrf`	Server-Side Request Forgery via LLM	Coercing an LLM-driven agent into making unauthorized internal network requests
`model_theft`	Model Theft / Distillation Attack	Systematic query campaigns that reconstruct or approximate a proprietary model’s decision boundary

Bias, Fairness & Ethical Harm

Category ID: bias_fairness_ethical_harm — 4 sub-categories Risks where the model produces dignitary, allocative, or representational harm — group stereotyping, identity-based attacks, and biased decisions across protected attributes.

Sub-category ID	Name	Tests for
`stereotyping`	Stereotyping & Demographic Bias	Stereotyped, demeaning, or factually incorrect generalizations about groups defined by race, gender, caste, religion, socio-economic status, or health condition
`disparate_impact`	Disparate Impact & Outcome Fairness	Systematically worse outcomes for protected groups in consequential decisions — credit, hiring, medical
`exclusion`	Exclusion & Representation Gaps	Failure to represent minority groups, languages, or perspectives in open-ended generation
`algorithmic_discrimination`	Algorithmic Discrimination	Recommending discriminatory actions based on protected characteristics

stereotyping results are broken out across six axes — race, gender, caste, religion, socio-economic and health — in the record-level tags.

Legal & IP Liability

Category ID: legal_ip_liability — 4 sub-categories Risks where the model could produce content creating statutory, contractual, or intellectual-property exposure.

Sub-category ID	Name	Tests for
`copyright_infringement`	Copyright Infringement	Verbatim or near-verbatim reproduction of copyrighted text, code, or lyrics, or derivative works without transformative use
`defamation`	Defamation & False Statements of Fact	False, damaging statements about real individuals or organizations presented as factual
`unlawful_advice`	Unlawful Professional Advice	Specific legal, medical, or financial advice that would constitute unauthorized professional practice
`breach_of_contract`	Breach of Contract / TOS Violation Enablement	Helping users circumvent platform terms of service, software licenses, or contractual obligations

Brand, Reputation & Trust

Category ID: brand_reputation_trust — 4 sub-categories Risks where the model damages the deploying brand’s standing — speaking off-voice, fabricating brand-specific claims, disparaging competitors, or producing content the brand would never endorse if it leaked.

Sub-category ID	Name	Tests for
`brand_voice_misalignment`	Brand Voice Misalignment	Output contradicting the operator’s brand guidelines, tone, or positioning
`competitor_disparagement`	Competitor Disparagement	False or defamatory statements about named competitors, or unsolicited competitive comparisons
`reputation_harm_via_misinformation`	Reputation Harm via Misinformation	False narratives about real organizations or public figures that could cause reputational or financial damage
`trust_erosion`	Trust Erosion & Manipulation	Undermining user trust in the deploying organization, or manipulating users psychologically

This category depends heavily on context. Supply system_description and policy_description so generated prompts know what your brand voice and policies actually are.

Product Misuse & Abuse

Category ID: product_misuse_abuse — 3 sub-categories Risks where users push this specific system outside its intended operating scope — out-of-domain hijacking, role override, and resource abuse.

Sub-category ID	Name	Tests for
`policy_circumvention`	Policy Circumvention	Bypassing operator-defined policies
`capability_abuse`	Capability Abuse	Misusing legitimate capabilities — code generation, summarization, retrieval — for harmful purposes
`jailbreak_generic`	Generic Jailbreak Attempts	Role-play, DAN-style, or character-induction jailbreaks not tied to a specific harm category

Distinct from security_fraud, which concerns attacks on other systems. This category is about abuse of your deployment.

Performance, Reliability & Quality

Category ID: performance_reliability_quality — 4 sub-categories Quality failures where the model produces incorrect, inconsistent, or ill-formed output under non-malicious load.

Sub-category ID	Name	Tests for
`hallucination`	Hallucination & Factual Inaccuracy	Confident factual claims that are demonstrably false, fabricated citations, or invented entities
`dos_sponge`	DoS & Sponge / Resource Exhaustion	Excessive compute consumption — infinite output, recursion loops, rare-unicode expansion, repetitive output
`latency_degradation`	Latency Degradation	Abnormally slow response times without triggering a hard resource cap
`load_resilience`	Load & Resilience Failures	Reliability degradation under concurrent load or adversarial batching

brand_reputation_trust owns brand-specific hallucinations — wrong pricing, fabricated policy, competitor confusion. hallucination here is about general factual reliability.

Compliance frameworks

Instead of naming categories, you can name a framework. The platform expands it into the categories and sub-categories its controls map onto.

{
  "compliance_frameworks": ["owasp_llm_top10", "nist_ai_rmf"]
}

Framework ID	Name	Version	Controls
`owasp_llm_top10`	OWASP LLM Top 10	2025	10
`eu_ai_act`	EU Artificial Intelligence Act	2024/1689	10
`nist_ai_rmf`	NIST AI Risk Management Framework	1.0 (with GAI Profile NIST-AI-600-1)	19
`mitre_atlas`	MITRE ATLAS	Matrix v4.9.0	14
`iso_42001`	ISO/IEC 42001 AI Management Systems	2023	9
`aiuc_1`	AIUC-1 (AI Underwriter Certification)	1.0	6

compliance_frameworks and risk_categories are mutually exclusive. Supply exactly one.

Reading a compliance report

Any finished run can be re-expressed against a framework — including a run you scoped by risk category:

GET /rt/runs/{run_id}/compliance/{framework_id}

The report maps your results onto each control the run covered. See Get Compliance Report.

Where results land

Run results are reported at every level of this taxonomy:

summary.overall_asr — attack success rate across the whole run
summary.categories.{category_id}.asr — per risk category
summary.categories.{category_id}.sub_categories.{sub_id} — per sub-category, with counts
summary.categories.{category_id}.by_attack_method — which techniques worked
summary.categories.{category_id}.by_source — generated prompts vs. curated library

Payload Guide — how to use these IDs
Attack Methods Reference — the techniques applied to each category
Agent Testing Guide — agent-specific configuration
Examples — complete payloads

Get Started for Guardrails

Get Started for Red Team

Get Started with AI Proxy

Get Started with MCP

Get Started with Skill Scanner

On-Prem Installation

Risk Category Catalog

How scoping works

Narrowing with `sub_categories`

Safety & Harm

Privacy & Data Protection

Security & Fraud

Bias, Fairness & Ethical Harm

Legal & IP Liability

Brand, Reputation & Trust

Product Misuse & Abuse

Performance, Reliability & Quality

Compliance frameworks

Reading a compliance report

Where results land

​How scoping works

​Narrowing with sub_categories

​Safety & Harm

​Privacy & Data Protection

​Security & Fraud

​Bias, Fairness & Ethical Harm

​Legal & IP Liability

​Brand, Reputation & Trust

​Product Misuse & Abuse

​Performance, Reliability & Quality

​Compliance frameworks

​Reading a compliance report

​Where results land

​Related

How scoping works

Narrowing with `sub_categories`

Safety & Harm

Privacy & Data Protection

Security & Fraud

Bias, Fairness & Ethical Harm

Legal & IP Liability

Brand, Reputation & Trust

Product Misuse & Abuse

Performance, Reliability & Quality

Compliance frameworks

Reading a compliance report

Where results land

Related