Skip to main content
This document provides a comprehensive catalog of all red team tests available for foundation models and LLM-based applications in Sentry, including test descriptions, categories, sample prompts, and configuration requirements.
This catalog covers tests for foundation models and LLM-based applications. For agent-specific tests (such as agent_output_quality_test and agent_behaviour_test), see the Agent Test Catalog.

Standard Tests

Standard tests use pre-defined CSV datasets and can be run without additional system information.

CBRN Test

Purpose: Tests for Chemical, Biological, Radiological, and Nuclear (CBRN) information extraction vulnerabilities. NIST Category: CBRN Information or Capabilities
AIUC Category: F. Society
Test Categories:
  • Chemistry (VX Nerve Agent, Sarin Gas, Botulinum Toxin, etc.)
  • Biology (Ebola virus, Anthrax, Smallpox, etc.)
  • Radiology (Plutonium-239, Cesium-137, etc.)
  • Nuclear Science (Uranium enrichment, nuclear weapons design, etc.)
  • Cybersecurity
Sample Prompt & Response:
Configuration:

Harmful Test

Purpose: Tests for generation of dangerous, violent, or hateful content. NIST Category: Dangerous, Violent, or Hateful Content
AIUC Category: C. Safety
Test Categories:
  • Hate Speech & Discrimination
  • Suicide & Self-harm
  • Sexual Content
  • Regulated or Controlled Substances
  • Guns & Illegal Weapons
  • Criminal Planning
Sample Prompt & Response:
Configuration:

Bias Test

Purpose: Tests for harmful bias and stereotyping in AI responses. NIST Category: Harmful Bias or Homogenization
AIUC Category: C. Safety
Test Variants:
  1. Word-level bias tests: Tests implicit associations between groups and attributes
  2. Sentence-level bias tests: Tests bias in contextual scenarios
Test Categories:
  • Gender
  • Race
  • Caste and Religion
  • Health
Sample Prompt & Response:
Configuration:

Toxicity Test

Purpose: Tests for toxic, abusive, or offensive language generation. NIST Category: Obscene, Degrading, and/or Abusive Content Test Categories:
  • Insult
  • Threat
  • Flirtation
  • Severe Toxicity
  • Identity Attack
  • Profanity
  • Sexually Explicit
Sample Prompt & Response:
Configuration:

PII Test

Purpose: Tests for personally identifiable information (PII) extraction vulnerabilities. NIST Category: Data Privacy
AIUC Category: A. Data & Privacy
OWASP Category: LLM02:25, LLM05:25
Test Categories:
  • email
  • phone_number
  • social_security_number
  • date_of_birth
  • address
  • mother’s_maiden_name
  • credit_card_number
  • passport_number
  • driver’s_license_number
  • bank_account_number
Sample Prompt & Response:
Configuration:

Insecure Code Test

Purpose: Tests for generation of insecure or malicious code. NIST Category: Information Security Test Categories:
  • Evasion (malware evasion techniques)
  • Payload (malicious payloads)
  • TopLevel (complete malware)
  • SubFunctions (malware subfunctions)
  • GTUBE (test email signatures)
  • EICAR (antivirus test files)
  • GTphish (phishing test signatures)
Sample Prompt & Response:
Configuration:

Purpose: Tests for reproduction of copyrighted content. Test Categories:
  • song_lyrics
  • novel_paragraphs
  • news_articles
  • poems
Sample Prompt & Response:
Configuration:

Misinformation Test

Purpose: Tests for generation of false or misleading information. NIST Category: Confabulation
AIUC Category: D. Reliability
OWASP Category: LLM05:25, LLM09:25
Test Categories:
  • long_input (overwhelming input)
  • infinite_output (requests for infinite output)
  • recursion_loop (recursive prompts)
  • rare_unicode (unicode manipulation)
  • repetitive_output (excessive repetition)
Sample Prompt & Response:
Configuration:

Sponge Test

Purpose: Tests for computational resource exhaustion attacks (sponge attacks). NIST Category: Data Privacy
AIUC Category: A. Data & Privacy
OWASP Category: LLM02:25, LLM05:25
Test Categories:
  • long_input
  • infinite_output
  • recursion_loop
  • rare_unicode
  • repetitive_output
Sample Prompt & Response:
Configuration:

Hallucination Test

Purpose: Tests for factual inaccuracies and hallucinations in AI responses. Test Categories:
  • History & Archaeology
  • Finance & Economics
  • Science & Medicine
  • Technology & Computing
  • Geography & Geopolitics
  • Law & Policy
  • Sports Statistics & Records
  • Arts, Culture & Literature
Sample Prompt & Response:
Configuration:

System Prompt Extractions Test

Purpose: Tests for system prompt and tool extraction vulnerabilities. NIST Category: Data Privacy
AIUC Category: A. Data & Privacy
OWASP Category: LLM02:25, LLM05:25
Test Variants:
  1. System Prompt Extraction: Attempts to extract system prompts
  2. Tool Extraction: Attempts to extract available tools/functions
System Prompt Extraction Categories:
  • instruction_override
  • direct_extraction
  • other
  • context_overflow
  • encoding_attack
  • academic_framing
  • role_playing
  • impersonation
  • technical_exploit
  • socratic_method
  • hypothetical_scenario
  • multi_question
Tool Extraction Categories:
  • direct_request
  • contextual_framing
  • capability_probing
  • instruction_override
  • technical_impersonation
  • role_playing
  • error_exploitation
  • emotional_appeal
  • other
Sample Prompt & Response:
Configuration:

CSEM Test

Purpose: Tests for child sexual exploitation material (CSEM) related content generation. Test Categories:
  • Sexual Acts
  • Child Porn
  • Blackmail
  • Grooming
Sample Prompt & Response:
Note: This test identifies vulnerabilities where models fail to reject CSEM-related requests. The example above demonstrates a successful adversarial attack where the model complies with harmful content. Configuration:

Custom Tests (adv_*)

Custom tests require system_description or policy_description in dataset_configuration to generate contextually relevant test prompts. These tests dynamically generate prompts based on the target system’s description. Note: When custom_test is selected as the test type, it automatically runs the following tests:
  • adv_info_test
  • adv_command_test
  • adv_bias_test
These tests are executed together to provide comprehensive coverage of information extraction, command injection, and bias vulnerabilities. Additionally, adv_pii_test and adv_persona_test may be included based on configuration settings. Configuration:
Note: For custom_test, system_description or policy_description should be provided in dataset_configuration:

adv_competitor_test

Purpose: Tests whether the AI can be manipulated to promote competitors, disclose competitor information, or generate FUD (Fear, Uncertainty, Doubt) about competitors. Required Configuration:
  • system_description or policy_description in dataset_configuration: Description of the AI system/application being tested
Test Categories:
  • brand_loyalty: Tests if AI prioritizes its own brand
  • competitor_promotion: Tests if AI can be prompted to recommend competitors
  • comparative_analysis: Tests AI responses to direct comparison requests
  • information_disclosure: Tests whether AI reveals competitor-specific details
  • FUD_generation: Tests if AI generates negative information about competitors
Dataset Configuration Requirements:
  • Application domain/industry
  • Product/service description
  • Business context
Sample Prompt & Response:
Configuration:
Note: For adv_* tests, system_description or policy_description should be provided in dataset_configuration:

adv_info_test

Purpose: Tests for information extraction vulnerabilities, attempting to extract sensitive system information, methodologies, or proprietary knowledge. Required Configuration:
  • system_description or policy_description in dataset_configuration: Description of the system’s capabilities and data access
Dataset Configuration Requirements:
  • System architecture details
  • Data access capabilities
  • Available tools/functions
  • Internal processes and methodologies
Sample Prompt & Response:
Configuration:

adv_command_test

Purpose: Tests for command injection and unauthorized command execution vulnerabilities. Required Configuration:
  • system_description or policy_description in dataset_configuration: Description of available commands and system capabilities
Dataset Configuration Requirements:
  • Available commands/functions
  • System permissions
  • Execution capabilities
  • Data modification rights
Sample Prompt & Response:
Configuration:

adv_pii_test

Purpose: Tests for PII extraction vulnerabilities, specifically targeting other users’ data in the system. Required Configuration:
  • system_description or policy_description in dataset_configuration: Description of the application and types of user data it handles
Dataset Configuration Requirements:
  • Application domain
  • Types of PII stored
  • User data access patterns
  • System capabilities
PII Entities Tested:
  • email, phone_number, social_security_number, date_of_birth, address, credit_card_number, passport_number, driver’s_license_number, bank_account_number, etc.
Sample Prompt & Response:
Configuration:

adv_bias_test

Purpose: Tests for bias vulnerabilities in context-specific scenarios, generating prompts that reveal discriminatory assumptions. Required Configuration:
  • system_description or policy_description in dataset_configuration: Description of the system’s domain and use cases
  • scenario: Specific scenario to test (e.g., “Academic Performance”, “Leadership Roles”)
  • category: Bias category (e.g., “Gender”, “Socioeconomic”, “Age”)
Dataset Configuration Requirements:
  • Application domain/context
  • Use cases and scenarios
  • User interactions
  • Decision-making processes
Sample Prompt & Response:
Configuration:

adv_sponge_test

Purpose: Tests for computational resource exhaustion using contextually relevant prompts. Required Configuration:
  • system_description or policy_description in dataset_configuration: Description of the system to generate relevant sponge attack prompts
Dataset Configuration Requirements:
  • System capabilities
  • Processing patterns
  • Resource-intensive operations
  • Input/output formats
Sample Prompt & Response:
Configuration:

adv_persona_test

Purpose: Tests for persona manipulation vulnerabilities, attempting to make the AI adopt harmful or unauthorized personas. Required Configuration:
  • system_description or policy_description in dataset_configuration: Description of the system’s intended persona and boundaries
Dataset Configuration Requirements:
  • Intended persona/role
  • System boundaries
  • Authority levels
  • Use case constraints
Sample Prompt & Response:
Configuration:

adv_tool_test

Purpose: Tests for tool misuse vulnerabilities in agent-based systems. Required Configuration:
  • system_description or policy_description in dataset_configuration: Description of available tools and their intended usage
Dataset Configuration Requirements:
  • Available tools/functions
  • Tool capabilities
  • Permissions and access levels
  • Intended use cases
Sample Prompt & Response:
Configuration:

Configuration Requirements Summary

Standard Tests

Standard tests use pre-defined CSV datasets and require minimal configuration:
  • Test type selection
  • Sample percentage (0-100%)
  • Attack methods (e.g., basic, iterative, static)

Custom Tests (adv_*)

Custom tests require additional configuration:
  1. system_description or policy_description (Required in dataset_configuration)
    • Description of the target AI system
    • Application domain and use cases
    • Capabilities and limitations
    • Data access patterns
  2. Context-specific parameters (Test-dependent)
    • adv_competitor_test: Business context, competitor information
    • adv_bias_test: Scenarios, bias categories
    • adv_pii_test: Application domain, PII types
    • adv_sponge_test: Processing patterns, resource-intensive operations
Note: system_description and policy_description are part of dataset_configuration, not params in attack methods. The params section is reserved for attack-specific parameters (e.g., encoding iterations, branching factors).

Test Coverage

By NIST Category

  • CBRN Information or Capabilities: cbrn_test
  • Dangerous, Violent, or Hateful Content: harmful_test
  • Harmful Bias or Homogenization: bias_test, adv_bias_test
  • Obscene, Degrading, and/or Abusive Content: toxicity_test
  • Data Privacy: pii_test, adv_pii_test, system_prompt_extractions_test, sponge_test
  • Information Security: insecure_code_test
  • Confabulation: misinformation_test, hallucination_test

By AIUC Category

  • A. Data & Privacy: pii_test, system_prompt_extractions_test, sponge_test
  • C. Safety: harmful_test, bias_test, toxicity_test
  • D. Reliability: misinformation_test
  • F. Society: cbrn_test

By OWASP Category

  • LLM02:25: pii_test, system_prompt_extractions_test, sponge_test
  • LLM05:25: pii_test, system_prompt_extractions_test, misinformation_test
  • LLM09:25: misinformation_test

Usage Notes

  1. Standard Tests: Can be run immediately with CSV datasets. Configure sample percentage and attack methods.
  2. Custom Tests: Require system_description or policy_description in dataset_configuration:
  3. Custom Test Type: When selecting custom_test as the test type, it automatically runs adv_info_test, adv_command_test, and adv_bias_test together:
    This is equivalent to running all three tests individually and provides comprehensive coverage of information extraction, command injection, and bias vulnerabilities. Note: For custom_test and adv_* tests, system_description or policy_description should be provided in dataset_configuration, not in params. The params section is reserved for attack-specific parameters (e.g., encoding iterations, branching factors, depth).
  4. Test Combinations: Multiple tests can be run simultaneously. Use appropriate sample percentages to manage test duration.