Current features

Standard Tests for everyone

Bias Exploitation

bias_test - Identifying and exposing biased outputs.

CBRN Exploitation

cbrn_test - Addressing vulnerabilities related to chemical, biological, radiological, and nuclear domains.

Harmful Exploitation

harmful_test - Eliciting responses that promote harm or danger.

Insecure Code Exploitation

insecure_code_test - Producing insecure or harmful code snippets.

Toxicity Exploitation

toxicity_test - Generating harmful or offensive content.

PII Exploitation

pii_test - Exposing personally identifiable information.

copyright_test - Exposing copyrighted material.

Misinformation Exploitation

misinformation_test - Exposing misinformation.

System Prompt Extractions Exploitation

system_prompt_extractions_test - Exposing system prompt extractions.

Sponge Exploitation

sponge_test - Exposing infinite loops.

Competitor Exploitation

competitor_test - Exposing information about competitors.

Custom Tests for generated datasets

custom_test - Custom dataset test.

Specialized Tests for generated datasets

Adversarial Bias Exploitation

adv_bias_test - Uncovering biased outputs through adversarial methods.

Adversarial Information Exploitation

adv_info_test - Extracting sensitive or unintended information from a generated dataset.

Adversarial Tool Exploitation

adv_tool_test - Misusing integrated tools or features.

Adversarial Command Exploitation

adv_command_test - Manipulating the model to execute unintended commands.

Adversarial PII Exploitation

adv_pii_test - Exposing personally identifiable information.

Adversarial Competitor Exploitation

adv_competitor_test - Extracting confidential information about competitors.

Agentic Tests

Alignment and Governance

alignment_and_governance_test - Testing the alignment and governance of the model.

Input and Content Integrity

input_and_content_integrity_test - Testing the integrity of the input and content.

Infrastructure and Integration

infrastructure_and_integration_test - Testing the infrastructure and integration of the model.

Security and Privacy

security_and_privacy_test - Testing the security and privacy of the model.

Human Factors and Societal Impact

human_factors_and_societal_impact_test - Testing the human factors and societal impact of the model.

Access Control

access_control_test - Testing the access control and permissions of the model.

Physical and Actuation Safety

physical_and_actuation_safety_test - Testing the physical and actuation safety of the model.

Reliability and Monitoring

reliability_and_monitoring_test - Testing the reliability and monitoring of the model.

Governance

governance_test - Testing the governance of the model.

Agent Output Quality

agent_output_quality_test - Testing the quality of the agent’s output.

Tool Misuse

tool_misuse_test - Testing the misuse of the model’s tools.

Privacy

privacy_test - Testing the privacy of the model.

Reliability and Observability

reliability_and_observability_test - Testing the reliability and observability of the model.

Agent Behaviour

agent_behaviour_test - Testing the behaviour of the agent.

Access Control and Permissions

access_control_and_permissions_test - Testing the access control and permissions of the model.

Tool Extraction

tool_extraction_test - Test if the agent tool infromation can be extracted in outputs.