Tutorials
RedTeam Models
Evaluating Model Vulnerabilities Across Different Providers
This tutorial guides you through assessing vulnerabilities in models from various providers using EnkryptAI’s red teaming API. We’ll cover setup and usage for OpenAI, Anyscale, Together, Replicate, Azure ML Studio, and HuggingFace Inference Endpoints.
Prerequisites
- Python programming basics
- Accounts with the model providers you want to test
- Python libraries: requests, os, dotenv, json
Setup
- Install required libraries:
pip install requests python-dotenv
- Set up EnkryptAI API key:
Create an account at app.enkryptai.com and generate an API key. Then export it as an environment variable:
export ENKRYPTAI_API_KEY=<your-api-key>
Running Red Team Tests
Use the following Python script template to run red team tests. Adjust the payload
for each provider:
import requests
import json
import os
url = "https://api.enkryptai.com/redteam/add-task"
payload = json.dumps({
"redteam_test_configurations": {
"jailbreak": {
"sample_percentage": 5,
"test_methods": ["single_shot", "iterative"]
},
"malware": {
"sample_percentage": 5,
"test_methods": ["malware"]
},
"toxicity": {
"sample_percentage": 5,
"test_methods": ["real_toxic_prompts"]
},
"bias": {
"sample_percentage": 5,
"test_methods": ["implicit_word_test", "implicit_sentence_test"]
}
},
"target_model_configuration": {
# Provider-specific details go here
}
})
headers = {
'Content-Type': 'application/json',
'api_key': os.getenv('ENKRYPTAI_API_KEY')
}
response = requests.post(url, headers=headers, data=payload)
print(json.dumps(response.json(), indent=4))
Provider-Specific Configurations
Replace the target_model_configuration
in the payload with the appropriate details for each provider:
OpenAI
"target_model_configuration": {
"model_name": "gpt-4",
"model_access_method": "OpenAI Endpoint",
"model_type": "text_2_text",
"model_source": "https://platform.openai.com/docs/models",
"model_provider": "Open AI",
"model_endpoint_url": "https://api.openai.com/v1/chat/completions",
"model_api_key": "OPENAI_API_KEY"
}
Anyscale
"target_model_configuration": {
"model_name": "google/gemma-7b-it",
"model_access_method": "Anyscale Endpoint",
"model_type": "text_2_text",
"model_source": "https://docs.anyscale.com",
"model_provider": "Google",
"model_endpoint_url": "https://api.endpoints.anyscale.com/v1/chat/completions",
"model_api_key": "ANYSCALE_API_KEY"
}
Together
"target_model_configuration": {
"model_name": "google/gemma-7b-it",
"model_access_method": "Together Endpoint",
"model_type": "text_2_text",
"model_source": "https://docs.together.ai/docs/inference-models",
"model_provider": "Google",
"model_endpoint_url": "https://api.together.xyz/v1/chat/completions",
"model_api_key": "TOGETHER_API_KEY"
}
Replicate
"target_model_configuration": {
"model_name": "google/gemma-7b-it",
"model_access_method": "Replicate Endpoint",
"model_type": "text_2_text",
"model_source": "https://replicate.com/google-deepmind/gemma-7b-it",
"model_provider": "Google",
"model_endpoint_url": "https://api.replicate.com/v1/predictions",
"model_api_key": "REPLICATE_API_KEY"
}
Azure ML Studio
"target_model_configuration": {
"model_name": "Meta-Llama-3-8B-Instruct",
"model_access_method": "Azure ML Studio Endpoint",
"model_type": "text_2_text",
"model_source": "https://ai.azure.com/explore/models?tid=2fad0b06-ed54-4316-b3b3-79e93b19079b",
"model_provider": "Meta",
"model_endpoint_url": "{AZURE_DEPLOYMENT_ID}/v1/chat/completions",
"model_api_key": "AZURE_ML_STUDIO_API_KEY"
}
HuggingFace Inference Endpoint
"target_model_configuration": {
"model_name": "google/gemma-7b-it",
"model_access_method": "HF Inference Endpoint",
"model_type": "text_2_text",
"model_source": "https://huggingface.co/inference-endpoints/dedicated",
"model_provider": "Google",
"model_endpoint_url": "HF_INFERENCE_ENDPOINT_URL",
"model_api_key": "HF_ENDPOINT_API_KEY"
}
Replace placeholder API keys and endpoints with your actual credentials for each provider.