​
Evaluating Model Vulnerabilities Across Different Providers

This tutorial guides you through assessing vulnerabilities in models from various providers using EnkryptAI’s red teaming API. We’ll cover setup and usage for OpenAI, Together, Replicate, Azure ML Studio, and HuggingFace Inference Endpoints.

​
Prerequisites

  • Python programming basics
  • Accounts with the model providers you want to test
  • Python libraries: requests, os, dotenv, json

​
Setup

  1. Install required libraries:
Shell
pip install requests python-dotenv tabulate pandas enkryptai-sdk
  1. Set up EnkryptAI API key:

Create an account at app.enkryptai.com and generate an API key. Then export it as an environment variable:

Shell
export ENKRYPTAI_API_KEY=<your-api-key>

​
Running Red Team Tests

Use the following Python script template to run red team tests. Adjust the payload for each provider:

import requests
import json
import os

url = "https://api.enkryptai.com/redteam/add-task"

payload = json.dumps({
    "test_name": "Test 1",
    "dataset_name": "standard", // Or your custom dataset name
    "redteam_test_configurations": {
        "bias_test": {
            "sample_percentage": 1,
            "attack_methods": {
                "basic": [
                    "basic"
                ]
            }
        },
        "cbrn_test": {
            "sample_percentage": 1,
            "attack_methods": {
                "basic": [
                    "basic"
                ]
            }
        },
        "harmful_test": {
            "sample_percentage": 1,
            "attack_methods": {
                "basic": [
                    "basic"
                ]
            }
        },
        "insecure_code_test": {
            "sample_percentage": 1,
            "attack_methods": {
                "basic": [
                    "basic"
                ]
            }
        },
        "toxicity_test": {
            "sample_percentage": 1,
            "attack_methods": {
                "basic": [
                    "basic"
                ]
            }
        }
    },
    "target_model_configuration": {
        # Provider-specific details go here
    }
})

headers = {
    'Content-Type': 'application/json',
    'apikey': os.getenv('ENKRYPTAI_API_KEY')
}

response = requests.post(url, headers=headers, data=payload)
print(json.dumps(response.json(), indent=4))

​
Provider-Specific Configurations

Replace the target_model_configuration in the payload with the appropriate details for each provider:

​
OpenAI

Python
"target_model_configuration": {
    "testing_for": "LLM",
    "model_name": "gpt-4",
    "model_type": "text_2_text",
    "model_source": "https://platform.openai.com/docs/models",
    "model_provider": "Open AI",
    "model_endpoint_url": "https://api.openai.com/v1/chat/completions",
    "model_api_key": "OPENAI_API_KEY"
}

​
Together

Python
"target_model_configuration": {
    "testing_for": "LLM",
    "model_name": "google/gemma-7b-it",
    "model_type": "text_2_text",
    "model_source": "https://docs.together.ai/docs/inference-models",
    "model_provider": "Google",
    "model_endpoint_url": "https://api.together.xyz/v1/chat/completions",
    "model_api_key": "TOGETHER_API_KEY"
}

​
Replicate

Python
"target_model_configuration": {
    "testing_for": "LLM",
    "model_name": "google/gemma-7b-it",
    "model_type": "text_2_text",
    "model_source": "https://replicate.com/google-deepmind/gemma-7b-it",
    "model_provider": "Google",
    "model_endpoint_url": "https://api.replicate.com/v1/predictions",
    "model_api_key": "REPLICATE_API_KEY"
}

​
Azure ML Studio

Python
"target_model_configuration": {
    "testing_for": "LLM",
    "model_name": "Meta-Llama-3-8B-Instruct",
    "model_type": "text_2_text",
    "model_source": "https://ai.azure.com/explore/models?tid=2fad0b06-ed54-4316-b3b3-79e93b19079b",
    "model_provider": "Meta",
    "model_endpoint_url": "{AZURE_DEPLOYMENT_ID}/v1/chat/completions",
    "model_api_key": "AZURE_ML_STUDIO_API_KEY"
}

​
HuggingFace Inference Endpoint

Python
"target_model_configuration": {
    "testing_for": "LLM",
    "model_name": "google/gemma-7b-it",
    "model_type": "text_2_text",
    "model_source": "https://huggingface.co/inference-endpoints/dedicated",
    "model_provider": "Google",
    "model_endpoint_url": "HF_INFERENCE_ENDPOINT_URL",
    "model_api_key": "HF_ENDPOINT_API_KEY"
}

Replace placeholder API keys and endpoints with your actual credentials for each provider.