Testing Models Across Providers

This tutorial covers how to point a red team run at models hosted by different providers. Only the target block changes between them — the rest of the payload is identical.

Prerequisites

Python basics
Accounts with the model providers you want to test
An Enkrypt AI API key from app.enkryptai.com

Setup

Shell

pip install requests python-dotenv enkryptai-sdk

Shell

export ENKRYPTAI_API_KEY="<your-enkrypt-api-key>"

Export a key for whichever provider you are testing:

Shell

export OPENAI_API_KEY="<your-key>"
export ANTHROPIC_API_KEY="<your-key>"
export TOGETHER_API_KEY="<your-key>"
# ...and so on

The script

This template works for every provider. Swap the target block from the sections below.

import json
import os
import time

import requests

BASE_URL = "https://api.enkryptai.com"
HEADERS = {
    "Content-Type": "application/json",
    "apikey": os.getenv("ENKRYPTAI_API_KEY"),
}

target = {
    "endpoint": "https://api.openai.com/v1/chat/completions",
    "api_key": os.getenv("OPENAI_API_KEY"),
    "model_name": "gpt-4o-mini",
    "provider": "openai",
    "system_type": "foundation_model",
    "input_modalities": ["text"],
    "output_modalities": ["text"],
}

payload = {
    "run_name": "provider-comparison",
    "target": target,
    "risk_categories": {
        "safety_harm": {"attack_config": {"basic": {}}},
        "bias_fairness_ethical_harm": {"attack_config": {"basic": {}}},
    },
    "generation_config": {"max_prompts": 30},
}

# Submit
run = requests.post(f"{BASE_URL}/rt/redteam", headers=HEADERS, data=json.dumps(payload)).json()
run_id = run["run_id"]
print("run_id:", run_id)

# Wait
while True:
    status = requests.get(f"{BASE_URL}/rt/runs/{run_id}", headers=HEADERS).json()
    if status["status"] in ("Finished", "Failed"):
        break
    time.sleep(30)

# Read
results = requests.get(f"{BASE_URL}/rt/runs/{run_id}/results", headers=HEADERS).json()
print(json.dumps(results["summary"], indent=2))

import os
import time

from dotenv import load_dotenv
from enkryptai_sdk import RedTeamClient

load_dotenv()

client = RedTeamClient(
    api_key=os.getenv("ENKRYPTAI_API_KEY"),
    base_url=os.getenv("ENKRYPTAI_BASE_URL") or "https://api.enkryptai.com",
)

target = {
    "endpoint": "https://api.openai.com/v1/chat/completions",
    "api_key": os.getenv("OPENAI_API_KEY"),
    "model_name": "gpt-4o-mini",
    "provider": "openai",
    "system_type": "foundation_model",
}

run = client.run_redteam(config={
    "run_name": "provider-comparison",
    "target": target,
    "risk_categories": {
        "safety_harm": {"attack_config": {"basic": {}}},
        "bias_fairness_ethical_harm": {"attack_config": {"basic": {}}},
    },
    "generation_config": {"max_prompts": 30},
})

run_id = run.run_id

while True:
    status = client.get_run_status(run_id=run_id)
    if status.status in ("Finished", "Failed"):
        break
    time.sleep(30)

print(client.get_run_results(run_id=run_id).to_dict())

Before spending a run, check the target is reachable with POST /rt/model-health. It calls the target once and returns the probe query and the reply. It is free.

Provider configurations

OpenAI

{
  "endpoint": "https://api.openai.com/v1/chat/completions",
  "api_key": "sk-...",
  "model_name": "gpt-4o",
  "provider": "openai",
  "input_modalities": ["text", "image"],
  "output_modalities": ["text"]
}

Anthropic

{
  "endpoint": "https://api.anthropic.com/v1/messages",
  "api_key": "sk-ant-...",
  "model_name": "claude-sonnet-4-5",
  "provider": "anthropic",
  "input_modalities": ["text", "image"],
  "output_modalities": ["text"]
}

Azure OpenAI

{
  "endpoint": "https://<your-resource>.openai.azure.com/openai/deployments/<deployment>/chat/completions?api-version=2024-10-21",
  "api_key": "<azure-key>",
  "model_name": "<deployment-name>",
  "provider": "azure_openai",
  "input_modalities": ["text"],
  "output_modalities": ["text"]
}

For Azure the model_name is your deployment name, not the underlying model name, and the API version belongs in the endpoint query string.

AWS Bedrock

{
  "endpoint": "https://bedrock-runtime.us-east-1.amazonaws.com",
  "api_key": "<aws-credential>",
  "model_name": "anthropic.claude-3-5-sonnet-20241022-v2:0",
  "provider": "bedrock",
  "input_modalities": ["text"],
  "output_modalities": ["text"]
}

Google Gemini

{
  "endpoint": "https://generativelanguage.googleapis.com/v1beta/models",
  "api_key": "<google-api-key>",
  "model_name": "gemini-2.0-flash",
  "provider": "gemini",
  "input_modalities": ["text", "image"],
  "output_modalities": ["text"]
}

Together

{
  "endpoint": "https://api.together.xyz/v1",
  "api_key": "<together-key>",
  "model_name": "meta-llama/Llama-3.3-70B-Instruct-Turbo",
  "provider": "together",
  "input_modalities": ["text"],
  "output_modalities": ["text"]
}

Replicate

{
  "endpoint": "https://api.replicate.com/v1/predictions",
  "api_key": "<replicate-token>",
  "model_name": "meta/meta-llama-3-70b-instruct",
  "provider": "replicate",
  "input_modalities": ["text"],
  "output_modalities": ["text"]
}

HuggingFace Inference Endpoint

{
  "endpoint": "https://<your-endpoint>.endpoints.huggingface.cloud/v1/chat/completions",
  "api_key": "hf_...",
  "model_name": "<model-id>",
  "provider": "hf_endpoint",
  "input_modalities": ["text"],
  "output_modalities": ["text"]
}

Any OpenAI-compatible endpoint

The general escape hatch — vLLM, Ollama, LM Studio, OpenRouter, Groq, Fireworks, self-hosted gateways and most other services expose an OpenAI-shaped API.

{
  "endpoint": "https://your-gateway.example.com/v1",
  "api_key": "...",
  "model_name": "your-org/your-model",
  "provider": "openai_compatible",
  "input_modalities": ["text"],
  "output_modalities": ["text"]
}

provider is a routing hint. When in doubt, use openai_compatible — if the endpoint speaks the OpenAI chat-completions shape, it will work. Other recognised values include mistral, cohere, deepseek, groq, fireworks, openrouter, xai, ai21, ollama, litellm and portkey.

Saved models

Rather than repeating a target block, save the model once through the Models API and reference it:

Shell

curl -X POST "https://api.enkryptai.com/rt/model/redteam" \
  -H "apikey: $ENKRYPTAI_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Enkrypt-Model: my-saved-model" \
  -H "X-Enkrypt-Model-Version: v1" \
  -d '{
    "risk_categories": { "safety_harm": { "attack_config": { "basic": {} } } },
    "generation_config": { "max_prompts": 30 }
  }'

The payload carries no target at all — the saved model supplies it. See Run Red Team with a Saved Model for the exact headers.

Targets inside a private network

If the model is not reachable from the public internet, route it through the Relay bridge with connect_via_relay on the target and a metadata.relay block naming the bridge. See Red Teaming Through the Relay Bridge for the end-to-end walkthrough, and Configuration Reference for the field reference.

Comparing providers

Because only target changes, you can run the same scope against several models and compare summary.overall_asr:

Python

targets = {
    "gpt-4o-mini": {
        "endpoint": "https://api.openai.com/v1/chat/completions",
        "api_key": os.getenv("OPENAI_API_KEY"),
        "model_name": "gpt-4o-mini",
        "provider": "openai",
    },
    "claude-sonnet-4-5": {
        "endpoint": "https://api.anthropic.com/v1/messages",
        "api_key": os.getenv("ANTHROPIC_API_KEY"),
        "model_name": "claude-sonnet-4-5",
        "provider": "anthropic",
    },
}

scope = {
    "risk_categories": {"safety_harm": {"attack_config": {"basic": {}, "encoding": {}}}},
    "generation_config": {"max_prompts": 50},
}

run_ids = {}
for label, target in targets.items():
    body = {"run_name": f"compare-{label}", "target": target, **scope}
    resp = requests.post(f"{BASE_URL}/rt/redteam", headers=HEADERS, data=json.dumps(body))
    run_ids[label] = resp.json()["run_id"]

# ...wait for all to finish, then:
for label, run_id in run_ids.items():
    results = requests.get(f"{BASE_URL}/rt/runs/{run_id}/results", headers=HEADERS).json()
    print(label, results["summary"]["overall_asr"])

For a like-for-like comparison, use the same generation_config.dataset_name across runs so every model sees an identical prompt set. Generate it once with threat modeling.

Quickstart — the full run cycle
Configuration Reference — every target field
Examples — complete payloads
Model Health — verify a target before running

Get Started for Guardrails

Get Started for Red Team

Get Started with AI Proxy

Get Started with MCP

Get Started with Skill Scanner

On-Prem Installation

Testing Models Across Providers

Prerequisites

Setup

The script

Provider configurations

OpenAI

Anthropic

Azure OpenAI

AWS Bedrock

Google Gemini

Together

Replicate

HuggingFace Inference Endpoint

Any OpenAI-compatible endpoint

Saved models

Targets inside a private network

Comparing providers

​Prerequisites

​Setup

​The script

​Provider configurations

​OpenAI

​Anthropic

​Azure OpenAI

​AWS Bedrock

​Google Gemini

​Together

​Replicate

​HuggingFace Inference Endpoint

​Any OpenAI-compatible endpoint

​Saved models

​Targets inside a private network

​Comparing providers

​Related

Prerequisites

Setup

The script

Provider configurations

OpenAI

Anthropic

Azure OpenAI

AWS Bedrock

Google Gemini

Together

Replicate

HuggingFace Inference Endpoint

Any OpenAI-compatible endpoint

Saved models

Targets inside a private network

Comparing providers

Related