Model Health (V3) - Enkrypt AI

Authorizations

apikey

string

header

required

Body

application/json

endpoint_configuration

ModelCustomRequestObject · object

required

Show child attributes

endpoint_configuration.model_config

Model Config · object

required

Show child attributes

endpoint_configuration.model_config.model_provider

enum<string>

required

Provider of the model which determines the request response format

Available options:

openai,

together,

huggingface,

groq,

azure_openai,

anthropic,

cohere,

bedrock,

gemini,

ai21,

fireworks,

alibaba,

portkey,

deepseek,

mistral,

llama,

openai_compatible,

cohere_compatible,

anthropic_compatible,

custom,

url,

enkryptai,

sierra,

boxai,

nutanix,

xactly

Example:

"together"

endpoint_configuration.model_config.hosting_type

enum<string>

Hosting type of the model

Available options:

External,

Internal

Example:

"External"

endpoint_configuration.model_config.model_source

string

Source of the model

Example:

"https://together.ai"

endpoint_configuration.model_config.tools

Tool · object[]

Array of tools available to the model

Show child attributes

endpoint_configuration.model_config.tools.name

string

required

Name of the tool

Example:

"web_search"

endpoint_configuration.model_config.tools.description

string

required

Description of what the tool does

Example:

"The tool web search is used to search the web for information related to finance."

Example:

[
  {
    "name": "web_search",
    "description": "The tool web search is used to search the web for information related to finance."
  }
]

endpoint_configuration.model_config.input_modalities

enum<string>[]

Types of input that the model can process

Available options:

text,

image,

audio

Example:

["text"]

endpoint_configuration.model_config.output_modalities

enum<string>[]

Types of output that the model can generate

Available options:

text

Example:

["text"]

endpoint_configuration.model_config.rate_per_min

number

< 100 won't enable async. > 100 will enable async mode. > 200 we can run boosted async (all tests in parallel). Default 20.

Example:

20

endpoint_configuration.model_config.system_prompt

string

System prompt

Example:

""

endpoint_configuration.model_config.endpoint

Endpoint · object

Show child attributes

endpoint_configuration.model_config.endpoint.scheme

string

required

Scheme of the endpoint

Example:

"https"

endpoint_configuration.model_config.endpoint.host

string

required

Host of the endpoint

Example:

"api.together.xyz"

endpoint_configuration.model_config.endpoint.port

integer

required

Port of the endpoint

Example:

443

endpoint_configuration.model_config.endpoint.base_path

string

required

Base path of the endpoint

Example:

"/v1"

endpoint_configuration.model_config.paths

Paths · object

Show child attributes

endpoint_configuration.model_config.paths.completions

string

Example:

"/completions"

endpoint_configuration.model_config.paths.chat

string

Example:

"/chat/completions"

endpoint_configuration.model_config.auth_data

Auth Data · object

Show child attributes

endpoint_configuration.model_config.auth_data.header_name

string

Example:

"Authorization"

endpoint_configuration.model_config.auth_data.header_prefix

string

Example:

"Bearer"

endpoint_configuration.model_config.auth_data.space_after_prefix

boolean

Example:

true

endpoint_configuration.model_config.auth_data.param_name

string

Example:

"token"

endpoint_configuration.model_config.auth_data.azure_client_id

string

Example:

"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"

endpoint_configuration.model_config.auth_data.azure_client_secret

string

Example:

"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

endpoint_configuration.model_config.auth_data.azure_tenant_id

string

Example:

"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"

endpoint_configuration.model_config.auth_data.gcp_service_account_json

string

Example:

"{\"type\":\"service_account\",\"project_id\":\"xxxxxxxx\",\"private_key_id\":\"xxxxxxxx\",\"private_key\":\"xxxxxxxx\",\"client_email\":\"xxxxxxxx\",\"client_id\":\"xxxxxxxx\",\"auth_uri\":\"https://accounts.google.com/o/oauth2/auth\",\"token_uri\":\"https://oauth2.googleapis.com/token\",\"auth_provider_x509_cert_url\":\"https://www.googleapis.com/oauth2/v1/certs\",\"client_x509_cert_url\":\"https://www.googleapis.com/robot/v1/metadata/x509/[email protected]\"}"

endpoint_configuration.model_config.auth_data.aws_access_key_id

string

Example:

"xxxxxxxx"

endpoint_configuration.model_config.auth_data.aws_secret_access_key

string

Example:

"xxxxxxxx"

endpoint_configuration.model_config.auth_data.allow_override

boolean

If enabled, the authorization header or parameter can be overridden in the request by the value configured here

Example:

false

endpoint_configuration.model_config.auth_data.boxai_client_id

string

BoxAI client ID for authentication

Example:

"BOX_CLIENT_ID"

endpoint_configuration.model_config.auth_data.boxai_client_secret

string

BoxAI client secret for authentication

Example:

"BOX_CLIENT_SECRET"

endpoint_configuration.model_config.auth_data.boxai_user_id

string

BoxAI user ID for authentication

Example:

"BOX_USER_ID"

endpoint_configuration.model_config.auth_data.boxai_default_file_id

string

BoxAI default file ID for operations

Example:

"BOX_FILE_ID"

endpoint_configuration.model_config.auth_data.nutanix_user_session_id

string

Nutanix user session ID for authentication

Example:

"NUTANIX_USER_SESSION_ID"

endpoint_configuration.model_config.auth_data.nutanix_email_id

string

Nutanix email ID for authentication

Example:

"NUTANIX_EMAIL_ID"

endpoint_configuration.model_config.auth_data.xactly_username

string

Xactly username for authentication

Example:

"XACTLY_USERNAME"

endpoint_configuration.model_config.auth_data.xactly_password

string

Xactly password for authentication

Example:

"XACTLY_PASSWORD"

endpoint_configuration.model_config.apikeys

string[]

Example:

["TOGETHER_AI_API_KEY"]

endpoint_configuration.model_config.model_auth_type

enum<string>

Available options:

apikey,

jwt

Example:

"apikey"

endpoint_configuration.model_config.model_jwt_config

Model Custom Auth Token Config · object

Show child attributes

endpoint_configuration.model_config.model_jwt_config.jwt_method

enum<string>

Available options:

POST,

GET

Example:

"POST"

endpoint_configuration.model_config.model_jwt_config.jwt_url

string

Example:

"https://example.com/api/auth"

endpoint_configuration.model_config.model_jwt_config.jwt_headers

Custom Token Headers · object[]

Array of custom headers to be sent with token requests

Show child attributes

endpoint_configuration.model_config.model_jwt_config.jwt_headers.key

string

required

The name/key of the custom header

Example:

"x-custom-header"

endpoint_configuration.model_config.model_jwt_config.jwt_headers.value

string

required

The value of the custom header

Example:

"custom-value"

Example:

[
  {
    "key": "content-type",
    "value": "application/x-www-form-urlencoded"
  },
  {
    "key": "x-custom-header",
    "value": "custom-value"
  }
]

endpoint_configuration.model_config.model_jwt_config.jwt_body

string

The body of the token request

Example:

"grant_type=client_credentials&client_id=xxxxxxxxxx&client_secret=xxxxxxxxxx"

endpoint_configuration.model_config.model_jwt_config.jwt_response_key

string

The key of the token in the response in jq format

Example:

".access_token"

endpoint_configuration.model_config.custom_curl_command

string

Example:

"curl --location 'https://example.com/api/chat' \\\n--header 'Accept: application/json, text/plain, */*' \\\n--header 'Accept-Language: en-US,en;q=0.9' \\\n--header 'Authorization: {token}' \\\n--header 'Connection: keep-alive' \\\n--header 'Content-Type: application/json' \\\n--data '{\"prompt\":\"{prompt}\"}'"

endpoint_configuration.model_config.custom_headers

Custom Headers · object[]

Array of custom headers to be sent with requests

Show child attributes

endpoint_configuration.model_config.custom_headers.key

string

required

The name/key of the custom header

Example:

"X-Custom-Header"

endpoint_configuration.model_config.custom_headers.value

string

required

The value of the custom header

Example:

"custom-value"

Example:

[
  {
    "key": "X-Custom-Header",
    "value": "custom-value"
  }
]

endpoint_configuration.model_config.custom_payload

Custom Payload · object

A flexible object that can contain any custom key-value pairs for the request payload. Only condition is to include {prompt} in the payload.

Example:

{
  "messages": [
    {
      "role": "developer",
      "content": "You are a helpful assistant."
    },
    { "role": "user", "content": "{prompt}" }
  ]
}

endpoint_configuration.model_config.custom_response_content_type

enum<string>

Content type of the custom response. Currently only supports JSON.

Available options:

json

Example:

"json"

endpoint_configuration.model_config.custom_response_format

string

Optional response format path. Jq format for json type (e.g. '.choices[0].message.content')

Example:

".choices[0].message.content"

endpoint_configuration.model_config.metadata

Metadata · object

Show child attributes

endpoint_configuration.model_config.metadata.max_tokens

integer

Example:

2048

endpoint_configuration.model_config.metadata.input_cost_1M_tokens

number

Example:

2.5

endpoint_configuration.model_config.metadata.output_cost_1M_tokens

number

Example:

10

endpoint_configuration.model_config.metadata.azure_instance

string

If Azure, it's instance type

Example:

"enkrypt2024"

endpoint_configuration.model_config.metadata.azure_api_version

string

If Azure, it's API version

Example:

"2024-02-01"

endpoint_configuration.model_config.metadata.azure_deployment_id

string

If Azure, it's deployment ID

Example:

"gpt3"

endpoint_configuration.model_config.metadata.anthropic_version

string

If Anthropic, it's version

Example:

""

endpoint_configuration.model_config.metadata.llama2_format

enum<string>

If Llama2, it's format

Available options:

openai

Example:

"openai"

endpoint_configuration.model_config.metadata.mistral_format

enum<string>

If Mistral, it's format

Available options:

openai,

ollama

Example:

"openai"

endpoint_configuration.model_config.metadata.gemini

Gemini · object

Show child attributes

endpoint_configuration.model_config.metadata.gemini.api_endpoint

string

If running Gemini on Vertex, specify the regional API endpoint (hostname only)

Example:

""

endpoint_configuration.model_config.metadata.gemini.project_id

string

If running Gemini on Vertex, specify the project ID

Example:

""

endpoint_configuration.model_config.metadata.gemini.location_id

string

If running Gemini on Vertex, specify the location ID

Example:

""

endpoint_configuration.model_config.metadata.bedrock

Bedrock · object

Show child attributes

endpoint_configuration.model_config.metadata.bedrock.aws_region

string

If using AWS providers (Bedrock) you can override the default AWS_REGION 'us-east-1' by setting this option

Example:

""

endpoint_configuration.model_config.metadata.enkryptai

EnkryptAI · object

Show child attributes

endpoint_configuration.model_config.metadata.enkryptai.deployment_name

string

If using EnkryptAI, specify the deployment name

Example:

"test-deployment-1"

endpoint_configuration.model_config.metadata.xactly

Xactly · object

Show child attributes

endpoint_configuration.model_config.metadata.xactly.assistant_id

string

Xactly assistant ID for authentication

Example:

"XACTLY_ASSISTANT_ID"

endpoint_configuration.model_config.default_request_options

Default Request Options · object

Show child attributes

endpoint_configuration.model_config.default_request_options.temperature

number

Example:

1

endpoint_configuration.model_config.default_request_options.top_p

number

Example:

1

endpoint_configuration.model_config.default_request_options.top_k

integer

Example:

0

endpoint_configuration.testing_for

enum<string>

Purpose of testing

Available options:

foundationModels,

chatbotsAndCopilots,

agents,

URL

Example:

"foundationModels"

endpoint_configuration.model_name

string

Name of the model. Required for foundationModels

Example:

"mistralai/Mixtral-8x7B-Instruct-v0.1"

endpoint_configuration.certifications

string[]

List of certifications

Example:

[
  "GDPR",
  "CCPA",
  "HIPAA",
  "SOC 2 Type II",
  "SOC 3"
]

Example:

{
  "testing_for": "foundationModels",
  "model_name": "mistralai/Mixtral-8x7B-Instruct-v0.1",
  "certifications": [],
  "model_config": {
    "model_provider": "together",
    "hosting_type": "External",
    "input_modalities": ["text"],
    "output_modalities": ["text"],
    "model_source": "https://together.ai",
    "rate_per_min": 20,
    "system_prompt": "",
    "endpoint": {
      "scheme": "https",
      "host": "api.together.xyz",
      "port": 443,
      "base_path": "/v1"
    },
    "paths": {
      "completions": "/completions",
      "chat": "/chat/completions"
    },
    "auth_data": {
      "header_name": "Authorization",
      "header_prefix": "Bearer",
      "space_after_prefix": true
    },
    "apikeys": ["xxxxx"],
    "metadata": {},
    "default_request_options": {}
  }
}

Response

200 - application/json

Successful Response

status

enum<string>

required

Available options:

healthy,

unhealthy

message

string

Example:

"Model health check completed successfully"

data

Data · object

Show child attributes

data.query

string

required

Example:

"What is the capital of India?"

data.response

string

required

Example:

"The capital of India is New Delhi."

error

string

Example:

"Error while checking model health"

API Reference

Authorizations

Body

Response