API docs by Redocly

TabbyAPI (0.1.0)

Download OpenAPI specification:Download

An OAI compatible exllamav2 API that's both lightweight and fast

This docs page is not meant to send requests! Please use a service like Postman or a frontend UI.

Completion Request

Generates a completion from a prompt.

If stream = true, this returns an SSE stream.

header Parameters
x-api-key
string (X-Api-Key)
authorization
string (Authorization)
Request Body schema: application/json
required
Max Tokens (integer) or Max Tokens (null) (Max Tokens)

Aliases: max_length

Min Tokens (integer) or Min Tokens (null) (Min Tokens)

Aliases: min_length

Generate Window (integer) or Generate Window (null) (Generate Window)
Stop (string) or (Array of Stop (strings or integers)) or Stop (null) (Stop)

Aliases: stop_sequence

Banned Strings (string) or Array of Banned Strings (strings) or Banned Strings (null) (Banned Strings)
Array of Banned Tokens (integers) or Banned Tokens (string) or Banned Tokens (null) (Banned Tokens)

Aliases: custom_token_bans

Array of Allowed Tokens (integers) or Allowed Tokens (string) or Allowed Tokens (null) (Allowed Tokens)

Aliases: allowed_token_ids

Token Healing (boolean) or Token Healing (null) (Token Healing)
Temperature (number) or Temperature (null) (Temperature)
Temperature Last (boolean) or Temperature Last (null) (Temperature Last)
Smoothing Factor (number) or Smoothing Factor (null) (Smoothing Factor)
Top K (integer) or Top K (null) (Top K)
Top P (number) or Top P (null) (Top P)
Top A (number) or Top A (null) (Top A)
Min P (number) or Min P (null) (Min P)
Tfs (number) or Tfs (null) (Tfs)
Typical (number) or Typical (null) (Typical)

Aliases: typical_p

Skew (number) or Skew (null) (Skew)
Xtc Probability (number) or Xtc Probability (null) (Xtc Probability)
Xtc Threshold (number) or Xtc Threshold (null) (Xtc Threshold)
Frequency Penalty (number) or Frequency Penalty (null) (Frequency Penalty)
Presence Penalty (number) or Presence Penalty (null) (Presence Penalty)
Repetition Penalty (number) or Repetition Penalty (null) (Repetition Penalty)

Aliases: rep_pen

Penalty Range (integer) or Penalty Range (null) (Penalty Range)

Aliases: repetition_range, repetition_penalty_range, rep_pen_range

Repetition Decay (integer) or Repetition Decay (null) (Repetition Decay)
Dry Multiplier (number) or Dry Multiplier (null) (Dry Multiplier)
Dry Base (number) or Dry Base (null) (Dry Base)
Dry Allowed Length (integer) or Dry Allowed Length (null) (Dry Allowed Length)
Dry Range (integer) or Dry Range (null) (Dry Range)

Aliases: dry_penalty_last_n

Dry Sequence Breakers (string) or Array of Dry Sequence Breakers (strings) or Dry Sequence Breakers (null) (Dry Sequence Breakers)
Mirostat (boolean) or Mirostat (null) (Mirostat)
Default: false
Mirostat Mode (integer) or Mirostat Mode (null) (Mirostat Mode)
Mirostat Tau (number) or Mirostat Tau (null) (Mirostat Tau)
Mirostat Eta (number) or Mirostat Eta (null) (Mirostat Eta)
Add Bos Token (boolean) or Add Bos Token (null) (Add Bos Token)
Ban Eos Token (boolean) or Ban Eos Token (null) (Ban Eos Token)

Aliases: ignore_eos

Skip Special Tokens (boolean) or Skip Special Tokens (null) (Skip Special Tokens)
Logit Bias (object) or Logit Bias (null) (Logit Bias)
Negative Prompt (string) or Negative Prompt (null) (Negative Prompt)
Json Schema (any) or Json Schema (null) (Json Schema)
Regex Pattern (string) or Regex Pattern (null) (Regex Pattern)
Grammar String (string) or Grammar String (null) (Grammar String)
Speculative Ngram (boolean) or Speculative Ngram (null) (Speculative Ngram)
Cfg Scale (number) or Cfg Scale (null) (Cfg Scale)

Aliases: guidance_scale

Max Temp (number) or Max Temp (null) (Max Temp)

Aliases: dynatemp_high

Min Temp (number) or Min Temp (null) (Min Temp)

Aliases: dynatemp_low

Temp Exponent (number) or Temp Exponent (null) (Temp Exponent)
Model (string) or Model (null) (Model)
Stream (boolean) or Stream (null) (Stream)
Default: false
ChatCompletionStreamOptions (object) or null
Logprobs (integer) or Logprobs (null) (Logprobs)
CompletionResponseFormat (object) or null
N (integer) or N (null) (N)
Best Of (integer) or Best Of (null) (Best Of)

Not parsed. Only used for OAI compliance.

Echo (boolean) or Echo (null) (Echo)
Default: false

Not parsed. Only used for OAI compliance.

Suffix (string) or Suffix (null) (Suffix)

Not parsed. Only used for OAI compliance.

User (string) or User (null) (User)

Not parsed. Only used for OAI compliance.

required
Prompt (string) or Array of Prompt (strings) (Prompt)

Responses

Request samples

Content type
application/json
{
  • "max_tokens": 150,
  • "min_tokens": 0,
  • "generate_window": 512,
  • "stop": "string",
  • "banned_strings": "string",
  • "banned_tokens": [
    ],
  • "allowed_tokens": [
    ],
  • "token_healing": true,
  • "temperature": 1,
  • "temperature_last": true,
  • "smoothing_factor": 0,
  • "top_k": -1,
  • "top_p": 1,
  • "top_a": 0,
  • "min_p": 0,
  • "tfs": 1,
  • "typical": 1,
  • "skew": 0,
  • "xtc_probability": 0,
  • "xtc_threshold": 0,
  • "frequency_penalty": 0,
  • "presence_penalty": 0,
  • "repetition_penalty": 1,
  • "penalty_range": 0,
  • "repetition_decay": 0,
  • "dry_multiplier": 0,
  • "dry_base": 0,
  • "dry_allowed_length": 0,
  • "dry_range": 0,
  • "dry_sequence_breakers": "string",
  • "mirostat": false,
  • "mirostat_mode": 0,
  • "mirostat_tau": 1.5,
  • "mirostat_eta": 0.3,
  • "add_bos_token": true,
  • "ban_eos_token": false,
  • "skip_special_tokens": true,
  • "logit_bias": {
    },
  • "negative_prompt": "string",
  • "json_schema": { },
  • "regex_pattern": "string",
  • "grammar_string": "string",
  • "speculative_ngram": true,
  • "cfg_scale": 1,
  • "max_temp": 1,
  • "min_temp": 1,
  • "temp_exponent": 1,
  • "model": "string",
  • "stream": false,
  • "stream_options": {
    },
  • "logprobs": 0,
  • "response_format": {
    },
  • "n": 1,
  • "best_of": 0,
  • "echo": false,
  • "suffix": "string",
  • "user": "string",
  • "prompt": "string"
}

Response samples

Content type
application/json
{
  • "id": "string",
  • "choices": [
    ],
  • "created": 0,
  • "model": "string",
  • "object": "text_completion",
  • "usage": {
    }
}

Chat Completion Request

Generates a chat completion from a prompt.

If stream = true, this returns an SSE stream.

header Parameters
x-api-key
string (X-Api-Key)
authorization
string (Authorization)
Request Body schema: application/json
required
Max Tokens (integer) or Max Tokens (null) (Max Tokens)

Aliases: max_length

Min Tokens (integer) or Min Tokens (null) (Min Tokens)

Aliases: min_length

Generate Window (integer) or Generate Window (null) (Generate Window)
Stop (string) or (Array of Stop (strings or integers)) or Stop (null) (Stop)

Aliases: stop_sequence

Banned Strings (string) or Array of Banned Strings (strings) or Banned Strings (null) (Banned Strings)
Array of Banned Tokens (integers) or Banned Tokens (string) or Banned Tokens (null) (Banned Tokens)

Aliases: custom_token_bans

Array of Allowed Tokens (integers) or Allowed Tokens (string) or Allowed Tokens (null) (Allowed Tokens)

Aliases: allowed_token_ids

Token Healing (boolean) or Token Healing (null) (Token Healing)
Temperature (number) or Temperature (null) (Temperature)
Temperature Last (boolean) or Temperature Last (null) (Temperature Last)
Smoothing Factor (number) or Smoothing Factor (null) (Smoothing Factor)
Top K (integer) or Top K (null) (Top K)
Top P (number) or Top P (null) (Top P)
Top A (number) or Top A (null) (Top A)
Min P (number) or Min P (null) (Min P)
Tfs (number) or Tfs (null) (Tfs)
Typical (number) or Typical (null) (Typical)

Aliases: typical_p

Skew (number) or Skew (null) (Skew)
Xtc Probability (number) or Xtc Probability (null) (Xtc Probability)
Xtc Threshold (number) or Xtc Threshold (null) (Xtc Threshold)
Frequency Penalty (number) or Frequency Penalty (null) (Frequency Penalty)
Presence Penalty (number) or Presence Penalty (null) (Presence Penalty)
Repetition Penalty (number) or Repetition Penalty (null) (Repetition Penalty)

Aliases: rep_pen

Penalty Range (integer) or Penalty Range (null) (Penalty Range)

Aliases: repetition_range, repetition_penalty_range, rep_pen_range

Repetition Decay (integer) or Repetition Decay (null) (Repetition Decay)
Dry Multiplier (number) or Dry Multiplier (null) (Dry Multiplier)
Dry Base (number) or Dry Base (null) (Dry Base)
Dry Allowed Length (integer) or Dry Allowed Length (null) (Dry Allowed Length)
Dry Range (integer) or Dry Range (null) (Dry Range)

Aliases: dry_penalty_last_n

Dry Sequence Breakers (string) or Array of Dry Sequence Breakers (strings) or Dry Sequence Breakers (null) (Dry Sequence Breakers)
Mirostat (boolean) or Mirostat (null) (Mirostat)
Default: false
Mirostat Mode (integer) or Mirostat Mode (null) (Mirostat Mode)
Mirostat Tau (number) or Mirostat Tau (null) (Mirostat Tau)
Mirostat Eta (number) or Mirostat Eta (null) (Mirostat Eta)
Add Bos Token (boolean) or Add Bos Token (null) (Add Bos Token)
Ban Eos Token (boolean) or Ban Eos Token (null) (Ban Eos Token)

Aliases: ignore_eos

Skip Special Tokens (boolean) or Skip Special Tokens (null) (Skip Special Tokens)
Logit Bias (object) or Logit Bias (null) (Logit Bias)
Negative Prompt (string) or Negative Prompt (null) (Negative Prompt)
Json Schema (any) or Json Schema (null) (Json Schema)
Regex Pattern (string) or Regex Pattern (null) (Regex Pattern)
Grammar String (string) or Grammar String (null) (Grammar String)
Speculative Ngram (boolean) or Speculative Ngram (null) (Speculative Ngram)
Cfg Scale (number) or Cfg Scale (null) (Cfg Scale)

Aliases: guidance_scale

Max Temp (number) or Max Temp (null) (Max Temp)

Aliases: dynatemp_high

Min Temp (number) or Min Temp (null) (Min Temp)

Aliases: dynatemp_low

Temp Exponent (number) or Temp Exponent (null) (Temp Exponent)
Model (string) or Model (null) (Model)
Stream (boolean) or Stream (null) (Stream)
Default: false
ChatCompletionStreamOptions (object) or null
Logprobs (integer) or Logprobs (null) (Logprobs)
CompletionResponseFormat (object) or null
N (integer) or N (null) (N)
Best Of (integer) or Best Of (null) (Best Of)

Not parsed. Only used for OAI compliance.

Echo (boolean) or Echo (null) (Echo)
Default: false

Not parsed. Only used for OAI compliance.

Suffix (string) or Suffix (null) (Suffix)

Not parsed. Only used for OAI compliance.

User (string) or User (null) (User)

Not parsed. Only used for OAI compliance.

required
Messages (string) or Array of Messages (objects) (Messages)
Prompt Template (string) or Prompt Template (null) (Prompt Template)
Add Generation Prompt (boolean) or Add Generation Prompt (null) (Add Generation Prompt)
Default: true
Template Vars (object) or Template Vars (null) (Template Vars)
Default: {}
Response Prefix (string) or Response Prefix (null) (Response Prefix)
Array of Tools (objects) or Tools (null) (Tools)
Array of Functions (objects) or Functions (null) (Functions)

Responses

Request samples

Content type
application/json
{
  • "max_tokens": 150,
  • "min_tokens": 0,
  • "generate_window": 512,
  • "stop": "string",
  • "banned_strings": "string",
  • "banned_tokens": [
    ],
  • "allowed_tokens": [
    ],
  • "token_healing": true,
  • "temperature": 1,
  • "temperature_last": true,
  • "smoothing_factor": 0,
  • "top_k": -1,
  • "top_p": 1,
  • "top_a": 0,
  • "min_p": 0,
  • "tfs": 1,
  • "typical": 1,
  • "skew": 0,
  • "xtc_probability": 0,
  • "xtc_threshold": 0,
  • "frequency_penalty": 0,
  • "presence_penalty": 0,
  • "repetition_penalty": 1,
  • "penalty_range": 0,
  • "repetition_decay": 0,
  • "dry_multiplier": 0,
  • "dry_base": 0,
  • "dry_allowed_length": 0,
  • "dry_range": 0,
  • "dry_sequence_breakers": "string",
  • "mirostat": false,
  • "mirostat_mode": 0,
  • "mirostat_tau": 1.5,
  • "mirostat_eta": 0.3,
  • "add_bos_token": true,
  • "ban_eos_token": false,
  • "skip_special_tokens": true,
  • "logit_bias": {
    },
  • "negative_prompt": "string",
  • "json_schema": { },
  • "regex_pattern": "string",
  • "grammar_string": "string",
  • "speculative_ngram": true,
  • "cfg_scale": 1,
  • "max_temp": 1,
  • "min_temp": 1,
  • "temp_exponent": 1,
  • "model": "string",
  • "stream": false,
  • "stream_options": {
    },
  • "logprobs": 0,
  • "response_format": {
    },
  • "n": 1,
  • "best_of": 0,
  • "echo": false,
  • "suffix": "string",
  • "user": "string",
  • "messages": "string",
  • "prompt_template": "string",
  • "add_generation_prompt": true,
  • "template_vars": { },
  • "response_prefix": "string",
  • "tools": [
    ],
  • "functions": [
    ]
}

Response samples

Content type
application/json
{
  • "id": "string",
  • "choices": [
    ],
  • "created": 0,
  • "model": "string",
  • "object": "chat.completion",
  • "usage": {
    }
}

Embeddings

header Parameters
x-api-key
string (X-Api-Key)
authorization
string (Authorization)
Request Body schema: application/json
required
required
Input (string) or Array of Input (strings) (Input)

List of input texts to generate embeddings for.

encoding_format
string (Encoding Format)
Default: "float"

Encoding format for the embeddings. Can be 'float' or 'base64'.

Model (string) or Model (null) (Model)

Name of the embedding model to use. If not provided, the default model will be used.

Responses

Request samples

Content type
application/json
{
  • "input": "string",
  • "encoding_format": "float",
  • "model": "string"
}

Response samples

Content type
application/json
{
  • "object": "list",
  • "data": [
    ],
  • "model": "string",
  • "usage": {
    }
}

Healthcheck

Get the current service health status

Responses

Response samples

Content type
application/json
{
  • "status": "healthy",
  • "issues": [
    ]
}

List Models

Lists all models in the model directory.

Requires an admin key to see all models.

header Parameters
x-api-key
string (X-Api-Key)
authorization
string (Authorization)

Responses

Response samples

Content type
application/json
{
  • "object": "list",
  • "data": [
    ]
}

List Models

Lists all models in the model directory.

Requires an admin key to see all models.

header Parameters
x-api-key
string (X-Api-Key)
authorization
string (Authorization)

Responses

Response samples

Content type
application/json
{
  • "object": "list",
  • "data": [
    ]
}

Current Model

Returns the currently loaded model.

header Parameters
x-api-key
string (X-Api-Key)
authorization
string (Authorization)

Responses

Response samples

Content type
application/json
{
  • "id": "test",
  • "object": "model",
  • "created": 0,
  • "owned_by": "tabbyAPI",
  • "logging": {
    },
  • "parameters": {
    }
}

List Draft Models

Lists all draft models in the model directory.

Requires an admin key to see all draft models.

header Parameters
x-api-key
string (X-Api-Key)
authorization
string (Authorization)

Responses

Response samples

Content type
application/json
{
  • "object": "list",
  • "data": [
    ]
}

Load Model

Loads a model into the model container. This returns an SSE stream.

header Parameters
x-admin-key
string (X-Admin-Key)
authorization
string (Authorization)
Request Body schema: application/json
required
model_name
required
string (Model Name)

Aliases: name

Max Seq Len (integer) or Max Seq Len (null) (Max Seq Len)

Leave this blank to use the model's base sequence length

Cache Size (integer) or Cache Size (null) (Cache Size)

Number in tokens, must be greater than or equal to max_seq_len

Tensor Parallel (boolean) or Tensor Parallel (null) (Tensor Parallel)
Gpu Split Auto (boolean) or Gpu Split Auto (null) (Gpu Split Auto)
Array of Autosplit Reserve (numbers) or Autosplit Reserve (null) (Autosplit Reserve)
Array of Gpu Split (numbers) or Gpu Split (null) (Gpu Split)
Rope Scale (number) or Rope Scale (null) (Rope Scale)

Automatically pulled from the model's config if not present

Rope Alpha (number) or "auto" (string) or Rope Alpha (null) (Rope Alpha)

Automatically calculated if set to "auto"

Cache Mode (string) or Cache Mode (null) (Cache Mode)
Chunk Size (integer) or Chunk Size (null) (Chunk Size)
Prompt Template (string) or Prompt Template (null) (Prompt Template)
Num Experts Per Token (integer) or Num Experts Per Token (null) (Num Experts Per Token)
DraftModelLoadRequest (object) or null
Skip Queue (boolean) or Skip Queue (null) (Skip Queue)
Default: false

Responses

Request samples

Content type
application/json
{
  • "model_name": "string",
  • "max_seq_len": 4096,
  • "cache_size": 4096,
  • "tensor_parallel": true,
  • "gpu_split_auto": true,
  • "autosplit_reserve": [
    ],
  • "gpu_split": [
    ],
  • "rope_scale": 1,
  • "rope_alpha": 1,
  • "cache_mode": "string",
  • "chunk_size": 0,
  • "prompt_template": "string",
  • "num_experts_per_token": 0,
  • "draft_model": {
    },
  • "skip_queue": false
}

Response samples

Content type
application/json
{
  • "model_type": "model",
  • "module": 0,
  • "modules": 0,
  • "status": "string"
}

Unload Model

Unloads the currently loaded model.

header Parameters
x-admin-key
string (X-Admin-Key)
authorization
string (Authorization)

Responses

Response samples

Content type
application/json
null

Download Model

Downloads a model from HuggingFace.

header Parameters
x-admin-key
string (X-Admin-Key)
authorization
string (Authorization)
Request Body schema: application/json
required
repo_id
required
string (Repo Id)
repo_type
string (Repo Type)
Default: "model"
Folder Name (string) or Folder Name (null) (Folder Name)
Revision (string) or Revision (null) (Revision)
Token (string) or Token (null) (Token)
include
Array of strings (Include)
exclude
Array of strings (Exclude)
Chunk Limit (integer) or Chunk Limit (null) (Chunk Limit)
Timeout (integer) or Timeout (null) (Timeout)

Responses

Request samples

Content type
application/json
{
  • "repo_id": "string",
  • "repo_type": "model",
  • "folder_name": "string",
  • "revision": "string",
  • "token": "string",
  • "include": [
    ],
  • "exclude": [
    ],
  • "chunk_limit": 0,
  • "timeout": 0
}

Response samples

Content type
application/json
{
  • "download_path": "string"
}

List All Loras

Lists all LoRAs in the lora directory.

Requires an admin key to see all LoRAs.

header Parameters
x-api-key
string (X-Api-Key)
authorization
string (Authorization)

Responses

Response samples

Content type
application/json
{
  • "object": "list",
  • "data": [
    ]
}

List All Loras

Lists all LoRAs in the lora directory.

Requires an admin key to see all LoRAs.

header Parameters
x-api-key
string (X-Api-Key)
authorization
string (Authorization)

Responses

Response samples

Content type
application/json
{
  • "object": "list",
  • "data": [
    ]
}

Active Loras

Returns the currently loaded loras.

header Parameters
x-api-key
string (X-Api-Key)
authorization
string (Authorization)

Responses

Response samples

Content type
application/json
{
  • "object": "list",
  • "data": [
    ]
}

Load Lora

Loads a LoRA into the model container.

header Parameters
x-admin-key
string (X-Admin-Key)
authorization
string (Authorization)
Request Body schema: application/json
required
required
Array of objects (Loras)
skip_queue
boolean (Skip Queue)
Default: false

Responses

Request samples

Content type
application/json
{
  • "loras": [
    ],
  • "skip_queue": false
}

Response samples

Content type
application/json
{
  • "success": [
    ],
  • "failure": [
    ]
}

Unload Loras

Unloads the currently loaded loras.

header Parameters
x-admin-key
string (X-Admin-Key)
authorization
string (Authorization)

Responses

Response samples

Content type
application/json
null

List Embedding Models

Lists all embedding models in the model directory.

Requires an admin key to see all embedding models.

header Parameters
x-api-key
string (X-Api-Key)
authorization
string (Authorization)

Responses

Response samples

Content type
application/json
{
  • "object": "list",
  • "data": [
    ]
}

Get Embedding Model

Returns the currently loaded embedding model.

header Parameters
x-api-key
string (X-Api-Key)
authorization
string (Authorization)

Responses

Response samples

Content type
application/json
{
  • "id": "test",
  • "object": "model",
  • "created": 0,
  • "owned_by": "tabbyAPI",
  • "logging": {
    },
  • "parameters": {
    }
}

Load Embedding Model

header Parameters
x-admin-key
string (X-Admin-Key)
authorization
string (Authorization)
Request Body schema: application/json
required
embedding_model_name
required
string (Embedding Model Name)

Aliases: name

Embeddings Device (string) or Embeddings Device (null) (Embeddings Device)
Default: "cpu"

Responses

Request samples

Content type
application/json
{
  • "embedding_model_name": "string",
  • "embeddings_device": "cpu"
}

Response samples

Content type
application/json
{
  • "model_type": "model",
  • "module": 0,
  • "modules": 0,
  • "status": "string"
}

Unload Embedding Model

Unloads the current embedding model.

header Parameters
x-admin-key
string (X-Admin-Key)
authorization
string (Authorization)

Responses

Response samples

Content type
application/json
null

Encode Tokens

Encodes a string or chat completion messages into tokens.

header Parameters
x-api-key
string (X-Api-Key)
authorization
string (Authorization)
Request Body schema: application/json
required
add_bos_token
boolean (Add Bos Token)
Default: true
encode_special_tokens
boolean (Encode Special Tokens)
Default: true
decode_special_tokens
boolean (Decode Special Tokens)
Default: true
required
Text (string) or Array of Text (objects) (Text)

Responses

Request samples

Content type
application/json
{
  • "add_bos_token": true,
  • "encode_special_tokens": true,
  • "decode_special_tokens": true,
  • "text": "string"
}

Response samples

Content type
application/json
{
  • "tokens": [
    ],
  • "length": 0
}

Decode Tokens

Decodes tokens into a string.

header Parameters
x-api-key
string (X-Api-Key)
authorization
string (Authorization)
Request Body schema: application/json
required
add_bos_token
boolean (Add Bos Token)
Default: true
encode_special_tokens
boolean (Encode Special Tokens)
Default: true
decode_special_tokens
boolean (Decode Special Tokens)
Default: true
tokens
required
Array of integers (Tokens)

Responses

Request samples

Content type
application/json
{
  • "add_bos_token": true,
  • "encode_special_tokens": true,
  • "decode_special_tokens": true,
  • "tokens": [
    ]
}

Response samples

Content type
application/json
{
  • "text": "string"
}

Key Permission

Gets the access level/permission of a provided key in headers.

Priority:

  • X-admin-key
  • X-api-key
  • Authorization
header Parameters
x-api-key
string (X-Api-Key)
authorization
string (Authorization)

Responses

Response samples

Content type
application/json
{
  • "permission": "string"
}

List Templates

Get a list of all templates.

Requires an admin key to see all templates.

header Parameters
x-api-key
string (X-Api-Key)
authorization
string (Authorization)

Responses

Response samples

Content type
application/json
{
  • "object": "list",
  • "data": [
    ]
}

List Templates

Get a list of all templates.

Requires an admin key to see all templates.

header Parameters
x-api-key
string (X-Api-Key)
authorization
string (Authorization)

Responses

Response samples

Content type
application/json
{
  • "object": "list",
  • "data": [
    ]
}

Switch Template

Switch the currently loaded template.

header Parameters
x-admin-key
string (X-Admin-Key)
authorization
string (Authorization)
Request Body schema: application/json
required
prompt_template_name
required
string (Prompt Template Name)

Aliases: name

Responses

Request samples

Content type
application/json
{
  • "prompt_template_name": "string"
}

Response samples

Content type
application/json
null

Unload Template

Unloads the currently selected template

header Parameters
x-admin-key
string (X-Admin-Key)
authorization
string (Authorization)

Responses

Response samples

Content type
application/json
null

List Sampler Overrides

List all currently applied sampler overrides.

Requires an admin key to see all override presets.

header Parameters
x-api-key
string (X-Api-Key)
authorization
string (Authorization)

Responses

Response samples

Content type
application/json
{
  • "selected_preset": "string",
  • "overrides": { },
  • "presets": [
    ]
}

List Sampler Overrides

List all currently applied sampler overrides.

Requires an admin key to see all override presets.

header Parameters
x-api-key
string (X-Api-Key)
authorization
string (Authorization)

Responses

Response samples

Content type
application/json
{
  • "selected_preset": "string",
  • "overrides": { },
  • "presets": [
    ]
}

Switch Sampler Override

Switch the currently loaded override preset

header Parameters
x-admin-key
string (X-Admin-Key)
authorization
string (Authorization)
Request Body schema: application/json
required
Preset (string) or Preset (null) (Preset)

Pass a sampler override preset name

Overrides (object) or Overrides (null) (Overrides)

Sampling override parent takes in individual keys and overrides. Ignored if preset is provided.

Responses

Request samples

Content type
application/json
{
  • "preset": "string",
  • "overrides": {
    }
}

Response samples

Content type
application/json
null

Unload Sampler Override

Unloads the currently selected override preset

header Parameters
x-admin-key
string (X-Admin-Key)
authorization
string (Authorization)

Responses

Response samples

Content type
application/json
null