OpenAI SDK Compatibility

The Gateway uses OpenAI request and response shapes at /v1/openai. There is no mapping layer and no request rewriting. The same chat.completions.create call reaches a vision model.

Pointing the SDK at the Gateway

export OPENAI_BASE_URL="https://gateway.vlm.run/v1/openai"
export OPENAI_API_KEY="<VLMRUN_API_KEY>"

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.vlm.run/v1/openai",
  apiKey: process.env.VLMRUN_API_KEY,
});

Authentication is optional at the moment. Omit the Authorization header for anonymous access. A model marked (paid) on Models returns 403 model_not_entitled without a paid account. Tiers and quotas are in Authentication.

Check the connection

curl https://gateway.vlm.run/v1/openai/models \
  -H "Authorization: Bearer $VLMRUN_API_KEY"

for model in client.models.list().data:
    print(model.id)

for (const model of (await client.models.list()).data) {
  console.log(model.id);
}

vlmrun gw models

Compatibility matrix

Feature	Status	Notes
`POST /chat/completions`	Same	Identical request and response shapes, streaming included.
`POST /embeddings`	Same	See Embeddings.
`POST /audio/transcriptions`	Same	See Transcriptions.
`GET /models`	Extended	Adds `methods`, `default_method`, `capabilities`, and `task` per model.
`GET /v1/models/{id}`	Extended	Served outside `/v1/openai`. `client.models.retrieve()` calls `/v1/openai/models/{id}` and gets a `404`, so use Get Model directly.
`text` and `image_url` content parts	Same	The shapes you already send.
`document_url` content part	Extended	Not in OpenAI. A PDF, fanned out per page.
`video_url` content part	Extended	Not in OpenAI. The model reads the clip natively.
`response_format: json_object`	Same	Returns the structured payload instead of the text rendering.
Sampling fields	Per model	Honoured where `supported_parameters` lists them. See supported parameters.
`usage.cost`	Extended	Per-request cost in USD, alongside the standard token counts.
`method`, `method_params`	Extended	Picks the operation a model runs.
Tool and function calling	Per model	Accepted where `supported_parameters` lists `tools`. OCR and region models do not call tools.

See all 13 rowsShow less Put request extensions at the top level of the body, or in extra_body on the OpenAI SDKs. The full list and defaults are in Chat Completions.

Supported parameters

Each model lists the OpenAI parameters it honours in supported_parameters on GET /v1/openai/models. On a model with a non-empty list, an unlisted temperature, top_p, top_k, seed, logprobs, frequency_penalty, or presence_penalty returns 400 capability_violation, and param names it. A model with an empty list accepts the standard fields and ignores them.

Parameter	Chat VLMs	Provider models	OCR models
`temperature`	●	●
`top_p`	●	●
`top_k`	●
`min_p`	●
`seed`	●	●
`max_tokens`	●	●
`max_completion_tokens`	●
`stop`	●	●
`presence_penalty`, `frequency_penalty`	●
`repetition_penalty`	●
`logprobs`, `top_logprobs`	●
`response_format: none`	●	●	●
`response_format: json_object`	●	●	●
`response_format: json_schema`	●	● except `meta/muse-glimmer-30b`
`stream`, `stream_options`	●	● `stream`	● `stream`
`tools`, `tool_choice`	● `qwen/qwen3.8-27b` only	● `tools`

● The model honours the parameter. Blank means the model does not.

Chat VLMs: qwen/qwen3.5-0.8b, qwen/qwen3.8-27b, and google/gemma-4-26b-a4b-it.
Provider models: google/gemini-*, meta/muse-*, moonshotai/kimi-k3, and minimax/minimax-m3.
OCR models: every model with an empty supported_parameters list. They accept sampling fields and ignore them. json_schema returns 400.
google/diffusiongemma-26b-a4b-it: max_tokens, top_p, top_k, stop, stream, logprobs, and top_logprobs only. temperature, seed, and any response_format return 400.

Embeddings take dimensions and encoding_format. google/embeddinggemma-2 also takes document_pages, document_max_pages, and prompt_name. Transcription takes language, response_format, and timestamp_granularities.

Defaults and limits

Topic	Behavior	See
Sampling fields	Honoured only where listed above. OCR and region models ignore them.	Get Model
`method`	Selects the operation. Omit it to apply `default_method`. A method the model does not publish returns `400`.	Methods
Documents	One request. Pass a `document_url` part, and the Gateway rasterizes it and fans out per page.	Document OCR
JSON mode	Buffered. With `stream: true`, the finished object arrives as `chat.completion.chunk` frames. Text mode streams.	Response Formats
`x-request-id`	On every response. Log it so support can trace a call.	Error Codes

Quickstart

First VQA and document OCR requests.

Chat Completions

Request reference, including every Gateway extension.

Methods

Select the operation with method and method_params.

Response Formats

JSON mode for code, text mode for agents.

TypeSafe SDK Compatibility

Get typed, calibrated decisions from the same Gateway.

Overview

Platform

OpenAI Compatibility

TypeSafe Compatibility

Realtime

MCP

Misc

OpenAI SDK Compatibility

Pointing the SDK at the Gateway

Check the connection

Compatibility matrix

Supported parameters

Defaults and limits

Quickstart

Chat Completions

Methods

Response Formats

TypeSafe SDK Compatibility

​Pointing the SDK at the Gateway

​Check the connection

​Compatibility matrix

​Supported parameters

​Defaults and limits

​Related

Quickstart

Chat Completions

Methods

Response Formats

TypeSafe SDK Compatibility

Pointing the SDK at the Gateway

Check the connection

Compatibility matrix

Supported parameters

Defaults and limits

Related