OpenAI SDK Compatibility

The Gateway uses OpenAI request and response shapes at /v1/openai. There is no mapping layer and no request rewriting. The same chat.completions.create call reaches a vision model.

Pointing the SDK at the Gateway

export OPENAI_BASE_URL="https://gateway.vlm.run/v1/openai"
export OPENAI_API_KEY="<VLMRUN_API_KEY>"

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.vlm.run/v1/openai",
  apiKey: process.env.VLMRUN_API_KEY,
});

Authentication is optional at the moment. Omit the Authorization header for anonymous access. Tiers and quotas are in Authentication.

Check the connection

curl https://gateway.vlm.run/v1/openai/models \
  -H "Authorization: Bearer $VLMRUN_API_KEY"

for model in client.models.list().data:
    print(model.id)

for (const model of (await client.models.list()).data) {
  console.log(model.id);
}

vlmrun gw models

Compatibility matrix

Feature	Status	Notes
`POST /chat/completions`	Same	Identical request and response shapes, streaming included.
`POST /embeddings`	Same	See Embeddings.
`POST /audio/transcriptions`	Same	See Transcriptions.
`GET /models`	Extended	Adds `methods`, `default_method`, `capabilities`, and `task` per model.
`GET /v1/models/{id}`	Extended	Served outside `/v1/openai`. `client.models.retrieve()` calls `/v1/openai/models/{id}` and gets a `404`, so use Get Model directly.
`text` and `image_url` content parts	Same	The shapes you already send.
`document_url` content part	Extended	Not in OpenAI. A PDF, fanned out per page.
`video_url` content part	Extended	Not in OpenAI. Encoded to images before dispatch.
`response_format: json_object`	Same	Returns the structured payload instead of the text rendering.
Sampling fields	Per model	Honoured where `supported_parameters` lists them. See supported parameters.
`usage.cost`	Extended	Per-request cost in USD, alongside the standard token counts.
`method`, `method_params`	Extended	Picks the operation a model runs.
Tool and function calling	Per model	Accepted where `supported_parameters` lists `tools`. OCR and region models do not call tools.

See all 13 rowsShow less Put request extensions at the top level of the body, or in extra_body on the OpenAI SDKs. The full list and defaults are in Chat Completions.

Supported parameters

Each model lists the OpenAI parameters it honours in supported_parameters on GET /v1/openai/models. A model with a non-empty list returns 400 capability_violation for any other parameter. A model with an empty list accepts the standard fields and ignores them.

Parameter	Chat VLMs	Provider models	OCR models
`temperature`	●	●
`top_p`	●	●
`top_k`	●
`min_p`	●
`seed`	●	●
`max_tokens`	●	●
`max_completion_tokens`	●
`stop`	●	●
`presence_penalty`, `frequency_penalty`	●
`repetition_penalty`	●
`logprobs`, `top_logprobs`	●
`response_format: none`	●	●	●
`response_format: json_object`	●	●	●
`response_format: json_schema`	●	● except `meta/muse-glimmer-30b`
`stream`, `stream_options`	●	● `stream`	Text mode only
`tools`, `tool_choice`	●	● `tools`

● The model honours the parameter. Blank means the model does not, and a non-empty list rejects it with 400.

Chat VLMs: qwen/qwen3.5-0.8b, qwen/qwen3.8-27b, and google/gemma-4-26b-a4b-it.
Provider models: google/gemini-*, meta/muse-*, moonshotai/kimi-k3, and minimax/minimax-m3.
OCR models: every model with an empty supported_parameters list. They accept sampling fields and ignore them.

Embeddings take dimensions and encoding_format. Transcription takes language, response_format, and timestamp_granularities.

Defaults and limits

Topic	Behavior	See
Sampling fields	Honoured only where listed above. OCR and region models ignore them.	Get Model
`method`	Selects the operation. Omit it to apply `default_method`. A method the model does not publish returns `400`.	Methods
Documents	One request. Pass a `document_url` part, and the Gateway rasterizes it and fans out per page.	Document OCR
JSON mode	Returns one complete object and is never streamed. Text mode streams.	Response Formats
`x-request-id`	On every response. Log it so support can trace a call.	Error Codes

Quickstart

First VQA and document OCR requests.

Chat Completions

Request reference, including every Gateway extension.

Methods

Select the operation with method and method_params.

Response Formats

JSON mode for code, text mode for agents.

TypeSafe SDK Compatibility

Get typed, calibrated decisions from the same Gateway.

Overview

Platform

OpenAI Compatibility

TypeSafe Compatibility

MCP

Misc

OpenAI SDK Compatibility

Pointing the SDK at the Gateway

Check the connection

Compatibility matrix

Supported parameters

Defaults and limits

Quickstart

Chat Completions

Methods

Response Formats

TypeSafe SDK Compatibility

​Pointing the SDK at the Gateway

​Check the connection

​Compatibility matrix

​Supported parameters

​Defaults and limits

​Related

Quickstart

Chat Completions

Methods

Response Formats

TypeSafe SDK Compatibility

Pointing the SDK at the Gateway

Check the connection

Compatibility matrix

Supported parameters

Defaults and limits

Related