OpenAI SDK Compatibility

The Gateway speaks OpenAI’s own request and response shapes under /v1/openai, so an existing client keeps working. There is no mapping layer and no request rewriting: the same chat.completions.create call you already send reaches a vision model here.

Pointing the SDK at the Gateway

Two values, no code changes:

export OPENAI_BASE_URL="https://gateway.vlm.run/v1/openai"
export OPENAI_API_KEY="<VLMRUN_API_KEY>"

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.vlm.run/v1/openai",
  apiKey: process.env.VLMRUN_API_KEY,
});

Authentication is optional at the moment; send Bearer vlmrun for anonymous access. See Authentication for tiers and quotas.

Compatibility matrix

Feature	Status	Notes
`POST /chat/completions`	Same	Identical request and response shapes, streaming included.
`POST /embeddings`	Same	See Embeddings.
`POST /audio/transcriptions`	Same	See Transcriptions.
`GET /models`, `GET /models/{id}`	Extended	Adds `methods`, `default_method`, `capabilities`, and `task` per model.
`text` and `image_url` content parts	Same	The shapes you already send.
`document_url` content part	Extended	Not in OpenAI. A PDF, fanned out per page.
`video_url` content part	Extended	Not in OpenAI. Encoded to images before dispatch.
`response_format: json_object`	Same	Returns the structured payload instead of the text rendering.
Sampling fields	Accepted	`temperature`, `max_tokens`, `top_p` and the rest are accepted but mostly ignored, see below.
`usage.cost`	Extended	Per-request cost in USD, alongside the standard token counts.
`method`, `method_params`	Extended	Picks the operation a model runs.
Tool and function calling	Not supported	A request carrying `tools` returns a `400`. These models classify, read and detect; they do not call tools.

The request extensions go at the top level of the body, or through extra_body in the OpenAI SDKs. Chat Completions has the full list with defaults.

Model names

Model ids are <org>/<slug>, for example paddleocr/pp-ocrv6. Most models also accept a short form (pp-ocrv6) and their Hugging Face repo id. There is no gpt-* model here, so a client carrying an OpenAI default model name has to be pointed at a Gateway id. GET /v1/openai/models is the live list. See Models for the catalog with capabilities and use-case guidance.

Things worth knowing

Sampling fields are accepted, not honoured: OCR and detection models are not free-form text generators, so temperature and friends usually change nothing. A model’s supported_parameters on Get Model is authoritative, and it is often empty. The chat VLMs are the exception.
method decides what a model computes: the same model can OCR, detect, or parse a layout. Omitting it applies the model’s default_method. A method a model does not publish is a 400. See Methods.
A document is one request, not one per page: pass a document_url part and the Gateway rasterizes and fans out per page for you. See Document OCR.
JSON mode is never streamed: response_format: {"type":"json_object"} is always served whole, because a single valid JSON object cannot be assembled from SSE deltas. Text mode streams normally.
Every response carries x-request-id, which is worth logging: it is what support needs to trace a call.

Check the connection

curl https://gateway.vlm.run/v1/openai/models \
  -H "Authorization: Bearer $VLMRUN_API_KEY"

for model in client.models.list().data:
    print(model.id)

for (const model of (await client.models.list()).data) {
  console.log(model.id);
}

vlmrun gw models

Quickstart

First VQA and document OCR requests, end to end.

Chat Completions

Full request reference, including every Gateway extension.

Methods

What method and response_format change about a reply.

TypeSafe SDK Compatibility

The other surface: typed, calibrated decisions.

Get Started

OpenAI-Compatible API

TypeSafe-Compatible API

Guides

API Reference

Tools

Platform

OpenAI SDK Compatibility

Pointing the SDK at the Gateway

Compatibility matrix

Model names

Things worth knowing

Check the connection

Quickstart

Chat Completions

Methods

TypeSafe SDK Compatibility

​Pointing the SDK at the Gateway

​Compatibility matrix

​Model names

​Things worth knowing

​Check the connection

​Related

Quickstart

Chat Completions

Methods

TypeSafe SDK Compatibility

Pointing the SDK at the Gateway

Compatibility matrix

Model names

Things worth knowing

Check the connection

Related