Chat Completions

import requests

url = "https://gateway.vlm.run/v1/openai/chat/completions"

payload = {}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({})
};

fetch('https://gateway.vlm.run/v1/openai/chat/completions', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

require 'uri'
require 'net/http'

url = URI("https://gateway.vlm.run/v1/openai/chat/completions")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Post.new(url)
request["Authorization"] = 'Bearer <token>'
request["Content-Type"] = 'application/json'
request.body = "{}"

response = http.request(request)
puts response.read_body

{
  "model": "<string>",
  "choices": [
    {
      "message": {
        "content": "<string>",
        "role": "assistant"
      },
      "index": 0,
      "finish_reason": "stop"
    }
  ],
  "id": "<string>",
  "object": "chat.completion",
  "created": 123,
  "served_model_id": "<string>",
  "backend": "<string>",
  "provider": "<string>",
  "usage": {
    "prompt_tokens": 0,
    "completion_tokens": 0,
    "total_tokens": 0
  }
}

{
  "detail": [
    {
      "loc": [
        "<string>"
      ],
      "msg": "<string>",
      "type": "<string>",
      "input": "<unknown>",
      "ctx": {}
    }
  ]
}

POST

openai

chat

completions

Chat Completions

import requests

url = "https://gateway.vlm.run/v1/openai/chat/completions"

payload = {}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({})
};

fetch('https://gateway.vlm.run/v1/openai/chat/completions', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

require 'uri'
require 'net/http'

url = URI("https://gateway.vlm.run/v1/openai/chat/completions")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Post.new(url)
request["Authorization"] = 'Bearer <token>'
request["Content-Type"] = 'application/json'
request.body = "{}"

response = http.request(request)
puts response.read_body

{
  "model": "<string>",
  "choices": [
    {
      "message": {
        "content": "<string>",
        "role": "assistant"
      },
      "index": 0,
      "finish_reason": "stop"
    }
  ],
  "id": "<string>",
  "object": "chat.completion",
  "created": 123,
  "served_model_id": "<string>",
  "backend": "<string>",
  "provider": "<string>",
  "usage": {
    "prompt_tokens": 0,
    "completion_tokens": 0,
    "total_tokens": 0
  }
}

{
  "detail": [
    {
      "loc": [
        "<string>"
      ],
      "msg": "<string>",
      "type": "<string>",
      "input": "<unknown>",
      "ctx": {}
    }
  ]
}

Authentication is optional for the VLM Run Gateway (at the moment). See Authentication for tiers and Rate Limits for per-tier quotas.

Models Supported

For available models and aliases, see Models. Standard OpenAI sampling fields (temperature, max_tokens, top_p, frequency_penalty, presence_penalty, stop, n) are accepted on every request, but OCR and detection models generally ignore sampling params since they are not free-form text generators. Check a model’s supported_parameters via Get Model if a field you send does not appear to change the output. response_format accepts {"type":"json_object"} for the JSON response object. Omitting the field returns the text rendering. See Methods & Response Format.

Gateway extensions

These fields are accepted at the top level of the request body (or via extra_body in the OpenAI Python SDK):

Field	Type	Default	Description
`method`	`string`	model default	Model-specific operation (`ocr`, `detect`, `markdown`, `text`, `parse_layout`, `chat`, …). It sets each page’s payload kind: see Document text output.
`method_params`	`object`	`null`	Keyword arguments for the selected `method`.
`precision`	`integer`	`4`	Decimal places (1-8) on normalized `bbox_xywh` / `poly_xy` / `point_xy` and `score`. Ignored for markdown payloads.
`llm`	`string`	`null`	Optional LLM for post-processing structured model output.
`document_dpi`	`integer`	`72`	DPI when rasterizing PDF pages.
`document_max_pages`	`integer`	`128`	Maximum PDF pages per request. A longer PDF is a `400` naming the ranges that cover it.
`document_pages`	`array`	`null` (all)	Zero-indexed page indices and/or `[start, stop]` half-open ranges to read. Selected pages are re-numbered from 0.
`image_resolution`	`string`	`null`	Square resize preset (`224x224` … `768x768`).
`video_max_frames`	`integer`	`null`	Cap on frames sampled from `video_url` inputs. Used by `qwen/qwen3.5-0.8b` and `qwen/qwen3.8-27b`.
`video_fps`	`number`	`null`	Target frames per second to sample from `video_url` inputs. Used by `qwen/qwen3.5-0.8b` and `qwen/qwen3.8-27b`.
`video_resolution`	`string`	`null`	Resize preset for sampled video frames (`256x192` … `640x480`). Used by `qwen/qwen3.5-0.8b` and `qwen/qwen3.8-27b`.
`video_encoder`	`string`	`mosaic`	How a `video_url` is encoded into image(s) before dispatch (`mosaic`, `frames`, `keyframes`). See Video encoding.
`video_encoder_params`	`object`	`null`	Keyword arguments for `video_encoder` (e.g. `tile_cols`, `tile_rows`, `num_frames`).

video_max_frames, video_fps, video_resolution, video_encoder, and video_encoder_params apply when the request includes a video_url content part on a video-capable model. See Video Inputs.

Video encoding

When a request includes a video_url, the Gateway encodes it into image(s) before dispatch so image-only models can read it. video_encoder selects the strategy and defaults to mosaic (sampled keyframes tiled into a single grid image); frames and keyframes are also available. Tune the chosen encoder with video_encoder_params (for example tile_cols, tile_rows, num_frames).

{
  "model": "paddleocr/pp-ocrv6",
  "video_encoder": "mosaic",
  "video_encoder_params": { "tile_cols": 3, "tile_rows": 3, "num_frames": 9 },
  "messages": [
    {
      "role": "user",
      "content": [
        { "type": "text", "text": "What is happening in this video?" },
        {
          "type": "video_url",
          "video_url": { "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/video.transcription/bakery.mp4" }
        }
      ]
    }
  ]
}

An undecodable video on the chat-completions video path returns 400 (invalid_request_error). See Error Codes.

Document text output

For document_url inputs, text mode (response_format omitted) always returns one <document> block per input PDF, wrapping one <page> block per rasterized page. method decides only each page’s payload kind, declared on the page as format:

`method`	Page `format`	Page body
`markdown` (the default on `zai-org/glm-ocr` and `rednote-hilab/dots.mocr`), `text`, `rednote-hilab/dots.mocr`’s `ocr`	`markdown`	That page’s Markdown string.
`ocr`, `detect`, `parse_layout`, `parse_layout_only`	`json`	That page’s block container, `{"object": "document.page.blocks", "items": […]}`.

In JSON mode a page’s content is document.page.blocks for every method: a whole-page read is one block carrying text, and a region read is one block per region. See The document block record. <document> attributes are file_name?, file_hash?, file_bytes?, mimetype, num_pages, dpi, language?; <page> attributes are page_index, format, page_width, page_height, and status?. A page that failed keeps its slot as a self-closing <page … status="error"/>. A single image returns the payload block alone, and a multi-document request returns one <document> block per input PDF in request order. JSON mode is unaffected by method: response_format={"type":"json_object"} always returns {"model": …, "method": …, "data": [{"object": "document", "pages": […]}]}, with each page’s content typed by the model and method. See JSON mode for full examples. Each model exposes supported method values on GET /v1/openai/models. See Methods for a per-model reference with examples.

Content parts

Messages use the standard OpenAI multimodal shape. See Multimodal Inputs for the full content part reference (text, image_url, document_url) and document-specific limits.

Response extensions

Non-streaming responses follow the OpenAI chat completion shape with one VLM Run Gateway extension:

Field	Description
`usage`	Token counts; may include `usage.cost` for metering.

Streaming

Set stream: true on any text-mode chat request. The streamed result is byte-identical to the non-streaming one. Document requests (document_url) emit SSE chunks in document order: a <document> open tag, one chunk per <page> block in ascending page order, then the matching </document> close tag. Each <page> carries its own format, so a streamed page is self-describing. Other chat requests (regular chat, single-image OCR) buffer the full reply and re-chunk it into the same OpenAI chat.completion.chunk contract (correct SSE, no time-to-first-token benefit yet). Pass stream_options={"include_usage": true} to receive a terminal chunk carrying aggregated usage (including usage.cost). A JSON response_format is always served non-streamed. See Flexible Document OCR for the full SSE walkthrough.

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

stream = client.chat.completions.create(
    model="qwen/qwen3.5-0.8b",
    stream=True,
    stream_options={"include_usage": True},
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What is happening in this image?"},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.caption/car.jpg"
                    },
                },
            ],
        }
    ],
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="")

curl https://gateway.vlm.run/v1/openai/chat/completions \
  -X POST \
  -H "Authorization: Bearer $VLMRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen/qwen3.5-0.8b",
    "stream": true,
    "stream_options": { "include_usage": true },
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "What is happening in this image?" },
          {
            "type": "image_url",
            "image_url": {
              "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.caption/car.jpg"
            }
          }
        ]
      }
    ]
  }'

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.vlm.run/v1/openai",
  apiKey: process.env.VLMRUN_API_KEY,
});

const stream = await client.chat.completions.create({
  model: "qwen/qwen3.5-0.8b",
  stream: true,
  stream_options: { include_usage: true },
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "What is happening in this image?" },
        {
          type: "image_url",
          image_url: {
            url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.caption/car.jpg",
          },
        },
      ],
    },
  ],
});

for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta?.content;
  if (delta) process.stdout.write(delta);
}

Document OCR

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

response = client.chat.completions.create(
    model="paddleocr/pp-ocrv6",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "document_url",
                    "document_url": {
                        "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf"
                    },
                }
            ],
        }
    ],
    extra_body={
        "method": "ocr",
        "document_dpi": 72,
    },
)

print(response.choices[0].message.content)

curl https://gateway.vlm.run/v1/openai/chat/completions \
  -X POST \
  -H "Authorization: Bearer $VLMRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "paddleocr/pp-ocrv6",
    "method": "ocr",
    "document_dpi": 72,
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "document_url",
            "document_url": {
              "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf"
            }
          }
        ]
      }
    ]
  }'

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.vlm.run/v1/openai",
  apiKey: process.env.VLMRUN_API_KEY,
});

const response = await client.chat.completions.create({
  model: "paddleocr/pp-ocrv6",
  messages: [
    {
      role: "user",
      content: [
        {
          type: "document_url",
          document_url: {
            url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf",
          },
        },
      ],
    },
  ],
  method: "ocr",
  document_dpi: 72,
});

console.log(response.choices[0].message.content);

Stream a document page-by-page

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

stream = client.chat.completions.create(
    model="paddleocr/pp-ocrv6",
    stream=True,
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "document_url",
                    "document_url": {
                        "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf",
                    },
                }
            ],
        }
    ],
    extra_body={
        "method": "ocr",
        "document_dpi": 72,
    },
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="")

curl https://gateway.vlm.run/v1/openai/chat/completions \
  -X POST \
  -H "Authorization: Bearer $VLMRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "paddleocr/pp-ocrv6",
    "stream": true,
    "method": "ocr",
    "document_dpi": 72,
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "document_url",
            "document_url": {
              "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf"
            }
          }
        ]
      }
    ]
  }'

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.vlm.run/v1/openai",
  apiKey: process.env.VLMRUN_API_KEY,
});

const stream = await client.chat.completions.create({
  model: "paddleocr/pp-ocrv6",
  stream: true,
  messages: [
    {
      role: "user",
      content: [
        {
          type: "document_url",
          document_url: {
            url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf",
          },
        },
      ],
    },
  ],
  method: "ocr",
  document_dpi: 72,
});

for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta?.content;
  if (delta) process.stdout.write(delta);
}

Errors and limits

Error codes: capability violations, invalid documents, model not found, and sanitized 500 responses.
Rate limits: 120 requests/min and 1000 requests/hr per IP (anonymous), or 240 requests/min and 10000 requests/hr per user (authenticated).

Every response includes an x-request-id header. You may send your own id on the request; otherwise the VLM Run Gateway mints one. Sanitized 500 responses also include the id as error.request_id in the JSON body.

List models

Query the live model catalog and capabilities.

Models

Full catalog with availability and use-case guidance.

Quickstart

First requests for VQA and document OCR.

Flexible Document OCR

Request knobs, page blocks, and streaming for PDFs.

Multimodal Inputs

Content part types, document limits, and format tradeoffs.

MCP Server

The same read pipeline as tools for MCP-aware agents.

Authorizations

Authorization

string

header

required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json

OpenAI-compatible chat completion request.

model

string

required

messages

ChatMessage · object[]

required

Hide child attributes

messages.role

enum<string>

required

Available options:

system,

user,

assistant,

tool

messages.content

messages.tool_calls

Tool Calls · object[] | null

messages.tool_call_id

string | null

messages.name

string | null

temperature

number

default:0.7

max_tokens

integer | null

top_p

number

default:1

frequency_penalty

number

default:0

presence_penalty

number

default:0

stop

stream

boolean

default:false

integer

default:1

llm

string | null

method

string | null

method_params

Method Params · object | null

video_max_frames

integer | null

video_fps

number | null

video_encoder

string | null

Allowed value: "native"

video_encoder_params

Video Encoder Params · object | null

image_resolution

enum<string> | null

Available options:

224x224,

336x336,

384x384,

448x448,

512x512,

768x768

video_resolution

enum<string> | null

Available options:

256x192,

320x240,

448x336,

512x384,

640x480

document_dpi

integer

default:96

Required range: x >= 1

document_max_pages

integer | null

document_pages

(integer | object)[] | null

document_tool

string | null

response_format

ResponseFormat · object | null

Requested serialization of the assistant reply.

Mirrors the OpenAI response_format object so existing client code keeps working:

{"type": "text"} (the default) returns the reply verbatim.
{"type": "json_object"} returns the reply as a single valid JSON object string: structured results (detections, OCR regions, document pages) are emitted as parseable JSON under a top-level data key.
{"type": "json_schema"} constrains generation to the supplied schema, on the models that declare :attr:ModelCapabilities.supports_json_schema — the chat VLMs served by vLLM and the routed provider models. Anything else (the OCR and detection models, whose reply is an envelope this gateway builds rather than text the model is free to shape) rejects it with a 400 capability violation instead of accepting it and ignoring it.

Hide child attributes

response_format.type

enum<string>

default:text

Available options:

text,

json_object,

json_schema

response_format.json_schema

Json Schema · object | null

precision

integer

default:4

Required range: 1 <= x <= 8

Response

Successful Response

OpenAI-compatible chat completion response.

Beyond the OpenAI shape we surface three extra fields so clients can audit which model actually produced the output:

served_model_id — the canonical id of the deployment that ran the request (e.g. "rednote-hilab/dots.mocr"), even when the caller used a short alias. Mirrors the "id" echoed in GET /v1/openai/models.
backend — static label naming the inference runtime that served the request ("vllm" / "transformers" / "llama.cpp" / "paddleocr" / ...). Lets clients see the runtime without opening a separate /models RPC.
provider — who ran the hardware. Absent for models served on VLM Run's own GPUs; otherwise the upstream that served the request ("google-vertex", "huggingface", ...). Reported per response rather than read from the catalog because a request can be routed to a different upstream than the last one — the same model id may be served by more than one, and they do not always agree on capabilities or price.

The OpenAI-standard model field still echoes the canonical id (matching OpenAI's own behaviour where a generic alias like gpt-4 resolves to the dated snapshot gpt-4-0613 in the response).

model

string

required

choices

Choice · object[]

required

Hide child attributes

choices.message

ChoiceMessage · object

required

Hide child attributes

choices.message.content

string

required

choices.message.role

string

default:assistant

Allowed value: "assistant"

choices.index

integer

default:0

choices.finish_reason

enum<string>

default:stop

Available options:

stop,

length

string

object

string

default:chat.completion

Allowed value: "chat.completion"

created

integer

served_model_id

string | null

backend

string | null

provider

string | null

usage

Usage · object

Hide child attributes

usage.prompt_tokens

integer

default:0

usage.completion_tokens

integer

default:0

usage.total_tokens

integer

default:0

usage.prompt_tokens_details

PromptTokensDetails · object | null

OpenAI-compatible breakdown of prompt token categories.

Hide child attributes

usage.prompt_tokens_details.cached_tokens

integer

default:0

usage.completion_tokens_details

CompletionTokensDetails · object | null

OpenAI-compatible breakdown of completion token categories.

Hide child attributes

usage.completion_tokens_details.reasoning_tokens

integer

default:0

usage.cost

number | null

Overview

Guides

API Reference

Misc

Chat Completions

Models Supported

Gateway extensions

Video encoding

Document text output

Content parts

Response extensions

Streaming

Document OCR

Stream a document page-by-page

Errors and limits

List models

Models

Quickstart

Flexible Document OCR

Multimodal Inputs

MCP Server

Authorizations

Body

Response

​Models Supported

​Gateway extensions

​Video encoding

​Document text output

​Content parts

​Response extensions

​Streaming

​Document OCR

​Stream a document page-by-page

​Errors and limits

​Related

List models

Models

Quickstart

Flexible Document OCR

Multimodal Inputs

MCP Server

Authorizations

Body

Response

Models Supported

Gateway extensions

Video encoding

Document text output

Content parts

Response extensions

Streaming

Document OCR

Stream a document page-by-page

Errors and limits

Related