Chat Completions

import requests

url = "https://gateway.vlm.run/v1/openai/chat/completions"

payload = {}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({})
};

fetch('https://gateway.vlm.run/v1/openai/chat/completions', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

require 'uri'
require 'net/http'

url = URI("https://gateway.vlm.run/v1/openai/chat/completions")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Post.new(url)
request["Authorization"] = 'Bearer <token>'
request["Content-Type"] = 'application/json'
request.body = "{}"

response = http.request(request)
puts response.read_body

curl --request POST \
  --url https://gateway.vlm.run/v1/openai/chat/completions \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{}'

{
  "model": "<string>",
  "choices": [
    {
      "message": {
        "content": "<string>",
        "role": "assistant"
      },
      "index": 0,
      "finish_reason": "stop"
    }
  ],
  "id": "<string>",
  "object": "chat.completion",
  "created": 123,
  "served_model_id": "<string>",
  "backend": "<string>",
  "usage": {
    "prompt_tokens": 0,
    "completion_tokens": 0,
    "total_tokens": 0
  }
}

{
  "detail": [
    {
      "loc": [
        "<string>"
      ],
      "msg": "<string>",
      "type": "<string>",
      "input": "<unknown>",
      "ctx": {}
    }
  ]
}

POST

openai

chat

completions

Chat Completions

import requests

url = "https://gateway.vlm.run/v1/openai/chat/completions"

payload = {}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({})
};

fetch('https://gateway.vlm.run/v1/openai/chat/completions', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

require 'uri'
require 'net/http'

url = URI("https://gateway.vlm.run/v1/openai/chat/completions")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Post.new(url)
request["Authorization"] = 'Bearer <token>'
request["Content-Type"] = 'application/json'
request.body = "{}"

response = http.request(request)
puts response.read_body

curl --request POST \
  --url https://gateway.vlm.run/v1/openai/chat/completions \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{}'

{
  "model": "<string>",
  "choices": [
    {
      "message": {
        "content": "<string>",
        "role": "assistant"
      },
      "index": 0,
      "finish_reason": "stop"
    }
  ],
  "id": "<string>",
  "object": "chat.completion",
  "created": 123,
  "served_model_id": "<string>",
  "backend": "<string>",
  "usage": {
    "prompt_tokens": 0,
    "completion_tokens": 0,
    "total_tokens": 0
  }
}

{
  "detail": [
    {
      "loc": [
        "<string>"
      ],
      "msg": "<string>",
      "type": "<string>",
      "input": "<unknown>",
      "ctx": {}
    }
  ]
}

Authentication is optional for the VLM Run Gateway (at the moment). See Authentication for tiers and Rate Limits for per-tier quotas.

Models Supported

For available models and aliases, see Models. Standard OpenAI sampling fields (temperature, max_tokens, top_p, frequency_penalty, presence_penalty, stop, n) are accepted on every request, but OCR and detection models generally ignore sampling params since they are not free-form text generators. Check a model’s supported_parameters via Get Model if a field you send does not appear to change the output.

Gateway extensions

These fields are accepted at the top level of the request body (or via extra_body in the OpenAI Python SDK):

Field	Type	Default	Description
`method`	`string`	model default	Model-specific operation (`ocr`, `detect`, `markdown`, …).
`method_params`	`object`	`null`	Keyword arguments for the selected `method`.
`llm`	`string`	`null`	Optional LLM for post-processing structured model output.
`document_dpi`	`integer`	`72`	DPI when rasterizing PDF pages.
`document_max_pages`	`integer`	`500`	Maximum PDF pages per request.
`image_resolution`	`string`	`null`	Square resize preset (`224x224` … `768x768`).
`video_max_frames`	`integer`	`null`	Cap on frames sampled from `video_url` inputs. Used by `qwen/qwen3.5-0.8b`.
`video_fps`	`number`	`null`	Target frames per second to sample from `video_url` inputs. Used by `qwen/qwen3.5-0.8b`.
`video_resolution`	`string`	`null`	Resize preset for sampled video frames (`256x192` … `640x480`). Used by `qwen/qwen3.5-0.8b`.

video_max_frames, video_fps, and video_resolution apply when the request includes a video_url content part on a video-capable model. See Video Inputs. Each model exposes supported method values on GET /v1/openai/models. See Methods for a per-model reference with examples.

Content parts

Messages use the standard OpenAI multimodal shape. See Multimodal Inputs for the full content part reference (text, image_url, document_url) and document-specific limits.

Response extensions

Non-streaming responses follow the OpenAI chat completion shape with one VLM Run Gateway extension:

Field	Description
`usage`	Token counts; may include `usage.cost` for metering.

Streaming

Set stream: true on document PDF requests (document_url). The VLM Run Gateway emits SSE chunks in document order: a <document> open tag, one chunk per <page> block in ascending page order, then the matching </document> close tag. A final chunk with aggregated usage follows. Image-only and text-only requests do not support streaming on the Gateway today. See Flexible Document OCR for the full SSE walkthrough.

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

response = client.chat.completions.create(
    model="qwen/qwen3.5-0.8b",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What is happening in this image?"},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.caption/car.jpg"
                    },
                },
            ],
        }
    ],
)

print(response.choices[0].message.content)

curl https://gateway.vlm.run/v1/openai/chat/completions \
  -X POST \
  -H "Authorization: Bearer $VLMRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen/qwen3.5-0.8b",
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "What is happening in this image?" },
          {
            "type": "image_url",
            "image_url": {
              "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.caption/car.jpg"
            }
          }
        ]
      }
    ]
  }'

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.vlm.run/v1/openai",
  apiKey: process.env.VLMRUN_API_KEY,
});

const response = await client.chat.completions.create({
  model: "qwen/qwen3.5-0.8b",
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "What is happening in this image?" },
        {
          type: "image_url",
          image_url: {
            url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.caption/car.jpg",
          },
        },
      ],
    },
  ],
});

console.log(response.choices[0].message.content);

Document OCR

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

response = client.chat.completions.create(
    model="paddleocr/pp-ocrv6",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "document_url",
                    "document_url": {
                        "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf"
                    },
                }
            ],
        }
    ],
    extra_body={
        "method": "ocr",
        "document_dpi": 72,
    },
)

print(response.choices[0].message.content)

curl https://gateway.vlm.run/v1/openai/chat/completions \
  -X POST \
  -H "Authorization: Bearer $VLMRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "paddleocr/pp-ocrv6",
    "method": "ocr",
    "document_dpi": 72,
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "document_url",
            "document_url": {
              "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf"
            }
          }
        ]
      }
    ]
  }'

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.vlm.run/v1/openai",
  apiKey: process.env.VLMRUN_API_KEY,
});

const response = await client.chat.completions.create({
  model: "paddleocr/pp-ocrv6",
  messages: [
    {
      role: "user",
      content: [
        {
          type: "document_url",
          document_url: {
            url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf",
          },
        },
      ],
    },
  ],
  method: "ocr",
  document_dpi: 72,
});

console.log(response.choices[0].message.content);

Stream a document page-by-page

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

stream = client.chat.completions.create(
    model="paddleocr/pp-ocrv6",
    stream=True,
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "document_url",
                    "document_url": {
                        "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf",
                    },
                }
            ],
        }
    ],
    extra_body={
        "method": "ocr",
        "document_dpi": 72,
    },
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="")

curl https://gateway.vlm.run/v1/openai/chat/completions \
  -X POST \
  -H "Authorization: Bearer $VLMRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "paddleocr/pp-ocrv6",
    "stream": true,
    "method": "ocr",
    "document_dpi": 72,
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "document_url",
            "document_url": {
              "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf"
            }
          }
        ]
      }
    ]
  }'

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.vlm.run/v1/openai",
  apiKey: process.env.VLMRUN_API_KEY,
});

const stream = await client.chat.completions.create({
  model: "paddleocr/pp-ocrv6",
  stream: true,
  messages: [
    {
      role: "user",
      content: [
        {
          type: "document_url",
          document_url: {
            url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf",
          },
        },
      ],
    },
  ],
  method: "ocr",
  document_dpi: 72,
});

for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta?.content;
  if (delta) process.stdout.write(delta);
}

Errors and limits

Error codes: capability violations, invalid documents, model not found, and sanitized 500 responses.
Rate limits: 60 requests/min and 1000 requests/hr per IP (anonymous) or per user (authenticated).

Every response includes an x-request-id header. You may send your own id on the request; otherwise the VLM Run Gateway mints one. Sanitized 500 responses also include the id as error.request_id in the JSON body.

List models

Query the live model catalog and capabilities.

Models

Full catalog with availability and use-case guidance.

Quickstart

First requests for VQA and document OCR.

Flexible Document OCR

Request knobs, page blocks, and streaming for PDFs.

Multimodal Inputs

Content part types, document limits, and format tradeoffs.

Authorizations

Authorization

string

header

required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json

OpenAI-compatible chat completion request.

model

string

required

messages

ChatMessage · object[]

required

Hide child attributes

messages.role

enum<string>

required

Available options:

system,

user,

assistant

messages.content

required

temperature

number

default:0.7

max_tokens

integer | null

top_p

number

default:1

frequency_penalty

number

default:0

presence_penalty

number

default:0

stop

stream

boolean

default:false

integer

default:1

llm

string | null

method

string | null

method_params

Method Params · object | null

video_max_frames

integer | null

video_fps

number | null

image_resolution

enum<string> | null

Available options:

224x224,

336x336,

384x384,

448x448,

512x512,

768x768

video_resolution

enum<string> | null

Available options:

256x192,

320x240,

448x336,

512x384,

640x480

document_dpi

integer | null

document_max_pages

integer | null

document_tool

string | null

Response

Successful Response

OpenAI-compatible chat completion response.

Beyond the OpenAI shape we surface two extra fields so clients can audit which model actually produced the output:

served_model_id — the canonical id of the deployment that ran the request (e.g. "rednote-hilab/dots.ocr"), even when the caller used a short alias. Mirrors the "id" echoed in GET /v1/openai/models.
backend — static label naming the inference runtime that served the request ("vllm" / "transformers" / "llama.cpp" / "paddleocr" / ...). Lets clients see the runtime without opening a separate /models RPC.

The OpenAI-standard model field still echoes the canonical id (matching OpenAI's own behaviour where a generic alias like gpt-4 resolves to the dated snapshot gpt-4-0613 in the response).

model

string

required

choices

Choice · object[]

required

Hide child attributes

choices.message

ChoiceMessage · object

required

Hide child attributes

choices.message.content

string

required

choices.message.role

string

default:assistant

Allowed value: "assistant"

choices.index

integer

default:0

choices.finish_reason

enum<string>

default:stop

Available options:

stop,

length

string

object

string

default:chat.completion

Allowed value: "chat.completion"

created

integer

served_model_id

string | null

backend

string | null

usage

Usage · object

Hide child attributes

usage.prompt_tokens

integer

default:0

usage.completion_tokens

integer

default:0

usage.total_tokens

integer

default:0

usage.prompt_tokens_details

PromptTokensDetails · object | null

OpenAI-compatible breakdown of prompt token categories.

Hide child attributes

usage.prompt_tokens_details.cached_tokens

integer

default:0

usage.completion_tokens_details

CompletionTokensDetails · object | null

OpenAI-compatible breakdown of completion token categories.

Hide child attributes

usage.completion_tokens_details.reasoning_tokens

integer

default:0

usage.cost

number | null

Health List Models

Overview

Guides

API Reference

Misc

Chat Completions

Models Supported

Gateway extensions

Content parts

Response extensions

Streaming

Document OCR

Stream a document page-by-page

Errors and limits

List models

Models

Quickstart

Flexible Document OCR

Multimodal Inputs

Authorizations

Body

Response

​Models Supported

​Gateway extensions

​Content parts

​Response extensions

​Streaming

​Document OCR

​Stream a document page-by-page

​Errors and limits

​Related

List models

Models

Quickstart

Flexible Document OCR

Multimodal Inputs

Authorizations

Body

Response

Models Supported

Gateway extensions

Content parts

Response extensions

Streaming

Document OCR

Stream a document page-by-page

Errors and limits

Related