Skip to main content
POST
Authentication is optional. See Authentication for tiers and Rate Limits for per-tier quotas.

Request

Response

An OpenAI chat completion, plus usage.cost in USD. The content depends on response_format. See Response Formats and Response Types.

Streaming

Set stream: true in text mode. The streamed text is byte-identical to the non-streaming reply, and a document request emits <document>, one <page> per page, then </document>.
  • Streams deltas: chat VLMs, and single-image OCR methods that return the model’s text unchanged.
  • Buffered: every other reply is built in full, then re-chunked as chat.completion.chunk SSE. That includes rewritten text, paddleocr/pp-ocrv6, structured envelopes, and JSON modes. There is no time-to-first-token benefit.
  • Usage: the last chunk carries aggregated usage, including usage.cost. It arrives whether or not you pass stream_options.include_usage.
See Document OCR for page-by-page streaming.

Errors and limits

Every response includes an x-request-id header. A sanitized 500 body repeats it as error.request_id. See Error Codes and Rate Limits.

List models

Query the live model catalog and capabilities.

Response Formats

Text, JSON, and JSON schema modes.

Supported Inputs

Content part types, document limits, and format tradeoffs.

Document OCR

Request knobs, page blocks, and streaming for PDFs.

Authorizations

Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json

OpenAI-compatible chat completion request.

model
string
required
messages
ChatMessage · object[]
required
temperature
number
default:0.7
max_tokens
integer | null
top_p
number
default:1
frequency_penalty
number
default:0
presence_penalty
number
default:0
stop
stream
boolean
default:false
n
integer
default:1
reasoning_effort
enum<string> | null
default:medium
Available options:
none,
minimal,
low,
medium,
high
chat_template_kwargs
Chat Template Kwargs · object | null
llm
string | null
method
string | null
method_params
Method Params · object | null
video_max_frames
integer | null
video_fps
number | null
video_encoder
string | null
Allowed value: "native"
video_encoder_params
Video Encoder Params · object | null
image_resolution
enum<string> | null
Available options:
224x224,
336x336,
384x384,
448x448,
512x512,
768x768
video_resolution
enum<string> | null
Available options:
256x192,
320x240,
448x336,
512x384,
640x480
document_dpi
integer
default:96
Required range: x >= 1
document_max_pages
integer | null
document_pages
(integer | object)[] | null
document_tool
string | null
response_format
ResponseFormat · object | null

Requested serialization of the assistant reply.

Mirrors the OpenAI response_format object so existing client code keeps working:

  • {"type": "text"} (the default) returns the reply verbatim.
  • {"type": "json_object"} returns the reply as a single valid JSON object string: structured results (detections, OCR regions, document pages) are emitted as parseable JSON under a top-level data key.
  • {"type": "json_schema"} constrains generation to the supplied schema, on the models that declare :attr:ModelCapabilities.supports_json_schema — the chat VLMs served by vLLM and the routed provider models. Anything else (the OCR and detection models, whose reply is an envelope this gateway builds rather than text the model is free to shape) rejects it with a 400 capability violation instead of accepting it and ignoring it.
precision
integer
default:4
Required range: 1 <= x <= 8

Response

Successful Response

OpenAI-compatible chat completion response.

Beyond the OpenAI shape we surface three extra fields so clients can audit which model actually produced the output:

  • served_model_id — the canonical id of the deployment that ran the request (e.g. "rednote-hilab/dots.mocr"), even when the caller used a short alias. Mirrors the "id" echoed in GET /v1/openai/models.
  • backend — static label naming the inference runtime that served the request ("vllm" / "transformers" / "llama.cpp" / "paddleocr" / ...). Lets clients see the runtime without opening a separate /models RPC.
  • provider — who ran the hardware. Absent for models served on VLM Run's own GPUs; otherwise the upstream that served the request ("google-vertex", "huggingface", ...). Reported per response rather than read from the catalog because a request can be routed to a different upstream than the last one — the same model id may be served by more than one, and they do not always agree on capabilities or price.

The OpenAI-standard model field still echoes the canonical id (matching OpenAI's own behaviour where a generic alias like gpt-4 resolves to the dated snapshot gpt-4-0613 in the response).

model
string
required
choices
Choice · object[]
required
id
string
object
string
default:chat.completion
Allowed value: "chat.completion"
created
integer
served_model_id
string | null
backend
string | null
provider
string | null
usage
Usage · object