Models

The VLM Run Gateway serves open-weight vision and document models through a single OpenAI-compatible API. Each model declares the input types it accepts (text, image_url, video_url, document_url) and the operations it supports via the method field. Use the standalone reference for any available model to see its accepted inputs, per-method output shapes, request, and response examples.

​

Output at a glance

Every model returns one of two payload kinds, json or markdown, and response_format decides how that payload is serialized. This holds across every model and method on this page.

• Text mode (response_format omitted, or {"type":"text"}): one block per input medium. A document is a <document> / <page> block; a single image is the bare payload.
• JSON mode ({"type":"json_object"}): one JSON object, {"model", "method", "data"}, with one self-describing entry per input medium.

The chat VLMs are the exemption: they pass their own output through verbatim in both modes. See Methods & Response Format for the full contract, the region schema, and the precision knob.

​

Chat and Visual Question Answering (VQA)

VQA models accept text plus up to 64 images, or one video, in the same message. They return the model’s reply verbatim, with no envelope.

​

Image Understanding

Single-image multi-task models that caption, detect, and OCR from one image_url. They do not accept document_url or video.

​

Document and Image OCR

These models accept a PDF via document_url or an image via image_url. The Gateway rasterizes each PDF page and runs per-page inference, returning one entry per input document. For images, the Gateway runs inference on the entire image. For consistency, structured output is not streamed token-by-token. For documents, every method returns the same <document> / <page> blocks in text mode. format="markdown" on the page marks a Markdown body (markdown, text, and dots.mocr’s ocr), and format="json" marks a region payload (detect, parse_layout, parse_layout_only). In JSON mode a page’s content is always document.page.blocks. See Text mode.

​

Document Layout Detection

Layout-only models locate and classify page regions and predict reading order; they return no recognized text. Same inputs as the OCR models above: an image via image_url or a PDF via document_url, rasterized and detected page by page. detect is the only method: this is a pure layout detector with no OCR path, and any other method is a 400. On an image the payload is the normalized region list (pp_doclayoutv3.detect.regions, with bbox_xywh, optional poly_xy, label, order, score); on a document page it is document.page.blocks with the same geometry under a zero-based index. precision applies to both.

​

Embeddings and Transcription

These models appear on GET /v1/openai/models with a task field other than chat. They use separate OpenAI-compatible endpoints, not chat completions. See Embeddings and Audio Transcriptions for full request schemas.

​

Model Aliases

Many models accept multiple request IDs:

1. Preferred: lowercase <org>/<slug> (listed on /models)
2. Short: slug only (e.g. pp-ocrv6, glm-ocr, dots.mocr, pp-doclayoutv3)
3. Hugging Face: upstream repo id (e.g. zai-org/GLM-OCR)

​

Next steps