Models

The VLM Run Gateway serves open-weight vision and document models through a single OpenAI-compatible API. Each model declares the input types it accepts (text, image_url, video_url, document_url) and the operations it supports via the method field. Available means that the model is in the served catalog. An individual request can still be aborted with a 504 inference timeout. Use the standalone reference for any available model to see its accepted inputs, per-method output shapes, request, and response examples.

​

Output at a glance

Every model returns one of two payload kinds, json or markdown, and response_format decides how that payload is serialized. This holds across every model and method on this page.

• Text mode (response_format omitted, or {"type":"text"}): one block per input medium. A document is a <document> / <page> block; a single image is the bare payload.
• JSON mode ({"type":"json_object"}): one JSON object, {"model", "method", "data"}, with one self-describing entry per input medium.

The chat VLMs are the exemption: they pass their own output through verbatim in both modes. baidu/unlimited-ocr and paddlepaddle/paddleocr-vl-1.6 keep the block-per-medium shape but use model-specific attribute and key names. See Methods & Response Format for the full contract, the region schema, and the precision knob.

​

Chat and Visual Question Answering (VQA)

VQA models accept text plus up to 64 images, or one video, in the same message. They return the model’s reply verbatim, with no envelope.

​

Image Understanding

Single-image multi-task models that caption, detect, and OCR from one image_url. They do not accept document_url or video.

​

Document and Image OCR

These models accept a PDF via document_url or an image via image_url. The Gateway rasterizes each PDF page and runs per-page inference, returning one entry per input document. For images, the Gateway runs inference on the entire image. For consistency, structured output is not streamed token-by-token. For documents, OCR methods usually return the same <document> / <page> blocks in text mode. format="markdown" on the page marks a Markdown body (markdown, text, free_ocr, and the ocr of dots.mocr and deepseek-ocr-2), and format="json" marks a region payload (detect, grounding_ocr, parse_layout, parse_layout_only). baidu/unlimited-ocr and paddlepaddle/paddleocr-vl-1.6 use a model-specific envelope; see unlimited-ocr and paddleocr-vl-1.6 for the exact shapes, and Text mode for the shared contract. PaddleOCR-VL 1.6 is the only model in the catalog that also reads a page’s structured content: its table, formula, and chart methods return Markdown tables, LaTeX, and a described series respectively, in the same envelope as ocr.

​

Pose Estimation

2D human pose estimation over one image_url or one video_url. Each detected person returns a bounding box and 17 normalized COCO keypoints.

​

Embeddings and Transcription

These models appear on GET /v1/openai/models with a task field other than chat. They use separate OpenAI-compatible endpoints, not chat completions. See Embeddings and Audio Transcriptions for full request schemas.

​

Model Aliases

Many models accept multiple request IDs:

1. Preferred: lowercase <org>/<slug> (listed on /models)
2. Short: slug only (e.g. pp-ocrv6, glm-ocr, dots.mocr, deepseek-ocr-2, paddleocr-vl-1.6, infinity-parser2-flash)
3. Hugging Face: upstream repo id (e.g. zai-org/GLM-OCR)

​

Routed models

The Gateway also routes to frontier vision AI models via upstream providers, so you can use the same API key and baseURL to access them as well. These models are passthrough VLMs supporting the same methods and method_params reported by their providers. Prices are the provider’s listed prices in USD per 1M tokens. See all 9 routed models →

​

Next steps