Models

The VLM Run Gateway serves open-weight vision and document models through a single OpenAI-compatible API. Each model declares the input types it accepts (text, image_url, document_url) and the operations it supports via the method field.

Visual Question Answering (VQA)

VQA models accept text plus up to 64 images, one video, or a PDF in the same message. PDFs fan out per-page chat instead of OCR.

Model	Status	Method	Accepted Inputs	Max images / videos
`qwen/qwen3.5-0.8b`	Available	—	`text`, `image_url`, `video_url`, `document_url`	64 images, 1 video
`google/gemma-4-26b-a4b`	Coming soon	—	—	—
`qwen/qwen3.6-35b-a3b`	Coming soon	—	—	—

See method parameters for more details.

qwen/qwen3.5-0.8b
Video Q&A

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

response = client.chat.completions.create(
    model="qwen/qwen3.5-0.8b",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What is happening in this image?"},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.caption/car.jpg"
                    },
                },
            ],
        }
    ],
)

print(response.choices[0].message.content)

curl https://gateway.vlm.run/v1/openai/chat/completions \
  -X POST \
  -H "Authorization: Bearer $VLMRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen/qwen3.5-0.8b",
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "What is happening in this image?" },
          {
            "type": "image_url",
            "image_url": {
              "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.caption/car.jpg"
            }
          }
        ]
      }
    ]
  }'

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

response = client.chat.completions.create(
    model="qwen/qwen3.5-0.8b",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Summarize what happens in this video."},
                {
                    "type": "video_url",
                    "video_url": {
                        "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/video.transcription/bakery.mp4"
                    },
                },
            ],
        }
    ],
    extra_body={"video_fps": 1.0, "video_max_frames": 8},
)

print(response.choices[0].message.content)

curl https://gateway.vlm.run/v1/openai/chat/completions \
  -X POST \
  -H "Authorization: Bearer $VLMRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen/qwen3.5-0.8b",
    "video_fps": 1.0,
    "video_max_frames": 8,
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "Summarize what happens in this video." },
          {
            "type": "video_url",
            "video_url": {
              "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/video.transcription/bakery.mp4"
            }
          }
        ]
      }
    ]
  }'

Document and Image OCR

These models accept a PDF via document_url or an image via image_url. The VLM Run Gateway rasterizes each page and runs per-page inference, returning <document>-wrapped page blocks, concatenated (non-streaming) or streamed in document order (streaming). Use method: "ocr" for full OCR, or method: "detect" when you only need text detection and bounding polygons. zai-org/glm-ocr and rednote-hilab/dots.mocr support both OCR and reading-order markdown output; dots.mocr also exposes layout parsing methods. For images, the Gateway runs inference on the entire image. For consistency purposes, we do not stream on a token-by-token basis.

Model	Status	Default method	Other methods	Accepted Inputs	Streaming
`paddleocr/pp-ocrv6`	Available	`ocr`	`detect`, `markdown`	`image_url`, `document_url`	Page-wise
`zai-org/glm-ocr`	Available	`ocr`	`markdown`	`image_url`, `document_url`	Page-wise
`rednote-hilab/dots.mocr`	Available	`parse_layout`	`parse_layout_only`, `ocr`, `markdown`	`image_url`, `document_url`	Page-wise

See method parameters for more details.

paddleocr/pp-ocrv6
zai-org/glm-ocr
rednote-hilab/dots.mocr

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

response = client.chat.completions.create(
    model="paddleocr/pp-ocrv6",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "document_url",
                    "document_url": {
                        "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf"
                    },
                },
            ],
        }
    ],
    extra_body={"method": "ocr", "document_dpi": 72},
)

print(response.choices[0].message.content)

curl https://gateway.vlm.run/v1/openai/chat/completions \
  -X POST \
  -H "Authorization: Bearer $VLMRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "paddleocr/pp-ocrv6",
    "method": "ocr",
    "document_dpi": 72,
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "document_url",
            "document_url": {
              "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf"
            }
          }
        ]
      }
    ]
  }'

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

response = client.chat.completions.create(
    model="zai-org/glm-ocr",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "document_url",
                    "document_url": {
                        "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf"
                    },
                },
            ],
        }
    ],
    extra_body={"method": "markdown", "document_dpi": 72},
)

print(response.choices[0].message.content)

curl https://gateway.vlm.run/v1/openai/chat/completions \
  -X POST \
  -H "Authorization: Bearer $VLMRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zai-org/glm-ocr",
    "method": "markdown",
    "document_dpi": 72,
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "document_url",
            "document_url": {
              "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf"
            }
          }
        ]
      }
    ]
  }'

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

response = client.chat.completions.create(
    model="rednote-hilab/dots.mocr",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "document_url",
                    "document_url": {
                        "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf"
                    },
                },
            ],
        }
    ],
    extra_body={"method": "parse_layout", "document_dpi": 72},
)

print(response.choices[0].message.content)

curl https://gateway.vlm.run/v1/openai/chat/completions \
  -X POST \
  -H "Authorization: Bearer $VLMRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "rednote-hilab/dots.mocr",
    "method": "parse_layout",
    "document_dpi": 72,
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "document_url",
            "document_url": {
              "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf"
            }
          }
        ]
      }
    ]
  }'

Embeddings and Transcription

These models appear on GET /v1/openai/models with a task field other than chat. They use separate OpenAI-compatible endpoints, not chat completions. See Embeddings and Audio Transcriptions for full request schemas.

Model	Status	Task	Endpoint
`qwen/qwen3-vl-embedding-2b`	Available	`embed`	`POST /v1/openai/embeddings` (text, `image_url`, `video_url`)
`nvidia/parakeet-tdt-0.6b-v3`	Available	`transcribe`	`POST /v1/openai/audio/transcriptions`

qwen/qwen3-vl-embedding-2b
nvidia/parakeet-tdt-0.6b-v3

Python

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

response = client.embeddings.create(
    model="qwen/qwen3-vl-embedding-2b",
    input="Extract a vector representation for this text.",
)

print(len(response.data[0].embedding))

Python

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

with open("sample.mp3", "rb") as audio_file:
    response = client.audio.transcriptions.create(
        model="nvidia/parakeet-tdt-0.6b-v3",
        file=audio_file,
    )

print(response.text)

Model Aliases

Many models accept multiple request IDs:

Preferred: lowercase <org>/<slug> (listed on /models)
Short: slug only (e.g. pp-ocrv6)
Hugging Face: upstream repo id (e.g. PaddlePaddle/PP-OCRv6_medium_det)

Next steps

Flexible Document OCR

End-to-end recipe from model selection to response parsing.

Methods

Per-model method and method_params reference.

Chat Completions API

Full request parameters, streaming, and error handling.

Error Codes

HTTP status mapping, correlation ids, and support details.

Overview

Guides

API Reference

Misc

Visual Question Answering (VQA)

Document and Image OCR

Embeddings and Transcription

Model Aliases

Next steps

Flexible Document OCR

Methods

Chat Completions API

Error Codes

​Visual Question Answering (VQA)

​Document and Image OCR

​Embeddings and Transcription

​Model Aliases

​Next steps

Flexible Document OCR

Methods

Chat Completions API

Error Codes

Visual Question Answering (VQA)

Document and Image OCR

Embeddings and Transcription

Model Aliases

Next steps