gateway - VLM Run

The vlmrun gateway command (aliased as vlmrun gw) runs OpenAI-compatible OCR and vision-language models hosted behind the VLM Run Gateway (https://gateway.vlm.run/v1), authenticating with the same VLMRUN_API_KEY used everywhere else. Unlike vlmrun chat (which uploads to the Files API and calls the Orion agent), the gateway is a raw passthrough to third-party models. Input files are sent inline as base64 data: URLs, with documents attached as document_url content parts and images as image_url.

Most gateway models, especially OCR models such as zai-org/glm-ocr and paddleocr/pp-ocrv6, do not accept text-only input, so vlmrun gw chat requires at least one input file.

Health

Check gateway reachability:

vlmrun gw health

Models

List available models with their methods, or detail a single model with copy-pasteable example commands:

# List every model with its methods
vlmrun gw models

# Methods, params, and example commands for one model
vlmrun gw models pp-ocrv6

# Raw JSON catalog
vlmrun gw models --json

The default method for each model is marked with *.

Chat

Run an OCR / VLM model over one or more documents or images:

# Parse a document (PDF -> text/markdown)
vlmrun gw chat document.pdf -m zai-org/glm-ocr

# Multiple documents
vlmrun gw chat doc1.pdf doc2.pdf -m paddleocr/pp-ocrv6

# OCR an image
vlmrun gw chat scan.jpg -m paddleocr/pp-ocrv6

# Prompt a model that supports text input
vlmrun gw chat image.jpg -p "describe this image" -m qwen/qwen3.5-0.8b

Methods

Each model exposes one or more methods (e.g. ocr, detect, markdown) with a default. Run vlmrun gw models <model> to see a model’s methods and params.

vlmrun gw chat img.jpg -m pp-ocrv6 --method detect

vlmrun gw chat img.jpg -m pp-ocrv6 --method ocr \
    --method-params '{"lang": "en", "score_threshold": 0.5}'

Extra completion kwargs

Forward extra chat.completions.create() kwargs as repeatable key=value pairs (values are JSON-parsed). Unknown keys are routed through extra_body to reach the gateway as top-level request fields.

vlmrun gw chat document.pdf -m zai-org/glm-ocr -e temperature=0 -e max_tokens=4096

Options

vlmrun gw chat [OPTIONS] FILES...

Arguments:
  FILES...                  Input document/image file(s) to process. Repeatable.

Options:
  -m, --model TEXT          Gateway model id (e.g. zai-org/glm-ocr) [required]
  -p, --prompt TEXT         Optional text prompt (only for models that support text input)
  -M, --method TEXT         Model method, e.g. ocr, detect, markdown
  --method-params TEXT      JSON object of method arguments, e.g. '{"lang": "en"}'
  -e, --extra TEXT          Extra create() kwarg as key=value (repeatable)
  -ns, --no-stream          Disable streaming
  -j, --json                Output raw JSON
  --timeout FLOAT           Request timeout in seconds
  --help                    Show this message and exit

Embed

Embed text, images, or video with a gateway embedding model. Every file and every -t/--text is embedded as its own vector; use --join to embed them together as a single vector (at most one file).

# Embed text
vlmrun gw embed -t "a blue parrot" -m qwen/qwen3-vl-embedding-2b

# Embed an image
vlmrun gw embed photo.jpg -m qwen/qwen3-vl-embedding-2b

# Embed an image and caption jointly
vlmrun gw embed photo.jpg -t "caption" --join -m qwen/qwen3-vl-embedding-2b

# Truncate vectors to a fixed number of dimensions
vlmrun gw embed -t "hi" -m qwen/qwen3-vl-embedding-2b --dimensions 64

# Output full vectors as JSON
vlmrun gw embed photo.jpg -m qwen/qwen3-vl-embedding-2b --json

Transcribe

Transcribe audio, or a video whose audio track is transcribed:

# Transcribe a local file
vlmrun gw transcribe clip.mp3 -m nvidia/parakeet-tdt-0.6b-v3

# Choose a response format (json, text, verbose_json, srt, vtt)
vlmrun gw transcribe clip.mp3 -m nvidia/parakeet-tdt-0.6b-v3 -f srt

# Provide a language hint
vlmrun gw transcribe clip.mp3 -m nvidia/parakeet-tdt-0.6b-v3 --language en

# Transcribe a hosted URL instead of a local file
vlmrun gw transcribe --url https://example.com/a.mp3 -m nvidia/parakeet-tdt-0.6b-v3

Command Reference

Command	Description
`vlmrun gw health`	Check gateway reachability
`vlmrun gw models [MODEL]`	List models, or detail one model (`--json` for raw output)
`vlmrun gw chat FILES... -m MODEL`	Run a model over one or more files
`vlmrun gw embed [FILES...] -m MODEL`	Embed text, images, or video
`vlmrun gw transcribe FILE -m MODEL`	Transcribe audio

Configuration

Override the gateway base URL with the VLMRUN_GATEWAY_URL environment variable (default: https://gateway.vlm.run/v1):

export VLMRUN_GATEWAY_URL="https://gateway.vlm.run/v1"

See the Gateway documentation for the full model catalog and methods reference, and client.gateway for the Python SDK equivalent.

​Health

​Models

​Chat

​Methods

​Extra completion kwargs

​Options

​Embed

​Transcribe

​Command Reference

​Configuration

Health

Models

Chat

Methods

Extra completion kwargs

Options

Embed

Transcribe

Command Reference

Configuration