gateway - VLM Run

The vlmrun gateway command (alias vlmrun gw) runs OpenAI-compatible OCR and vision-language models on the VLM Run Gateway (https://gateway.vlm.run/v1), using the same VLMRUN_API_KEY as the rest of the CLI. Unlike vlmrun chat, it is a raw passthrough: no Files API upload, no Orion agent. Every input is a local file path or an http(s) URL. URLs pass by reference; local files are inlined as base64 data: URLs. The CLI picks the content part from the media type:

Media	Content part	Matches
Document	`document_url`	`.pdf`, `.doc`, `.docx`
Image	`image_url`	any image type: `.jpg`, `.png`, `.webp`, …
Video	`video_url`	any video type: `.mp4`, `.mov`, `.mkv`, …
Other	`file_url`	everything else

gw chat needs a file, a URL, or -p/--prompt. OCR models such as zai-org/glm-ocr and paddleocr/pp-ocrv6 reject text-only input, so use -p alone only with chat models like qwen/qwen3.5-0.8b.

Model ids are the full <org>/<name> reported by vlmrun gw models. Short aliases (for example pp-ocrv6 for paddleocr/pp-ocrv6) also work.

Health

Check gateway reachability. The command exits non-zero when the check fails.

vlmrun gw health

Models

List models, or detail one with runnable examples:

# List every model with its methods
vlmrun gw models

# Methods, params, and example commands for one model
vlmrun gw models pp-ocrv6

# Raw JSON catalog
vlmrun gw models --json

The default method is marked *. --json returns the raw catalog records, and adds a commands array on a detail view. See Gateway Models for the maintained catalog.

paddleocr/pp-ocrv6

task      chat
methods   ocr*, detect, text
inputs    image_url, document_url, file_url, file
aliases   pp-ocrv6
note      document_url PDF (plain text per page)
examples
├── vlmrun gw chat doc.pdf -m paddleocr/pp-ocrv6 --method ocr
├── vlmrun gw chat doc.pdf -m paddleocr/pp-ocrv6 --method text
└── vlmrun gw chat doc.pdf -m paddleocr/pp-ocrv6 --method ocr --method-params '{"lang": "en", "score_threshold": 0.5}'

Embed

Embed text, images, or video with a gateway embedding model. Each file and each -t/--text becomes its own vector; --join combines them into one (at most one file). File inputs must be images or video: use -t for text.

# Embed text
vlmrun gw embed -t "a blue parrot" -m qwen/qwen3-vl-embedding-2b

# Embed an image
vlmrun gw embed photo.jpg -m qwen/qwen3-vl-embedding-2b

# Embed an image and caption jointly
vlmrun gw embed photo.jpg -t "caption" --join -m qwen/qwen3-vl-embedding-2b

# Truncate vectors to a fixed number of dimensions
vlmrun gw embed -t "hi" -m qwen/qwen3-vl-embedding-2b --dimensions 64

# Output full vectors as JSON
vlmrun gw embed photo.jpg -m qwen/qwen3-vl-embedding-2b --json

Without --json the CLI prints one row per input with the vector’s dimensionality and first four components, under a model · T:<total> tokens · <latency>s footer. qwen/qwen3-vl-embedding-2b returns 2048 dimensions by default; --dimensions truncates that vector.

Options

Option	Short	Description
`FILES...`		Image/video file(s) to embed. Repeatable
`--model`	`-m`	Embedding model id (required)
`--text`	`-t`	Text to embed (repeatable)
`--join`		Embed all inputs together as one vector (max one file)
`--dimensions`	`-d`	Truncate vectors to this many dimensions
`--json`	`-j`	Output raw JSON, including full vectors
`--timeout`		Request timeout in seconds

Transcribe

Transcribe audio, or a video whose audio track is transcribed:

# Transcribe a local file
vlmrun gw transcribe clip.mp3 -m nvidia/parakeet-tdt-0.6b-v3

# Transcribe a video's audio track
vlmrun gw transcribe clip.mp4 -m nvidia/parakeet-tdt-0.6b-v3

# Choose a response format (json, text, verbose_json, srt, vtt)
vlmrun gw transcribe clip.mp3 -m nvidia/parakeet-tdt-0.6b-v3 -f srt

# Provide a language hint
vlmrun gw transcribe clip.mp3 -m nvidia/parakeet-tdt-0.6b-v3 --language en

# Bias the transcript toward known proper nouns
vlmrun gw transcribe clip.mp3 -m nvidia/parakeet-tdt-0.6b-v3 -p "VLM Run, Orion"

# Transcribe a hosted URL instead of a local file
vlmrun gw transcribe --url https://example.com/a.mp3 -m nvidia/parakeet-tdt-0.6b-v3

Pass either a file or --url, not both. The transcript prints in a Transcript panel with a model · <format> · <latency>s footer.

Options

Option	Short	Description
`FILE`		Audio file, or a video whose audio track is transcribed
`--model`	`-m`	Transcription model id (required)
`--url`		Hosted audio URL instead of a local file
`--format`	`-f`	Response format: `json`, `text`, `verbose_json`, `srt`, `vtt` (default: `json`)
`--language`	`-l`	ISO-639-1 language hint, e.g. `en`
`--prompt`	`-p`	Context to bias transcription (proper nouns)
`--json`	`-j`	Output raw JSON
`--timeout`		Request timeout in seconds

Command Reference

Command	Description
`vlmrun gw health`	Check gateway reachability (exits non-zero when unreachable)
`vlmrun gw models [MODEL]`	List models, or detail one model (`-j, --json` for raw output)
`vlmrun gw chat INPUTS... -m MODEL`	Run a model over one or more files or URLs
`vlmrun gw embed [FILES...] -m MODEL`	Embed text, images, or video
`vlmrun gw transcribe [FILE] -m MODEL`	Transcribe audio (or `--url`)
`vlmrun gw systemone [INPUTS...]`	Typed, calibrated decisions (alias `vlmrun gw s1`)

Configuration

Variable	Purpose	Default
`VLMRUN_API_KEY`	Gateway authentication, shared with the rest of the CLI	none
`VLMRUN_GATEWAY_URL`	Gateway base URL	`https://gateway.vlm.run/v1`

export VLMRUN_API_KEY="your-api-key"
export VLMRUN_GATEWAY_URL="https://gateway.vlm.run/v1"

The OpenAI-compatible surface is that base URL plus /openai, which is the base_url to give an OpenAI SDK client directly. Gateway requests default to a 600 second timeout instead of the CLI’s 120, since multi-page OCR runs long; --timeout overrides it either way. See the Gateway documentation for the full model catalog and methods reference, and client.gateway for the Python SDK equivalent.

​Health

​Models

​Embed

​Options

​Transcribe

​Options

​Command Reference

​Configuration

Health

Models

Embed

Options

Transcribe

Options

Command Reference

Configuration