gateway

vlmrun gateway (alias vlmrun gw) runs OpenAI-compatible OCR and vision-language models. It calls the VLM Run Gateway at https://gateway.vlm.run/v1. It uses the same VLMRUN_API_KEY as the rest of the CLI. Unlike vlmrun chat, requests are a raw passthrough. No Files API upload. No Orion agent. Every input is a local file path or an http(s) URL. URLs pass by reference. Local files are inlined as base64 data: URLs. The content part follows the media type:

Media	Content part	Matches
Document	`document_url`	`.pdf`, `.doc`, `.docx`
Image	`image_url`	any image type: `.jpg`, `.png`, `.webp`, …
Video	`video_url`	any video type: `.mp4`, `.mov`, `.mkv`, …
Other	`file_url`	everything else

gw chat needs a file, a URL, or -p/--prompt. OCR models such as zai-org/glm-ocr and paddleocr/pp-ocrv6 reject text-only input. Use -p alone only with chat models like qwen/qwen3.5-0.8b.

Model ids are the full <org>/<name> from vlmrun gw models. Short aliases work. pp-ocrv6 maps to paddleocr/pp-ocrv6.

Health

Exits non-zero when the check fails. The reply includes version and commit. See Health.

Models

# List every model with its methods
vlmrun gw models

# Methods, params, and example commands for one model
vlmrun gw models pp-ocrv6

# Raw JSON catalog
vlmrun gw models --json

The default method is marked *. --json returns the raw catalog records. A detail view adds a commands array. See Gateway Models for the catalog.

paddleocr/pp-ocrv6

task      chat
methods   ocr*, detect, text
inputs    image_url, document_url, file_url, file
aliases   pp-ocrv6
note      document_url PDF (plain text per page)
examples
├── vlmrun gw chat doc.pdf -m paddleocr/pp-ocrv6 --method ocr
├── vlmrun gw chat doc.pdf -m paddleocr/pp-ocrv6 --method text
└── vlmrun gw chat doc.pdf -m paddleocr/pp-ocrv6 --method ocr --method-params '{"lang": "en", "score_threshold": 0.5}'

Embed

Each file and each -t / --text becomes its own vector. --join combines inputs into one vector. At most one file. File inputs must be images or video. Use -t for text.

# Embed text
vlmrun gw embed -t "a blue parrot" -m qwen/qwen3-vl-embedding-2b

# Embed an image
vlmrun gw embed photo.jpg -m qwen/qwen3-vl-embedding-2b

# Embed an image and caption jointly
vlmrun gw embed photo.jpg -t "caption" --join -m qwen/qwen3-vl-embedding-2b

# Truncate vectors to a fixed number of dimensions
vlmrun gw embed -t "hi" -m qwen/qwen3-vl-embedding-2b --dimensions 64

# Output full vectors as JSON
vlmrun gw embed photo.jpg -m qwen/qwen3-vl-embedding-2b --json

Without --json, the CLI prints one row per input. Each row shows the vector’s dimensionality and first four components. The footer is model · T:<total> tokens · <latency>s. qwen/qwen3-vl-embedding-2b returns 2048 dimensions by default. --dimensions truncates that vector.

Options

Option	Short	Description
`FILES...`		Image/video file(s) to embed. Repeatable
`--model`	`-m`	Embedding model id (required)
`--text`	`-t`	Text to embed (repeatable)
`--join`		Embed all inputs together as one vector (max one file)
`--dimensions`	`-d`	Truncate vectors to this many dimensions
`--json`	`-j`	Output raw JSON, including full vectors
`--timeout`		Request timeout in seconds

Transcribe

# Transcribe a local file
vlmrun gw transcribe clip.mp3 -m nvidia/parakeet-tdt-0.6b-v3

# Transcribe a video's audio track
vlmrun gw transcribe clip.mp4 -m nvidia/parakeet-tdt-0.6b-v3

# Choose a response format (json, text, verbose_json, srt, vtt)
vlmrun gw transcribe clip.mp3 -m nvidia/parakeet-tdt-0.6b-v3 -f srt

# Provide a language hint
vlmrun gw transcribe clip.mp3 -m nvidia/parakeet-tdt-0.6b-v3 --language en

# Bias the transcript toward known proper nouns
vlmrun gw transcribe clip.mp3 -m nvidia/parakeet-tdt-0.6b-v3 -p "VLM Run, Orion"

# Transcribe a hosted URL instead of a local file
vlmrun gw transcribe --url https://example.com/a.mp3 -m nvidia/parakeet-tdt-0.6b-v3

Pass a file or --url. Not both. The transcript prints in a Transcript panel. The footer is model · <format> · <latency>s.

Options

Option	Short	Description
`FILE`		Audio file, or a video whose audio track is transcribed
`--model`	`-m`	Transcription model id (required)
`--url`		Hosted audio URL instead of a local file
`--format`	`-f`	Response format: `json`, `text`, `verbose_json`, `srt`, `vtt` (default: `json`)
`--language`	`-l`	ISO-639-1 language hint, e.g. `en`
`--prompt`	`-p`	Context to bias transcription (proper nouns)
`--json`	`-j`	Output raw JSON
`--timeout`		Request timeout in seconds

Command Reference

Command	Description
`vlmrun gw health`	Check gateway reachability (exits non-zero when unreachable)
`vlmrun gw models [MODEL]`	List models, or detail one model (`-j, --json` for raw output)
`vlmrun gw chat INPUTS... -m MODEL`	Run a model over one or more files or URLs
`vlmrun gw embed [FILES...] -m MODEL`	Embed text, images, or video
`vlmrun gw transcribe [FILE] -m MODEL`	Transcribe audio (or `--url`)
`vlmrun gw systemone [INPUTS...]`	Typed, calibrated decisions (alias `vlmrun gw s1`)

Configuration

Variable	Purpose	Default
`VLMRUN_API_KEY`	Gateway authentication, shared with the rest of the CLI	none
`VLMRUN_GATEWAY_URL`	Gateway base URL	`https://gateway.vlm.run/v1`

export VLMRUN_API_KEY="your-api-key"
export VLMRUN_GATEWAY_URL="https://gateway.vlm.run/v1"

The OpenAI-compatible surface is the base URL plus /openai. Pass that URL as base_url to an OpenAI SDK client. Gateway requests default to a 600 second timeout. The CLI default is 120 seconds. --timeout overrides the gateway timeout. See the Gateway documentation for the model catalog, methods for the methods reference, and client.gateway for the Python SDK.

Overview

SDKs

CLI

CLI - Gateway

CLI - Orion

Health

Models

Embed

Options

Transcribe

Options

Command Reference

Configuration

​Health

​Models

​Embed

​Options

​Transcribe

​Options

​Command Reference

​Configuration

Health

Models

Embed

Options

Transcribe

Options

Command Reference

Configuration