client.gateway - VLM Run

The client.gateway object provides access to the VLM Run Gateway, an OpenAI-compatible surface for third-party OCR and vision-language models (e.g. zai-org/glm-ocr, paddleocr/pp-ocrv6, qwen/qwen3.5-0.8b). It authenticates with the same VLMRUN_API_KEY used everywhere else in the SDK. Under the hood the gateway points the OpenAI SDK at https://gateway.vlm.run/v1/openai, so client.gateway.completions, client.gateway.embeddings, and client.gateway.transcriptions are the standard OpenAI resource objects. You get the familiar chat-completions, embeddings, and audio-transcription interfaces without configuring a base URL yourself.

The gateway is a raw passthrough to the underlying models. Unlike client.agent (which calls the Orion agent), most gateway models, especially OCR models, do not accept text-only input, so you generally need at least one image_url/document_url content part.

Installation

The gateway uses the OpenAI SDK, available via the openai extra:

pip install "vlmrun[openai]"

Chat Completions

Run an OCR / VLM model over a document or image. Pass PDFs as document_url content parts and images as image_url; gateway-specific fields (such as method and document_dpi) travel in extra_body.

from vlmrun.client import VLMRun

client = VLMRun(api_key="<VLMRUN_API_KEY>")

response = client.gateway.completions.create(
    model="paddleocr/pp-ocrv6",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "document_url",
                    "document_url": {
                        "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf"
                    },
                }
            ],
        }
    ],
    extra_body={"method": "ocr", "document_dpi": 72},
)

print(response.choices[0].message.content)

Visual Q&A

Models that accept text input (e.g. qwen/qwen3.5-0.8b) can take a text content part alongside an image:

response = client.gateway.completions.create(
    model="qwen/qwen3.5-0.8b",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What is happening in this image?"},
                {"type": "image_url", "image_url": {"url": "https://example.com/image.jpg"}},
            ],
        }
    ],
)

print(response.choices[0].message.content)

Streaming

The gateway streams document (PDF) requests one SSE chunk per page:

stream = client.gateway.completions.create(
    model="zai-org/glm-ocr",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "document_url", "document_url": {"url": "https://example.com/doc.pdf"}},
            ],
        }
    ],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)

An async client is available via client.gateway.async_completions.

Embeddings

Embed text, images, or video with a gateway embedding model (e.g. qwen/qwen3-vl-embedding-2b).

Multimodal input nests content parts one level deeper than plain text: input is a list whose items are either a string or a list of content parts.

response = client.gateway.embeddings.create(
    model="qwen/qwen3-vl-embedding-2b",
    input=[[{"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}}]],
)

print(len(response.data[0].embedding))

Transcriptions

Transcribe audio (or a video’s audio track) with a gateway transcription model (e.g. nvidia/parakeet-tdt-0.6b-v3):

with open("clip.mp3", "rb") as fh:
    response = client.gateway.transcriptions.create(
        model="nvidia/parakeet-tdt-0.6b-v3",
        file=fh,
    )

print(response.text)

Listing Models

List the models available on the gateway. Each model carries extra metadata (input/output pricing, modality support, supported methods) beyond the standard OpenAI fields, preserved on the object’s model_extra.

for model in client.gateway.models():
    print(model.id)

Health Check

Check whether the gateway is reachable and authenticated:

if client.gateway.health():
    print("Gateway is healthy")

Configuration

By default the gateway targets https://gateway.vlm.run/v1. Override it with the VLMRUN_GATEWAY_URL environment variable:

export VLMRUN_GATEWAY_URL="https://gateway.vlm.run/v1"

SDK Reference

Attribute / Method	Description
`client.gateway.completions`	OpenAI-compatible chat completions (synchronous)
`client.gateway.async_completions`	OpenAI-compatible chat completions (asynchronous)
`client.gateway.embeddings`	OpenAI-compatible embeddings interface
`client.gateway.transcriptions`	OpenAI-compatible audio transcriptions interface
`client.gateway.models()`	List models available on the gateway
`client.gateway.health()`	Return `True` if the gateway is reachable and authenticated
`client.gateway.base_url`	Gateway base URL (without trailing slash)

See the Gateway documentation for the full model catalog, methods, and multimodal input reference, and the vlmrun gw CLI for the terminal equivalent.

​Installation

​Chat Completions

​Visual Q&A

​Streaming

​Embeddings

​Transcriptions

​Listing Models

​Health Check

​Configuration

​SDK Reference

Installation

Chat Completions

Visual Q&A

Streaming

Embeddings

Transcriptions

Listing Models

Health Check

Configuration

SDK Reference