Skip to main content

​
Getting started

Not to try it. Anonymous calls work on every endpoint, limited per IP to 10/min, 30/hr, and 100/day. A key from app.vlm.run raises that to 240/min and 10,000/hr, tied to your account. Use a key for anything beyond evaluation. See Authentication and Rate Limits.
Yes. Set base_url to https://gateway.vlm.run/v1/openai and pass your VLM Run key as api_key. Chat completions, embeddings, audio transcriptions, and the model list use the OpenAI request shape. Gateway-only fields such as method go in extra_body. See OpenAI SDK Compatibility.
Pick by input and output:The model catalog lists each model’s methods, inputs, and price. Try the cheapest model that fits, and move up only where your own data shows a gain. See Models and Methods.

​
Inputs and outputs

Images, PDFs, video, and audio, as a URL or a base64 data URI. PDFs go in a document_url part, and the Gateway runs each page and returns one <page> per page. Video support depends on the model. A part the model cannot read returns 400 with capability_violation. See Supported Inputs and Document OCR.
On chat and provider models, pass response_format={"type":"json_schema", ...}. Output is constrained to your schema. capabilities.supports_json_schema on GET /v1/openai/models lists the models that accept it. OCR and detection models return 400 for a schema, because their reply is an envelope the Gateway builds.For extraction with schemas you manage, use the VLM Run API: structured responses, custom schemas, and the VLM Run Hub. See Response Formats.
Call a region model such as facebook/sam3.1 or usyd-community/vitpose-plus-large. Each returns one record shape for images and per-frame records for video. See Visual Intelligence.

​
Cost and limits

Rates are public. Read them before you call, then check the charge after.
  • Before: pricing on GET /v1/openai/models, or the model catalog.
  • After: usage.cost on the response, in USD.
  • Unit: most models bill per token. facebook/sam3.1 bills per image or video. Transcription bills per audio second.
  • Floor: every call bills at least $0.001.
See Pricing.
Run 10 to 20 representative inputs and sum usage.cost. Multiply by your volume. Token cost varies with page count and document_dpi, so sample your real documents, not a short test file. Lower image_resolution or document_dpi when the task allows it.
Anonymous: 10/min, 30/hr, 100/day per IP. Authenticated: 240/min and 10,000/hr per user. A 429 carries Retry-After, the seconds to wait. For higher limits, email support@vlm.run. See Rate Limits.
Retry with backoff. A ttft_timeout usually clears once the deployment has scaled. Split large jobs into smaller requests, for example a page range of a long PDF. Keep the client timeout above the model’s total deadline (600 seconds by default), or set stream=true. See Inference timeout (504).

​
Platforms

System One answers typed questions about text, JSON, or images with a calibrated probability per label. It does not generate text. Use it for decisions, such as classify, score, or route, where you need a label and a confidence. Use chat completions when you need free text or extraction. The route is wire-compatible with TypeSafe, so the typesafe-sdk works with a new base_url. See System One and TypeSafe SDK Compatibility.
Two ways:
  • MCP server: https://gateway.vlm.run/mcp exposes read_document, read_image, read_audio, and read_video. It takes an API key or anonymous access. See MCP Server.
  • Skill: npx skills add vlm-run/skills@vlmrun-gw teaches an agent the vlmrun gw commands and request shapes. See Skills.

​
Trust

No. The Gateway does not store your inputs or outputs, and it does not train on them. It keeps request metadata for billing, such as usage and cost. Log x-request-id so support can trace a call.
The Gateway is in beta. Limits and the model catalog can change. Use an API key in production, since anonymous traffic shares a per-IP budget. Every response includes usage.cost and x-request-id, and errors are sanitized. Track changes in the Changelog.

Introduction

OpenAI-compatible API, one base URL, one API key.

Models

Availability, methods, and accepted inputs.

Error Codes

Every error type and its fix.

Quickstart

First requests for VQA and document OCR.