Skip to main content
Every public OpenAI route shares one per-caller budget: chat completions, audio transcriptions, embeddings, the model catalog routes, and GET /v1/completions/{completion_id}. A catalog listing and an inference call draw on the same counter. When a bucket fills, the API returns 429 Too Many Requests. See Authentication for how tiers are determined.

​
Limits by Tier

The first-party bucket is for VLM Run tooling, such as vlmrun-cli/1.2. It needs both signals. A vlmrun- prefixed User-Agent without the vlmrun bearer token stays on the anonymous bucket. The windows stack: an anonymous request has to fit all three of 10/min, 30/hr, and 100/day, and the first one to fill returns the 429. Attribution differs too: anonymous requests are capped per client IP, so callers behind one NAT or proxy share a bucket, while an API key is capped per user account. See Authentication for how the Gateway resolves tiers from the Authorization header.
Sign up at app.vlm.run for a free API key. An API key lifts you from 10 requests a minute to 240, attributes usage to your account, and stops you sharing a bucket with other callers on the same IP.

​
Response headers

Anonymous responses carry advisory quota headers, so a client can pace itself before it is refused: Retry-After reports the longest exhausted window, not the one that tripped, so a client that waits that long does not immediately hit a second 429. A 429 body names the tier that was limited:

​
Quota is not queue backpressure

A 429 also arrives when the model’s request queue is full, which is capacity, not quota: the queue holds 128 waiting requests, and past that the Gateway refuses the call. Retry it. A model whose deployment is cold or not up answers 503 with code: "model_unavailable" and Retry-After: 30, and a call that runs past the inference timeout (270 seconds by default) answers 504. See Error Codes.

​
Tips

  • Back off and retry after Retry-After seconds when you receive a 429.
  • Prefer URL-based document inputs over large base64 payloads to keep request latency predictable under load.
  • For production document workloads, use an authenticated API key so limits apply to your account rather than a shared IP bucket.
  • Read a long PDF in page ranges rather than in one call, so a single request does not sit in the queue for minutes.
Counters live in the serving process unless the deployment sets VLMRT_RATE_LIMIT_STORAGE_URI to shared storage. Without shared storage, a caller’s effective limit multiplies by the number of replicas, so treat the published numbers as the guaranteed floor.
If you need higher limits, contact support@vlm.run.