Skip to main content
Public routes share one budget per user. That covers chat completions, audio transcriptions, embeddings, the model catalog, and GET /v1/completions/{completion_id}. A catalog listing and an inference call draw from the same counter. When the bucket is full, the Gateway returns 429 Too Many Requests.

​
Limits by Tier

Sign up at app.vlm.run for a free API key to get higher rate limits.
Those windows apply together.
  • Anonymous: a request has to fit 10/min, 30/hr, and 100/day. The first window to fill returns 429.
  • Per client IP: anonymous requests are capped per client IP, so callers behind one NAT or proxy share a bucket.
  • Per user: an API key is capped per user account.
Authentication describes how the Gateway resolves tiers from the Authorization header.

​
Rate limit exceptions

A request over its limit returns 429 Too Many Requests with Retry-After, the seconds to wait. Waiting that long avoids an immediate second 429. The body names the limited tier:

​
Tips

  • Back off and retry after Retry-After seconds on a 429.
  • Prefer URL document inputs over large base64 payloads.
  • Read a long PDF in page ranges to keep each request small.
Counters live in the serving process unless the deployment sets VLMRT_RATE_LIMIT_STORAGE_URI to shared storage. Without shared storage, a caller’s effective limit multiplies by the number of replicas. Treat the published numbers as the guaranteed floor.
Contact support@vlm.run for a higher limit.