Realtime

A realtime session reads a stream of frames over one WebSocket. You open the socket with your API key, describe the read once in session.create, then send each frame as raw image bytes. Each frame gets one reply event, and a frame the session cannot read now is dropped, never queued.

​

When to use a session

Use a session when you read many frames from one source with the same model and method: a camera, a screen, or a decoded video. Use a POST for single images, documents, and anything you read once. We drop instead of queue because a queued frame answers for a moment you have already moved past. A drop tells you to send fewer frames, or to raise max_inflight on the session.

​

Session lifecycle

Both sockets share this lifecycle. Only the session.create fields and the reply event differ.

1. Upgrade. Open the socket with Authorization: Bearer <VLMRUN_API_KEY> on the upgrade request. A request with no key, or with Bearer vlmrun, is refused with 401 before the socket opens.
2. session.create. Send it as the first JSON text message. The Gateway checks your plan, model access, credit, and capacity once, then replies session.created with the session’s bounds. A refusal is an error event, then a close.
3. Frames. Send each frame as one binary message of raw JPEG, PNG, WebP, or GIF bytes. Each frame gets one event back: its reply, frame.dropped, or an error that names the frame.
4. session.close. Reads in flight finish and return. The server sends session.stats with the session’s counts and cost, then closes with 1000.

The standard WebSocket API cannot set request headers, so a browser cannot send Authorization. Open sessions from a server. The Node.js samples use the ws package, which takes headers. The Python samples pass the header to websockets.connect as additional_headers.

​

Close codes

The server explains a refusal or a cut with an error event before it closes. The error_type values are listed on each socket’s reference page. A replica restart can also cut the connection with no close frame. Reconnect and open a new session. A session does not resume: frames, counts, and spend start again from zero.

​

Supported models

A model must opt in to realtime. On /v1/ws, a model without realtime support is refused at session.create with invalid_request_error and close 1008. max_fps is the read rate the model allows, on average. A frame that arrives sooner than that after the last read is dropped with reason: "rate". A short burst bank absorbs network jitter.

​

Billing

On /v1/ws, idle time bills like streaming time, so close the session when you stop sending frames. On /v1/ws/systemone, dropped frames and failed reads bill nothing. On /v1/ws, a session open for 10 minutes bills 600 seconds at $3 / hour, which is $0.50, on either model. The session total is in session.stats.cost, and its id reads back with GET /v1/completions/{id}.

​

Limits

Sessions are limited per organization plan, not per request. A session does not spend the per-minute request budget.

• Sessions at once: 1 on Free, up to 5 on paid plans. Then rate_limit_error and close 1013.
• Per session: 10 minutes, 30 seconds idle, $5 of spend, and 2 MiB per frame.
• Capacity: the Gateway and some models cap open sessions. A full cap is capacity_error with close 1013.

The full tables are in Rate Limits.

​

Next steps