Realtime System One

Answers one set of System One questions for every frame you send. Each decision carries the same answers and usage as POST /typesafe/v1/systemone. For samples, see Realtime System One. For the lifecycle, close codes, and models, see Realtime. The socket is not in the OpenAPI schema, because the schema does not describe WebSocket routes.

Connection


URL	`wss://gateway.vlm.run/v1/ws/systemone`
Deprecated alias	`wss://gateway.vlm.run/typesafe/ws`, with the same protocol and bounds
Auth	`Authorization: Bearer <VLMRUN_API_KEY>` on the upgrade request
Anonymous	No key, or `Bearer vlmrun`: `401` at the upgrade
Events	JSON text messages, each with a `type`
Frames	Binary messages
First event	`session.create`, within 30 seconds of the upgrade

Client events

session.create

Opens the session and resolves the read plan. Every schema error surfaces here. An unknown field is refused.

Field	Type	Required	Description
`type`	`"session.create"`	Yes
`model`	`string`	Yes	A System One model id or alias. See Supported models.
`questions`	`object`	Yes	A map of question id to question, at least one.
`state`	`string` \| `object` \| `array`	No	What the questions are about. Default: empty.
`detail`	`"auto"` \| `"low"` \| `"high"`	No	Vision tokens per frame, as on the HTTP route. Default `auto`.
`steps`	`integer`	No	Diffusion engines only: denoise steps per read, 1 to 8. Default `1`.
`samples`	`integer`	No	Noise draws to average, 1 to 32. Every draw is billed.
`reasoning_effort`	`string`	No	Generative engines only: `none` (default), `minimal`, `low`, `medium`, or `high`.
`window`	`integer`	No	Frames per read, the newest ones, 1 to 8. Default `1`.
`stride`	`integer`	No	Frames between reads. The frames between are context for the next read. Default `1`.
`max_inflight`	`integer`	No	Reads outstanding at once, 1 to 8. Default `1`.

Frame (binary message)

One frame: raw JPEG, PNG, WebP, or GIF bytes, with no base64 and no JSON wrapper, at most max_frame_bytes. A read fires every stride frames, over the newest window frames. A frame that fires a read gets one decision, frame.dropped, or error back.

state

Field	Type	Description
`type`	`"state"`
`state`	`string` \| `object` \| `array`	The state later frames read against.

Re-pins the state. It is not a read and gets no reply.

decide

Field	Type	Description
`type`	`"decide"`
`id`	`string`	Optional. Echoed on the `decision`.
`state`	`string` \| `object` \| `array`	Optional. The state for this read only. Default: the pinned state.
`content`	`array` \| `string`	Optional. This read’s media as content parts, validated as on the HTTP route.

One read with its own body, against the session’s questions. A decide is never dropped.

session.close

{ "type": "session.close" }

Reads in flight finish and return their events. The server then sends session.stats and closes with 1000.

Server events

session.created

Field	Type	Description
`session_id`	`string`	The session’s id.
`model`	`string`	The canonical model id.
`questions`	`array`	The question ids, in order.
`groups`	`integer`	Question groups in the read plan. Groups are read concurrently.
`canvas_widths`	`array`	The answer canvas width of each group.
`window`	`integer`	Frames per read.
`stride`	`integer`	Frames between reads.
`max_inflight`	`integer`	Reads outstanding at once.
`max_frame_bytes`	`integer`	The largest frame, in bytes: `2097152` (2 MiB).
`max_cost`	`number`	The session’s spend ceiling, in USD.
`expires_in`	`number`	Seconds the session may last, from the upgrade.

decision

Field	Type	Description
`frame`	`integer`	The frame that fired the read.
`answers`	`object`	Answers keyed by question id, as on the HTTP route.
`usage`	`object`	Tokens, reads, and `cost` for this decision, as on the HTTP route. This is what the decision bills.
`latency_ms`	`number`	Time the read took, in milliseconds.
`id`	`string` \| `null`	The `id` of the `decide` event that asked for it, else `null`.

frame.dropped

Field	Type	Description
`frame`	`integer`	The frame that was not read.
`inflight`	`integer`	Reads outstanding when it arrived.
`reason`	`"busy"`	Every read slot was taken.

error

Field	Type	Description
`error_type`	`string`	One of the error types.
`message`	`string`	What went wrong.
`frame`	`integer` \| `null`	The frame the error is for. `null` for an event or the session.
`retry_after`	`integer` \| `null`	Seconds to wait before you retry, when set.

session.stats

Sent after your session.close.

Field	Type	Description
`session_id`	`string`	The session’s id.
`frames`	`integer`	Frames received.
`decisions`	`integer`	Decisions returned.
`dropped`	`integer`	Frames dropped.
`errors`	`integer`	Frames and events that returned an `error`.
`reads`	`integer`	Reads billed, over every decision.
`input_tokens`	`integer`	Input tokens, over every decision.
`cached_tokens`	`integer`	Input tokens served from the prefix cache.
`cost`	`number`	The sum of every decision’s `usage.cost`, in USD.
`p50_ms`	`number`	Median read latency, in milliseconds.
`p95_ms`	`number`	95th percentile read latency, in milliseconds.
`fps`	`number`	Frames received per second, over the session.

Errors

`error_type`	When	Close
`invalid_request_error`	A bad event, question set, or frame, or a model this route does not serve.	`1008` when `session.create` is bad. None for a later event or frame.
`permission_error`	The key has no access to the model.	`1008`
`billing_error`	The organization is out of credit.	`1008`
`rate_limit_error`	Your plan’s sessions are all open.	`1013`
`rate_limit_error`	Your plan’s daily sessions are spent. Has `retry_after`.	`1008`
`rate_limit_error`	The session reached `max_cost`.	`1000`, reason `spent`
`capacity_error`	A session cap is full.	`1013`
`model_provisioning_error`	The model has no running replica. The Gateway starts one. Has `retry_after`.	`1013`
`overloaded_error`	The engine was busy for one read. Has `retry_after`.	None
`api_error`	One read failed. Has `retry_after`.	None

Reads in flight when the spend ceiling trips are still answered. The Gateway also closes with 1000 and reason idle after 30 seconds with no frame and no read outstanding, and with reason expired at expires_in. What to do after each close code is in Close codes. Session limits are in Rate Limits.

OpenAI Compatible API

TypeSafe Compatible API

Realtime API

Realtime System One

Connection

Client events

session.create

Frame (binary message)

state

decide

session.close

Server events

session.created

decision

frame.dropped

error

session.stats

Errors

​Connection

​Client events

​session.create

​Frame (binary message)

​state

​decide

​session.close

​Server events

​session.created

​decision

​frame.dropped

​error

​session.stats

​Errors

Connection

Client events

session.create

Frame (binary message)

state

decide

session.close

Server events

session.created

decision

frame.dropped

error

session.stats

Errors