Skip to main content
Send every frame of a video to a model over one WebSocket and read one reply per frame. This guide counts the people in each frame of a 20 s clip with usyd-community/vitpose-plus-large. A session bills every second it is open, at $3 / hour, whatever the frame count.

1. Pick the socket

This guide uses /v1/ws. The models it serves, and their methods, are in Supported models.

2. Cut the video into frames

Each frame is one binary message of raw JPEG, PNG, WebP, or GIF bytes, at most 2 MiB. Cut the video at the rate you want to read, then send the files as they are.
That writes 80 JPEG frames of 640 x 360 pixels, about 30 KB each. The frame count does not change the price. A lower rate leaves more read slots free for the frames you send.

3. Stream the frames

The sample opens one pose session, sends the 80 frames at 4 per second, and keeps the person count of each result. Every frame gets exactly one event back: a result, a frame.dropped, or an error. Frames are numbered from 1, in the order you send them.
Output
The count peaks at 11 people at 12.00 s, inside the bakery. All 80 frames were read, with a median read of 34 ms. The session was open about 20 s, which bills $0.0169 at $3 / hour.

4. Read the result event

content is the method’s output as a JSON string, the same string Chat Completions returns for one image. For pose, it is an img.pose.kpts container with one item per person: a box, 17 keypoints, and a score per keypoint.
Coordinates are fractions of the frame width and height, from 0 to 1, so they do not depend on the frame size you send. usage.cost is null on every frame. The session bills as a whole, and session.stats.cost is its total. Every field of every event is in the WS /v1/ws reference. To read other output, change model and method in session.create:

5. Keep up with the stream

The session drops a frame it cannot read now. It never queues one, because a queued frame answers for a moment the video has moved past. A dropped frame gets a frame.dropped event with a reason, so you can count what you skipped. A slow read shows up as drops, not as a backlog. In one run of this sample, one read took 12.5 s: 12 of the 80 frames were dropped, and the other 68 were read.

6. Close the session

Send session.close when the last reply arrives. Reads in flight finish, then the Gateway sends session.stats and waits up to 5 seconds for you to close the socket. A session left open bills its idle seconds until the 30 second idle close. A session lasts up to 10 minutes and stops at $5 of spend. For a longer video, open a new session per 10 minutes of stream. Sessions per plan are on Rate Limits, and the close codes are on Realtime.

System One Decisions

Ask the same questions of every frame and act on the probability.

Realtime

Session lifecycle, close codes, supported models, and billing.

WS /v1/ws reference

Every client and server event, with its fields.

Response Types

The payload each method returns.