> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vlm.run/llms.txt
> Use this file to discover all available pages before exploring further.

# Webcam from the Browser

> Stream a browser camera to a realtime model through a relay that keeps your API key on the server

Stream a user's camera from a web page to a realtime model, and draw each reply on the video as it arrives. The browser sends frames to a small relay on your server, and the relay holds the API key and the Gateway socket. This guide draws body keypoints with `usyd-community/vitpose-plus-large` at 5 frames per second.

![A street scene with body keypoints drawn as blue dots on a walking man](https://mintcdn.com/autonomiai/PRWL1LD_l1Le04c7/images/gateway-live-webcam.jpg?fit=max\&auto=format\&n=PRWL1LD_l1Le04c7\&q=85\&s=03180970806256b1011d2adeff19d564)

The page below with a recorded street scene as the camera. Each dot is a keypoint from the latest result.

## 1. Why a relay

The browser `WebSocket` API cannot set request headers, so a page cannot send `Authorization` on the upgrade. Putting the key in the page would also give it to every visitor. The relay solves both: the browser connects to your server, and your server opens the Gateway socket with the key.

| Hop | Connection | Authenticates with |
| - | - | - |
| Browser to relay | `ws://your-host/live` | Your own session or cookie |
| Relay to Gateway | `wss://gateway.vlm.run/v1/ws` | `Authorization: Bearer <VLMRUN_API_KEY>` |

The relay forwards every message unchanged, in both directions. The page speaks the [`WS /v1/ws`](https://docs.vlm.run/gateway/api-reference/ws-realtime) protocol, so `session.create`, frames, and events are the same as in [Video over WebSocket](https://docs.vlm.run/gateway/guides/websocket-video.md).

## 2. Run the relay

The relay serves the page at `/` and relays the socket at `/live`. It opens one Gateway session for each browser connection, and closes it when the browser leaves.

```javascript Node.js [expandable] theme={"theme":{"light":"github-light","dark":"dark-plus"}}
import { createReadStream } from "node:fs";
import { createServer } from "node:http";
import WebSocket, { WebSocketServer } from "ws";

const GATEWAY = "wss://gateway.vlm.run/v1/ws";
const PORT = 8080;

const http = createServer((req, res) => {
  res.setHeader("Content-Type", "text/html");
  createReadStream("public/index.html").pipe(res);
});

const relay = new WebSocketServer({ server: http, path: "/live" });

relay.on("connection", (browser) => {
  const upstream = new WebSocket(GATEWAY, {
    headers: { Authorization: `Bearer ${process.env.VLMRUN_API_KEY}` },
  });
  const pending = [];

  browser.on("message", (data, isBinary) => {
    if (upstream.readyState === WebSocket.OPEN) upstream.send(data, { binary: isBinary });
    else pending.push([data, isBinary]);
  });
  upstream.on("open", () => {
    for (const [data, isBinary] of pending.splice(0)) upstream.send(data, { binary: isBinary });
  });
  upstream.on("message", (data) => browser.send(data.toString()));

  upstream.on("close", (code, reason) => browser.close(1000, reason.toString().slice(0, 120)));
  upstream.on("error", () => browser.close(1011));
  browser.on("close", () => upstream.close(1000));
});

http.listen(PORT, () => console.log(`open http://localhost:${PORT}`));
```

```python Python [expandable] theme={"theme":{"light":"github-light","dark":"dark-plus"}}
import asyncio
import os
import pathlib
from http import HTTPStatus

import websockets
from websockets.asyncio.server import serve
from websockets.datastructures import Headers
from websockets.http11 import Response

GATEWAY = "wss://gateway.vlm.run/v1/ws"
PAGE = pathlib.Path("public/index.html").read_bytes()


def page(connection, request):
    if request.path != "/live":
        return Response(HTTPStatus.OK, "OK", Headers({"Content-Type": "text/html"}), PAGE)


async def relay(browser) -> None:
    async with websockets.connect(
        GATEWAY, additional_headers={"Authorization": f"Bearer {os.environ['VLMRUN_API_KEY']}"}
    ) as upstream:

        async def pipe(source, sink) -> None:
            async for message in source:
                await sink.send(message)

        tasks = [asyncio.create_task(pipe(browser, upstream)), asyncio.create_task(pipe(upstream, browser))]
        await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)
        for task in tasks:
            task.cancel()
    print("upstream closed")


async def main() -> None:
    async with serve(relay, "localhost", 8080, process_request=page) as server:
        print("open http://localhost:8080")
        await server.serve_forever()


asyncio.run(main())
```

```bash theme={"theme":{"light":"github-light","dark":"dark-plus"}}
npm install ws && node server.mjs              # Node.js
pip install websockets && python relay.py      # Python, tested with websockets 17
```

Messages the browser sends before the Gateway socket opens wait, then go out in order. When either side closes, the relay closes the other.

> [!WARNING]
>
> A session on `/v1/ws` bills every second it is open, with or without frames. The relay closes the Gateway session when the browser socket closes. Keep that line if you change the relay, or a closed tab bills until the 30 second idle close.

## 3. Build the page

Save the page as `public/index.html`. It asks for the camera, opens the relay socket, and sends one JPEG of the video every 200 ms once `session.created` arrives.

```html public/index.html [expandable] theme={"theme":{"light":"github-light","dark":"dark-plus"}}
<!doctype html>
<html>
  <body>
    <div style="position: relative; width: 640px">
      <video id="camera" autoplay muted playsinline width="640"></video>
      <canvas id="overlay" style="position: absolute; inset: 0"></canvas>
    </div>
    <pre id="status"></pre>
    <script type="module">
      const FPS = 5;
      const video = document.getElementById("camera");
      const overlay = document.getElementById("overlay");
      const status = document.getElementById("status");
      const grab = document.createElement("canvas");

      video.srcObject = await navigator.mediaDevices.getUserMedia({ video: { width: 640 } });
      await video.play();
      overlay.width = grab.width = video.videoWidth;
      overlay.height = grab.height = video.videoHeight;

      const ws = new WebSocket(`ws://${location.host}/live`);
      ws.binaryType = "arraybuffer";
      let timer;

      ws.onopen = () => ws.send(JSON.stringify({
        type: "session.create",
        model: "usyd-community/vitpose-plus-large",
        method: "pose",
        max_inflight: 2,
      }));

      ws.onmessage = ({ data }) => {
        const event = JSON.parse(data);
        if (event.type === "session.created") timer = setInterval(sendFrame, 1000 / FPS);
        if (event.type === "result") drawPose(JSON.parse(event.response.choices[0].message.content));
        if (event.type === "error") status.textContent = `${event.error_type}: ${event.message}`;
        if (event.type === "result" || event.type === "frame.dropped") {
          status.textContent = `frame ${event.frame}: ${event.type}`;
        }
      };
      ws.onclose = () => clearInterval(timer);

      function sendFrame() {
        grab.getContext("2d").drawImage(video, 0, 0);
        grab.toBlob(async (jpeg) => ws.send(await jpeg.arrayBuffer()), "image/jpeg", 0.8);
      }

      function drawPose(pose) {
        const ctx = overlay.getContext("2d");
        ctx.clearRect(0, 0, overlay.width, overlay.height);
        ctx.fillStyle = "#1265FF";
        for (const person of pose.items) {
          person.kpts_xy.forEach(([x, y], i) => {
            if (person.kpts_score[i] < 0.5) return;
            ctx.beginPath();
            ctx.arc(x * overlay.width, y * overlay.height, 4, 0, 2 * Math.PI);
            ctx.fill();
          });
        }
      }

      addEventListener("pagehide", () => ws.close());
    </script>
  </body>
</html>
```

Open `http://localhost:8080` and allow the camera. The status line shows the latest frame number and its event, and the overlay redraws on every `result`. Keypoints are fractions of the frame from 0 to 1, so the page multiplies them by the canvas size. Keypoints with a score below 0.5 are not drawn.

`getUserMedia` runs only on a secure origin: `https` in production, or `localhost` while you develop. In production, serve the page over `https` and connect with `wss://`.

## 4. Keep the stream live

The page sends on a timer and does not wait for replies. A frame that arrives while `max_inflight` reads are outstanding is dropped with `reason: "busy"`, so a slow read costs you one stale frame, never a growing delay. Each event names its `frame`, so you can always tell how old the drawing is.

| Setting | This page | Raise it when | Lower it when |
| - | - | - | - |
| `FPS` | 5 | Motion between frames is too large to follow | You see many `busy` drops |
| `max_inflight` | 2, up to 8 | Reads take longer than `1000 / FPS` ms | You want fewer reads in flight per user |
| JPEG quality | 0.8 | The model misses small detail | Frames approach the 2 MiB cap |

In a test with a recorded camera feed, the page sent 5 frames per second and drew a `result` for each one.

## 5. Before you ship

* **Authenticate your users at the relay.** The relay spends your API key. Check the user's session on the `/live` upgrade, and refuse it when the check fails.
* **Cap sessions per user.** Each browser connection is one Gateway session. Your plan allows 1 at once on Free and up to 5 on paid plans. See [Rate Limits](https://docs.vlm.run/gateway/rate-limits.md#websocket-sessions).
* **Reconnect on `1013`.** A full plan or a starting model closes with `1013`. Wait `retry_after` when the `error` event has one, then open a new socket. See [Close codes](https://docs.vlm.run/gateway/realtime.md#close-codes).
* **Reopen at 10 minutes.** A session lasts up to 10 minutes. Open a new one when it closes with the reason `expired`.

## Related

> [card] [Video over WebSocket](https://docs.vlm.run/gateway/guides/websocket-video.md)
>
> The same socket from a script, with a recorded video.

> [card] [Masks per Frame](https://docs.vlm.run/gateway/guides/live-masks.md)
>
> Segment every frame with `facebook/sam3.1`.

> [card] [Hand Tracking](https://docs.vlm.run/gateway/guides/live-hands.md)
>
> 2D and 3D hand joints per frame with `geopavlakos/hamer`.

> [card] [Realtime](https://docs.vlm.run/gateway/realtime.md)
>
> Session lifecycle, close codes, and billing.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.