> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vlm.run/llms.txt
> Use this file to discover all available pages before exploring further.

# MCP Server

> Connect any MCP-aware agent to the VLM Run Gateway over Streamable HTTP

The VLM Run Gateway exposes a **Model Context Protocol (MCP)** server so any
MCP-aware agent framework (Pydantic AI, LangChain, Mastra, OpenAI Agents SDK,
Claude Code) can read documents, audio, and video through the same gateway
pipeline as the REST API. It is a [FastMCP](https://gofastmcp.com) server served
over **Streamable HTTP**, and it re-enters the same ingress as
[`chat/completions`](/gateway/api-reference/post-chat-completions), so
authentication, billing, and metering behave identically.

| Property  | Value                                    |
| --------- | ---------------------------------------- |
| Endpoint  | `POST https://gateway.vlm.run/mcp`       |
| Transport | Streamable HTTP (FastMCP)                |
| Auth      | `Authorization: Bearer <VLMRUN_API_KEY>` |

Authentication follows the same tiers as the REST API. See
[Authentication](/gateway/authentication) and [Rate Limits](/gateway/rate-limits).

## Tools

The server exposes one read tool per modality plus a model-discovery tool. Every
`read_*` tool takes a `url` and returns the extracted text.

| Tool            | Signature                                                                                          | Returns                                                  |
| --------------- | -------------------------------------------------------------------------------------------------- | -------------------------------------------------------- |
| `read_document` | `read_document(url, model?, dpi?, pages?, response_format?)`                                       | Markdown OCR                                             |
| `read_audio`    | `read_audio(url, model?, language?, response_format?)`                                             | Transcript                                               |
| `read_video`    | `read_video(url, model?, prompt?, fps?, max_frames?, encoder?, encoder_params?, response_format?)` | Text description / transcript                            |
| `list_models`   | `list_models(modality?)`                                                                           | Models the gateway serves, and which read tool each fits |

### `url`

`url` is always an `http(s)` URL, a `data:` URI, or a bare base64 string. It is
never a local filesystem path. Host the file or inline it as base64 before
calling a read tool.

### `response_format`

`response_format` selects how the result is serialized:

* The **native form** (`markdown` for documents, `text` for audio and video)
  returns a plain string.
* `json` returns the structured envelope.

### Return shape (`ReadResult`)

Each `read_*` tool returns the clean text as the tool's content **and** a typed
structured payload, so an agent gets usable text directly while still being able
to read which model actually ran and what it cost:

| Field   | Description                                               |
| ------- | --------------------------------------------------------- |
| `text`  | The extracted text (markdown / transcript / description). |
| `data`  | The structured envelope (present for `json` output).      |
| `model` | The model the gateway actually ran.                       |
| `cost`  | The metered cost of the call.                             |

### Errors

A rejected or unsupported model returns a clean, actionable message plus a next
step (for example, "call the `list_models` tool") rather than a raw error
envelope, so the agent can recover on its own.

## Page selection with `pages`

`read_document` accepts a `pages` argument to read only specific PDF pages
instead of the whole document, saving cost and latency.

* Pages are **0-indexed**. Pass a list whose items are single page indices and/or
  `[start, stop]` **half-open** ranges (numpy-style: `start` inclusive, `stop`
  exclusive). **Negative indices** count from the end.
* Only the selected pages are rasterized and read. They are **re-numbered from 0**
  in the response.

| `pages`            | Reads               |
| ------------------ | ------------------- |
| `[0, 2, 4]`        | pages 0, 2, 4       |
| `[[0, 3]]`         | pages 0, 1, 2       |
| `[[0, 3], [5, 7]]` | pages 0, 1, 2, 5, 6 |
| `[-1]`             | the last page       |

<Note>
  Page selection is specific to the `read_document` MCP tool. The REST
  [`chat/completions`](/gateway/api-reference/post-chat-completions) API reads
  every page (up to `document_max_pages`).
</Note>

## Video encoding with `encoder`

`read_video` encodes a video into image(s) before dispatch so image-only models
can read it. Control this with `encoder` and `encoder_params`:

* `encoder` defaults to **`mosaic`** (sampled keyframes tiled into a single grid
  image). `frames` and `keyframes` are also available.
* `encoder_params` tunes the encoder (for example `tile_cols`, `tile_rows`,
  `num_frames`).

On the REST API these map to
[`video_encoder` and `video_encoder_params`](/gateway/api-reference/post-chat-completions#video-encoding).

## Connect an agent

Every MCP-aware client connects the same way: point it at
`https://gateway.vlm.run/mcp` with a bearer token, and the four tools above show
up alongside the agent's other tools. Set your key once:

```bash theme={"theme":{"light":"github-light","dark":"dark-plus"}}
export VLMRUN_API_KEY="your-api-key"
```

<Tabs>
  <Tab title="Pydantic AI">
    ```python theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    from pydantic_ai import Agent
    from pydantic_ai.capabilities import MCP

    agent = Agent(
        "anthropic:claude-sonnet-5",
        capabilities=[
            MCP(
                "https://gateway.vlm.run/mcp",
                headers={"Authorization": f"Bearer {KEY}"},
            )
        ],
    )
    ```

    `native=False` (the default) runs the MCP client locally: Pydantic AI
    connects to the gateway and calls the tools. `native=True` advertises the
    URL to the model provider (OpenAI Responses, Anthropic, xAI) so the provider
    connects to the server directly.
  </Tab>

  <Tab title="LangChain">
    ```python theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    from langchain_mcp_adapters.client import MultiServerMCPClient
    from langgraph.prebuilt import create_react_agent

    client = MultiServerMCPClient(
        {
            "vlmrun": {
                "url": "https://gateway.vlm.run/mcp",
                "transport": "streamable_http",
                "headers": {"Authorization": f"Bearer {KEY}"},
            }
        }
    )

    tools = await client.get_tools()
    agent = create_react_agent("anthropic:claude-sonnet-5", tools)
    ```

    Install the adapter with `pip install langchain-mcp-adapters langgraph`.
  </Tab>

  <Tab title="Mastra">
    ```typescript theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    import { MCPClient } from "@mastra/mcp";

    const mcp = new MCPClient({
      servers: {
        vlmrun: {
          url: new URL("https://gateway.vlm.run/mcp"),
          requestInit: {
            headers: { Authorization: `Bearer ${process.env.VLMRUN_API_KEY}` },
          },
        },
      },
    });

    const tools = await mcp.getTools();
    ```

    Pass `tools` to any Mastra `Agent`. Install with `npm install @mastra/mcp`.
  </Tab>

  <Tab title="OpenAI Agents SDK">
    ```python theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    from agents import Agent, Runner
    from agents.mcp import MCPServerStreamableHttp

    async with MCPServerStreamableHttp(
        params={
            "url": "https://gateway.vlm.run/mcp",
            "headers": {"Authorization": f"Bearer {KEY}"},
        }
    ) as server:
        agent = Agent(
            name="Assistant",
            mcp_servers=[server],
        )
        result = await Runner.run(agent, "Summarize the attached PDF.")
        print(result.final_output)
    ```
  </Tab>

  <Tab title="Claude Code">
    ```bash theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    claude mcp add --transport http vlmrun https://gateway.vlm.run/mcp \
      --header "Authorization: Bearer $VLMRUN_API_KEY"
    ```

    The `read_document`, `read_audio`, `read_video`, and `list_models` tools are
    now available in your Claude Code session.
  </Tab>
</Tabs>

## End-to-end example

Once connected, the agent calls the read tools on its own. Ask a question about a
document and it will fetch, OCR, and answer, all through the gateway:

```python theme={"theme":{"light":"github-light","dark":"dark-plus"}}
from pydantic_ai import Agent
from pydantic_ai.capabilities import MCP

agent = Agent(
    "anthropic:claude-sonnet-5",
    capabilities=[
        MCP(
            "https://gateway.vlm.run/mcp",
            headers={"Authorization": f"Bearer {KEY}"},
        )
    ],
)

result = agent.run_sync(
    "Read https://example.com/invoice.pdf and tell me the total amount due."
)
print(result.output)
```

The agent picks `read_document`, receives the markdown OCR plus the `ReadResult`
payload (including the model that ran and its cost), and reasons over the text to
answer, no extraction schema or glue code required.

## Explore the server

Point the [MCP Inspector](https://github.com/modelcontextprotocol/inspector) at
the gateway to list tools and call them interactively before wiring up an agent:

```bash theme={"theme":{"light":"github-light","dark":"dark-plus"}}
npx @modelcontextprotocol/inspector
```

Set the transport to **Streamable HTTP**, the URL to
`https://gateway.vlm.run/mcp`, and add an `Authorization: Bearer <VLMRUN_API_KEY>`
header. Start with `list_models` to see what the gateway currently serves and
which read tool each model fits.

## Related

<CardGroup cols={2}>
  <Card title="Chat Completions API" icon="comments" href="/gateway/api-reference/post-chat-completions">
    The REST surface behind the same gateway pipeline.
  </Card>

  <Card title="List models" icon="table-list" href="/gateway/api-reference/get-models">
    Query the live model catalog and capabilities.
  </Card>

  <Card title="Authentication" icon="key" href="/gateway/authentication">
    Bearer tokens, tiers, and anonymous access.
  </Card>

  <Card title="Multimodal Inputs" icon="images" href="/gateway/multimodal-inputs">
    Content part types, document limits, and video knobs.
  </Card>
</CardGroup>