MCP Quickstart - VLM Run

Connect your AI agent to VLM Run’s remote MCP server and start building agentic workflows with visual AI in minutes.

Installation

Get your API key

Head over to the VLM Run Dashboard to get your API key ($VLMRUN_API_KEY). We’ll use this to authenticate your requests to the MCP server next.

Ping the MCP server to test

Copy-paste the server URL linked below in your browser and you should see a ping response from the MCP server.

https://mcp.vlm.run/${VLMRUN_API_KEY}/sse

Authentication using the above approach is purely experimental, and is subject to change. We’ll be announcing our OAuth 2.1-based authentication, as per the MCP spec, soon.

Add the server to your MCP client

Add this configuration to your Claude Desktop MCP settings:

Location (Mac) ~/Library/Application Support/Claude/claude_desktop_config.json
Location (Windows) %APPDATA%/Claude/claude_desktop_config.json See docs for more details.

{
  "mcpServers": {
    "vlm-run-mcp": {
      "args": [
        "mcp-remote",
        "https://mcp.vlm.run/${VLMRUN_API_KEY}/sse",
        "--transport",
        "sse-only",
      ],
      "env": {
        "VLMRUN_API_KEY": "${VLMRUN_API_KEY}"
      }
    }
  }
}

Restart Claude Desktop after adding the configuration.

Add this configuration to your Claude Desktop MCP settings:

Location (Mac) ~/Library/Application Support/Claude/claude_desktop_config.json
Location (Windows) %APPDATA%/Claude/claude_desktop_config.json See docs for more details.

{
  "mcpServers": {
    "vlm-run-mcp": {
      "args": [
        "mcp-remote",
        "https://mcp.vlm.run/${VLMRUN_API_KEY}/sse",
        "--transport",
        "sse-only",
      ],
      "env": {
        "VLMRUN_API_KEY": "${VLMRUN_API_KEY}"
      }
    }
  }
}

Restart Claude Desktop after adding the configuration.

Add VLM Run MCP Server to Claude.ai (requires Claude Pro, Max, Team, or Enterprise):

Steps:

Go to claude.ai and navigate to Settings > Integrations
Click “Add custom integration” at the bottom of the Integrations section
Enter the server URL: https://mcp.vlm.run/${VLMRUN_API_KEY}/sse
Click “Add” to finish configuration
In any chat, use the Search and tools menu to enable VLM Run tools

Custom integrations are available for Claude Pro, Max, Team, and Enterprise plans only.

Usage: Once configured, simply upload an image and ask:

Extract invoice data from this image using document.invoice domain

Use VLM Run MCP Server with OpenAI’s responses API:

import openai

client = openai.Client()

# Create a response with MCP tools
response = client.responses.create(
    model="gpt-4.1",
    tools=[{
        "type": "mcp",
        "server_label": "vlm-run-mcp",
        "server_url": "https://mcp.vlm.run/mcp/sse",
        "require_approval": "never"
    }],
    input="Extract data from this invoice image"
)

print(response.output_text)

Add VLM Run MCP Server to your Cursor configuration:

Location: .cursor/mcp_servers.json in your project root

{
  "mcpServers": {
    "vlm-run-mcp": {
      "url": "https://mcp.vlm.run/${VLMRUN_API_KEY}/sse"
    }
  }
}

Restart Cursor after adding the configuration.

Test the connection

Ask your AI agent to list available tools

List available VLM Run tools

Expected response: A list of available tools like put_image_url, put_document_url, detect_faces, etc.

Try your VLM Run MCP tool interaction

Let’s extract details from an invoice image using VLM Run MCP tools:

Extract invoice data from this image (https://storage.googleapis.com/vlm-data-public-prod/hub/examples/document.invoice/invoice_1.jpg) using the document.invoice domain

Claude will automatically use the VLM Run MCP server to process your image and return structured JSON data.

Extract invoice data from this image (https://storage.googleapis.com/vlm-data-public-prod/hub/examples/document.invoice/invoice_1.jpg) using the document.invoice domain

Claude will automatically use the VLM Run MCP server to process your image and return structured JSON data.

# Extract from uploaded image
response = client.responses.create(
    model="gpt-4.1",
    tools=[{
        "type": "mcp",
        "server_label": "vlm-run-mcp",
        "server_url": "https://mcp.vlm.run/{api_key}/sse".format(api_key=VLMRUN_API_KEY),
        "require_approval": "never"
    }],
    input="Extract invoice data from this image (https://storage.googleapis.com/vlm-data-public-prod/hub/examples/document.invoice/invoice_1.jpg) using document.invoice domain",
    files=["invoice.jpg"]
)

Extract invoice data from this image (https://storage.googleapis.com/vlm-data-public-prod/hub/examples/document.invoice/invoice_1.jpg) using the document.invoice domain

Claude will automatically use the VLM Run MCP server to process your image and return structured JSON data.

Extract invoice data from this image (https://storage.googleapis.com/vlm-data-public-prod/hub/examples/document.invoice/invoice_1.jpg) using the document.invoice domain

Cursor will automatically use the VLM Run MCP server to process your image and return structured JSON data.

Troubleshooting

Claude Desktop not showing MCP tools

OpenAI Responses API SDK not recognizing MCP server

Claude Pro (Web) integration issues

Cursor not showing MCP tools

Connection timeout or server unreachable

Tool not found or extraction errors

Need help? Join our Discord channel or contact our support team to get help.

On this page

Installation
Try your VLM Run MCP tool interaction
Troubleshooting

Connect your AI agent to VLM Run’s remote MCP server and start building agentic workflows with visual AI in minutes.

Installation

Get your API key

Head over to the VLM Run Dashboard to get your API key ($VLMRUN_API_KEY). We’ll use this to authenticate your requests to the MCP server next.

Ping the MCP server to test

Copy-paste the server URL linked below in your browser and you should see a ping response from the MCP server.

https://mcp.vlm.run/${VLMRUN_API_KEY}/sse

Authentication using the above approach is purely experimental, and is subject to change. We’ll be announcing our OAuth 2.1-based authentication, as per the MCP spec, soon.

Add the server to your MCP client

Add this configuration to your Claude Desktop MCP settings:

Location (Mac) ~/Library/Application Support/Claude/claude_desktop_config.json
Location (Windows) %APPDATA%/Claude/claude_desktop_config.json See docs for more details.

{
  "mcpServers": {
    "vlm-run-mcp": {
      "args": [
        "mcp-remote",
        "https://mcp.vlm.run/${VLMRUN_API_KEY}/sse",
        "--transport",
        "sse-only",
      ],
      "env": {
        "VLMRUN_API_KEY": "${VLMRUN_API_KEY}"
      }
    }
  }
}

Restart Claude Desktop after adding the configuration.

Add this configuration to your Claude Desktop MCP settings:

Location (Mac) ~/Library/Application Support/Claude/claude_desktop_config.json
Location (Windows) %APPDATA%/Claude/claude_desktop_config.json See docs for more details.

{
  "mcpServers": {
    "vlm-run-mcp": {
      "args": [
        "mcp-remote",
        "https://mcp.vlm.run/${VLMRUN_API_KEY}/sse",
        "--transport",
        "sse-only",
      ],
      "env": {
        "VLMRUN_API_KEY": "${VLMRUN_API_KEY}"
      }
    }
  }
}

Restart Claude Desktop after adding the configuration.

Add VLM Run MCP Server to Claude.ai (requires Claude Pro, Max, Team, or Enterprise):

Steps:

Go to claude.ai and navigate to Settings > Integrations
Click “Add custom integration” at the bottom of the Integrations section
Enter the server URL: https://mcp.vlm.run/${VLMRUN_API_KEY}/sse
Click “Add” to finish configuration
In any chat, use the Search and tools menu to enable VLM Run tools

Custom integrations are available for Claude Pro, Max, Team, and Enterprise plans only.

Usage: Once configured, simply upload an image and ask:

Extract invoice data from this image using document.invoice domain

Use VLM Run MCP Server with OpenAI’s responses API:

import openai

client = openai.Client()

# Create a response with MCP tools
response = client.responses.create(
    model="gpt-4.1",
    tools=[{
        "type": "mcp",
        "server_label": "vlm-run-mcp",
        "server_url": "https://mcp.vlm.run/mcp/sse",
        "require_approval": "never"
    }],
    input="Extract data from this invoice image"
)

print(response.output_text)

Add VLM Run MCP Server to your Cursor configuration:

Location: .cursor/mcp_servers.json in your project root

{
  "mcpServers": {
    "vlm-run-mcp": {
      "url": "https://mcp.vlm.run/${VLMRUN_API_KEY}/sse"
    }
  }
}

Restart Cursor after adding the configuration.

Test the connection

Ask your AI agent to list available tools

List available VLM Run tools

Expected response: A list of available tools like put_image_url, put_document_url, detect_faces, etc.

Try your VLM Run MCP tool interaction

Let’s extract details from an invoice image using VLM Run MCP tools:

Extract invoice data from this image (https://storage.googleapis.com/vlm-data-public-prod/hub/examples/document.invoice/invoice_1.jpg) using the document.invoice domain

Claude will automatically use the VLM Run MCP server to process your image and return structured JSON data.

Extract invoice data from this image (https://storage.googleapis.com/vlm-data-public-prod/hub/examples/document.invoice/invoice_1.jpg) using the document.invoice domain

Claude will automatically use the VLM Run MCP server to process your image and return structured JSON data.

# Extract from uploaded image
response = client.responses.create(
    model="gpt-4.1",
    tools=[{
        "type": "mcp",
        "server_label": "vlm-run-mcp",
        "server_url": "https://mcp.vlm.run/{api_key}/sse".format(api_key=VLMRUN_API_KEY),
        "require_approval": "never"
    }],
    input="Extract invoice data from this image (https://storage.googleapis.com/vlm-data-public-prod/hub/examples/document.invoice/invoice_1.jpg) using document.invoice domain",
    files=["invoice.jpg"]
)

Extract invoice data from this image (https://storage.googleapis.com/vlm-data-public-prod/hub/examples/document.invoice/invoice_1.jpg) using the document.invoice domain

Claude will automatically use the VLM Run MCP server to process your image and return structured JSON data.

Extract invoice data from this image (https://storage.googleapis.com/vlm-data-public-prod/hub/examples/document.invoice/invoice_1.jpg) using the document.invoice domain

Cursor will automatically use the VLM Run MCP server to process your image and return structured JSON data.

Troubleshooting

Claude Desktop not showing MCP tools

OpenAI Responses API SDK not recognizing MCP server

Claude Pro (Web) integration issues

Cursor not showing MCP tools

Connection timeout or server unreachable

Tool not found or extraction errors

Need help? Join our Discord channel or contact our support team to get help.

On this page

Installation
Try your VLM Run MCP tool interaction
Troubleshooting

​Installation

​Try your VLM Run MCP tool interaction

​Troubleshooting

Node.js SDK

​Installation

​Try your VLM Run MCP tool interaction

​Troubleshooting

Installation

Try your VLM Run MCP tool interaction

Troubleshooting

Installation

Try your VLM Run MCP tool interaction

Troubleshooting