> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vlm.run/llms.txt
> Use this file to discover all available pages before exploring further.

# Introduction

> Introducing VLM Run Orion – the first visual agent that sees, reasons, and acts.

<Frame caption="At the core of Orion is a unified architecture that powers understanding, reasoning, and action across every visual modality.">
  <img src="https://mintcdn.com/autonomiai/8ymL7yyXv_4QkO9f/images/vlmrun-orion-arch.png?fit=max&auto=format&n=8ymL7yyXv_4QkO9f&q=85&s=dfb272709c57c97d9d536bd20dd682a9" width="100%" data-path="images/vlmrun-orion-arch.png" />
</Frame>

Today's frontier Vision-Language Models like GPT, Claude, and Gemini can describe images and answer questions, but they operate as monolithic inference engines. They generate descriptive outputs but cannot *act* on visual data with the precision, determinism, or compositional control required for production-grade workflows.

[Orion](https://vlm.run/orion) introduces a new paradigm for agentic visual reasoning and execution. Unlike monolithic VLMs, Orion orchestrates specialized computer vision tools – OCR, detection, segmentation, keypoint localization, diffusion, and geometric analysis – to execute complex multi-step visual workflows from natural language instructions. This marks the transition from passive visual understanding to **autonomous, tool-augmented visual intelligence** that bridges neural perception with symbolic execution.

<Tip>
  Looking to chat with VLM Run's Orion agents? Visit [chat.vlm.run](https://chat.vlm.run).
</Tip>

<Note>
  Read more about [VLM Run Orion](https://vlm.run/orion) in our [technical whitepaper](https://vlm.run/orion/whitepaper).
</Note>

## Agents Supported

Orion is available in two families, **Orion-1** (tool-calling agents) and **Orion-2** (code-execution agents), each with `fast`, `auto`, and `pro` tiers.

### Orion-1: Tool-Calling Agents

Orion-1 agents orchestrate specialized CV tools (OCR, detection, segmentation, etc.) via structured tool calls. Each tool invocation is a discrete API call managed by the agent.

<CardGroup cols={1}>
  <Card title="vlmrun-orion-1:fast" icon="bolt" horizontal>
    Our fast visual agent for simple multi-modal workflows. Optimized for speed and quick responses.
  </Card>

  <Card title="vlmrun-orion-1:auto" icon="wand-magic-sparkles" horizontal>
    Automatically selects the best model, tool and thinking budget based on your task complexity. Balanced performance and capability.
  </Card>

  <Card title="vlmrun-orion-1:pro" icon="gauge-high" horizontal>
    Our most capable visual agent for complex, multi-step workflows. Handles long tool-trajectories and advanced reasoning with a high thinking budget.
  </Card>
</CardGroup>

### Orion-2: Code-Execution Agents

Orion-2 agents write and execute Python code in a secure sandbox, composing CV operations programmatically. This enables multi-step pipelines, iterative refinement, and complex data transformations within a single turn. See the [Code Execution](/agents/code-execution) guide for details.

<CardGroup cols={1}>
  <Card title="vlmrun-orion-2:fast" icon="bolt" horizontal>
    Fast code-execution agent for quick pipelines. Uses lightweight models for rapid iteration.
  </Card>

  <Card title="vlmrun-orion-2:auto" icon="wand-magic-sparkles" horizontal>
    Automatically routes to the best backend model (Qwen, Gemma, or frontier models) based on task complexity. Default tier for Orion-2.
  </Card>

  <Card title="vlmrun-orion-2:pro" icon="gauge-high" horizontal>
    Most capable code-execution agent for complex multi-step pipelines with extended reasoning budgets.
  </Card>
</CardGroup>

<Note>
  Orion-2 also supports pinned backend variants for advanced use cases: `vlmrun-orion-2:qwen3.6-35b-a3b`, `vlmrun-orion-2:gemma4-26b-a4b`, `vlmrun-orion-2:kimi-2.6`, `vlmrun-orion-2:gpt-5.5`, and `vlmrun-orion-2:claude-opus-4.8`.
</Note>

## What makes VLM Run Agents unique?

Here are some key features of VLM Run Agents that set it apart from other AI agent platforms:

<CardGroup cols={2}>
  <Card title="Multi-Modal, Multi-Turn Reasoning" icon="brain">
    Execute complex multi-step visual workflows with adaptive context management across extended conversations.
  </Card>

  <Card title="First-class Visual AI Tools" icon="eye">
    Comprehensive suite of specialized tools across document, image, video, and multimodal processing—composable into multi-stage pipelines.
  </Card>

  <Card title="OpenAI-Compatible API" icon="code">
    Use our OpenAI Chat Completions endpoint to interact with VLM Run's Orion agents with just 2 lines of code change.
  </Card>

  <Card title="Enterprise-Ready" icon="shield">
    Our agents are SOC2-Type 2 and HIPAA-compliant, production-ready with automatic validation, with support for full traceability and auditability.
  </Card>
</CardGroup>

## How is VLM Run's Orion different from frontier models?

Unlike monolithic Vision-Language Models (VLMs like GPT-5, Claude 4.5, and Gemini 2.5), VLM Run's Orion family of visual agents delivers comprehensive capabilities across all modalities and tasks. The table below highlights key differences that matter for building production-grade visual workflows:

<div style={{ overflowX: 'auto', marginBottom: '2rem' }}>
  <table className="rotated-row-headers" style={{ tableLayout: 'fixed', width: '100%' }}>
    <thead>
      <tr>
        <th style={{ width: '4%', textAlign: 'center' }} />

        <th style={{ width: '16%', textAlign: 'center' }}>Task</th>
        <th style={{ width: '16%', textAlign: 'center' }}>VLM Run Orion</th>
        <th style={{ width: '16%', textAlign: 'center' }}>OpenAI GPT-5</th>
        <th style={{ width: '16%', textAlign: 'center' }}>Google Gemini 2.5</th>
        <th style={{ width: '16%', textAlign: 'center' }}>Anthropic Claude Sonnet 4.5</th>
        <th style={{ width: '16%', textAlign: 'center' }}>Alibaba Qwen3-VL 235B-A22B</th>
      </tr>
    </thead>

    <tbody style={{ textAlign: 'center' }}>
      <tr>
        <td rowspan="5" style={{ verticalAlign: 'middle' }}><span style={{ writingMode: 'vertical-rl', transform: 'rotate(180deg)', whiteSpace: 'nowrap' }}><strong>Image / Video</strong></span></td>
        <td>Understanding</td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#f59e0b', fontSize: '1.2em', fontWeight: 'bold'}}>⚠</span></td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#f59e0b', fontSize: '1.2em', fontWeight: 'bold'}}>⚠</span></td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
      </tr>

      <tr>
        <td>Reasoning</td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#ef4444', fontSize: '1.2em', fontWeight: 'bold'}}>✗</span></td>
        <td><span style={{color: '#ef4444', fontSize: '1.2em', fontWeight: 'bold'}}>✗</span></td>
        <td><span style={{color: '#ef4444', fontSize: '1.2em', fontWeight: 'bold'}}>✗</span></td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
      </tr>

      <tr>
        <td>Structured Outputs</td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
      </tr>

      <tr>
        <td><span style={{color: '#999999'}}>Multi-modal</span> Tool-Calling</td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#ef4444', fontSize: '1.2em', fontWeight: 'bold'}}>✗</span></td>
        <td><span style={{color: '#ef4444', fontSize: '1.2em', fontWeight: 'bold'}}>✗</span></td>
        <td><span style={{color: '#ef4444', fontSize: '1.2em', fontWeight: 'bold'}}>✗</span></td>
        <td><span style={{color: '#f59e0b', fontSize: '1.2em', fontWeight: 'bold'}}>⚠</span></td>
      </tr>

      <tr>
        <td>Specialized Skills</td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#ef4444', fontSize: '1.2em', fontWeight: 'bold'}}>✗</span></td>
        <td><span style={{color: '#f59e0b', fontSize: '1.2em', fontWeight: 'bold'}}>⚠</span></td>
        <td><span style={{color: '#f59e0b', fontSize: '1.2em', fontWeight: 'bold'}}>⚠</span></td>
        <td><span style={{color: '#ef4444', fontSize: '1.2em', fontWeight: 'bold'}}>✗</span></td>
      </tr>

      <tr />

      <tr>
        <td rowspan="5" style={{ verticalAlign: 'middle' }}><span style={{ writingMode: 'vertical-rl', transform: 'rotate(180deg)', whiteSpace: 'nowrap' }}><strong>Document</strong></span></td>
        <td>Understanding</td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
      </tr>

      <tr>
        <td>Reasoning</td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#ef4444', fontSize: '1.2em', fontWeight: 'bold'}}>✗</span></td>
      </tr>

      <tr>
        <td>Structured Outputs</td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
      </tr>

      <tr>
        <td><span style={{color: '#999999'}}>Multi-modal</span> Tool-Calling</td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#f59e0b', fontSize: '1.2em', fontWeight: 'bold'}}>⚠</span></td>
        <td><span style={{color: '#f59e0b', fontSize: '1.2em', fontWeight: 'bold'}}>⚠</span></td>
        <td><span style={{color: '#f59e0b', fontSize: '1.2em', fontWeight: 'bold'}}>⚠</span></td>
        <td><span style={{color: '#ef4444', fontSize: '1.2em', fontWeight: 'bold'}}>✗</span></td>
      </tr>

      <tr>
        <td>Specialized Skills</td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#f59e0b', fontSize: '1.2em', fontWeight: 'bold'}}>⚠</span></td>
        <td><span style={{color: '#22c55e', fontSize: '1.2em', fontWeight: 'bold'}}>✓</span></td>
        <td><span style={{color: '#ef4444', fontSize: '1.2em', fontWeight: 'bold'}}>✗</span></td>
      </tr>
    </tbody>
  </table>
</div>

<Info>
  In the table above, we refer to **Specialized Skills** as tasks such as object localization, segmentation, image-generation / editing, or geometric tools typically found in specialized computer vision applications.
</Info>

**Key advantages for developers:**

* **Mixed-modality Reasoning**: Only VLM Run's Orion agents provide full reasoning across images, documents, and video - critical for building multi-step visual workflows.
* **Multi-modal Tool-Calling**: With unique tool-calling support for images, videos and documents, VLM Run's Orion agents enable multi-modal reasoning and execution that other models cannot perform.
* **Production-Ready Structured Outputs**: Consistent structured output support across all modalities with automatic validation and retry logic

## Let's get started!

Below you'll find the API reference and code samples so you can start building intelligent agents for your use case.
Sign up for an API key on our [platform](https://app.vlm.run/?utm_source=docs\&utm_medium=link\&utm_campaign=chat), then check out some of our [cookbooks](https://github.com/autonomi-ai/vlm-cookbook) to learn how to use VLM Run Agents to build sophisticated visual AI workflows.

<CardGroup cols={2}>
  <Card title="Chat" icon="comment" href="https://chat.vlm.run/?utm_source=docs&utm_medium=link&utm_campaign=chat">
    Chat with our visual agent direcly in your browser.
  </Card>

  <Card title="Capabilities" icon="trophy" href="/agents/capabilities/image/captioning">
    See the complete catalog of visual AI capabilities and tools.
  </Card>

  <Card title="SDK Reference" icon="book-open" href="/sdk-reference/components/agent">
    Enough talk, show me the code.
  </Card>

  <Card title="Cookbooks" icon="code" href="https://github.com/vlm-run/vlmrun-cookbook">
    Various cookbooks showcasing VLM Run Agents in action.
  </Card>
</CardGroup>