> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vlm.run/llms.txt
> Use this file to discover all available pages before exploring further.

# client.agent

> Learn how to use Agent Chat Completions with the VLM Run Node.js SDK

The `agent` component provides methods for interacting with VLM Run's Orion Agents for multi-modal chat completions.

## Chat Completions

Generate responses from the agent using the chat completions:

```typescript theme={"theme":{"light":"github-light","dark":"dark-plus"}}
import { VlmRun } from "vlmrun";

// Initialize the client
const client = new VlmRun({
  apiKey: "<VLMRUN_API_KEY>",
  baseURL: "https://api.vlm.run/v1"
});

// Basic text completion
const response = await client.agent.completions.create({
  model: "vlmrun-orion-1:auto",
  messages: [
    { role: "user", content: "What is the capital of France?" }
  ]
});

console.log(response.choices[0].message.content);
```

## Image Analysis

Analyze images using the agent:

```typescript theme={"theme":{"light":"github-light","dark":"dark-plus"}}
import { VlmRun } from "vlmrun";

const client = new VlmRun({
  apiKey: "<VLMRUN_API_KEY>",
  baseURL: "https://api.vlm.run/v1"
});

// Analyze an image
const response = await client.agent.completions.create({
  model: "vlmrun-orion-1:auto",
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "Describe this image in detail" },
        { type: "image_url", image_url: { url: "https://example.com/image.jpg", detail: "auto" } }
      ]
    }
  ]
});

console.log(response.choices[0].message.content);
```

## Video Analysis

Analyze videos using the agent:

```typescript theme={"theme":{"light":"github-light","dark":"dark-plus"}}
import { VlmRun } from "vlmrun";

const client = new VlmRun({
  apiKey: "<VLMRUN_API_KEY>",
  baseURL: "https://api.vlm.run/v1"
});

// Analyze a video
const response = await client.agent.completions.create({
  model: "vlmrun-orion-1:auto",
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "Summarize this video" },
        { type: "video_url", video_url: { url: "https://example.com/video.mp4" } }
      ]
    }
  ]
});

console.log(response.choices[0].message.content);
```

## Structured Outputs

Get structured JSON responses using TypeScript interfaces:

```typescript theme={"theme":{"light":"github-light","dark":"dark-plus"}}
import { VlmRun } from "vlmrun";

// Define response schema
interface ImageCaption {
  caption: string;
  tags: string[];
}

const client = new VlmRun({
  apiKey: "<VLMRUN_API_KEY>",
  baseURL: "https://api.vlm.run/v1"
});

// Get structured response
const response = await client.agent.completions.create({
  model: "vlmrun-orion-1:auto",
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "Generate a caption and tags for this image" },
        { type: "image_url", image_url: { url: "https://example.com/image.jpg" } }
      ]
    }
  ],
  response_format: {
    type: "json_schema",
    schema: {
      type: "object",
      properties: {
        caption: { type: "string", description: "Detailed caption of the image" },
        tags: { type: "array", items: { type: "string" }, description: "Tags describing the image" }
      },
      required: ["caption", "tags"]
    }
  }
});

// Parse the response
const result: ImageCaption = JSON.parse(response.choices[0].message.content);
console.log(result);
```

## Document Analysis

Analyze documents and PDFs:

```typescript theme={"theme":{"light":"github-light","dark":"dark-plus"}}
import { VlmRun } from "vlmrun";

const client = new VlmRun({
  apiKey: "<VLMRUN_API_KEY>",
  baseURL: "https://api.vlm.run/v1"
});

// Analyze a PDF document
const response = await client.agent.completions.create({
  model: "vlmrun-orion-1:auto",
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "Extract key information from this document" },
        { type: "file_url", file_url: { url: "https://example.com/document.pdf" } }
      ]
    }
  ]
});

console.log(response.choices[0].message.content);
```

## SDK Reference

### `client.agent.completions.create()`

Create a chat completion with the agent.

**Parameters:**

| Parameter         | Type             | Description                                   |
| ----------------- | ---------------- | --------------------------------------------- |
| `model`           | `string`         | Model to use (e.g., `vlmrun-orion-1:auto`)    |
| `messages`        | `Message[]`      | List of messages in the conversation          |
| `response_format` | `ResponseFormat` | Optional JSON schema for structured output    |
| `stream`          | `boolean`        | Enable streaming responses (default: `false`) |

**Returns:** `Promise<ChatCompletionResponse>`

### Message Content Types

| Type        | Description                    |
| ----------- | ------------------------------ |
| `text`      | Plain text content             |
| `image_url` | Image URL for image analysis   |
| `video_url` | Video URL for video analysis   |
| `file_url`  | File URL for document analysis |

## Best Practices

1. **Structured Outputs**
   * Define clear JSON schemas for predictable responses
   * Use TypeScript interfaces for type safety

2. **Multi-Modal Inputs**
   * Use appropriate content types (`image_url`, `video_url`, `file_url`)
   * Set `detail` level for images based on analysis needs

3. **Error Handling**
   * Always wrap API calls in try-catch blocks
   * Handle rate limits and timeouts appropriately