> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vlm.run/llms.txt
> Use this file to discover all available pages before exploring further.

# Image Tools

> Image tools for cropping, rotating, enhancing, and transforming images

VLM Run's Orion agents can leverage various image-editing tools such as cropping, rotating, super-resolution, and de-oldify. These tools are designed to help you enhance image quality, extract specific regions, correct orientation, and restore historical photos with modern AI techniques.

<Tip>
  For best results, we recommend using the [Structured Outputs API](/agents/structured-responses) to get responses in a structured and validated data format.
</Tip>

### 1. Image Cropping

Extract specific regions or focus on particular subjects within an image.

```mdx wrap theme={"theme":{"light":"github-light","dark":"dark-plus"}}
Crop the clock to tell the time more clearly.
```

<Frame caption="Example of intelligent cropping to focus on the clock.">
  <div style={{ display: 'flex', gap: '20px' }}>
    <div style={{ flex: '0 0 50%', maxWidth: '33%' }}>
      <img width="100%" src="https://mintcdn.com/autonomiai/n0UzFHKSRrWx1Yji/agents/assets/clock.jpg?fit=max&auto=format&n=n0UzFHKSRrWx1Yji&q=85&s=91daa94decc4a6052e298a0ba80b00aa" style={{ border: '1px solid #ddd', borderRadius: '4px' }} data-path="agents/assets/clock.jpg" />
    </div>

    <div style={{ flex: '0 0 50%', maxWidth: '67%' }}>
      <img width="95%" src="https://mintcdn.com/autonomiai/n0UzFHKSRrWx1Yji/agents/assets/clock-crop.jpg?fit=max&auto=format&n=n0UzFHKSRrWx1Yji&q=85&s=343542e42a1ca2ce56ca4b2126aa20c8" style={{ border: '1px solid #ddd', borderRadius: '4px' }} data-path="agents/assets/clock-crop.jpg" />
    </div>
  </div>
</Frame>

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  from vlmrun.client import VLMRun

  # Initialize the VLMRun client
  client = VLMRun(api_key="<VLMRUN_API_KEY>")

  # Crop image to focus on main subject
  response = client.agent.completions.create(
      model="vlmrun-orion-1:auto",
      messages=[
          {
              "role": "user",
              "content": [
                  {"type": "text", "text": "Crop the clock to tell the time more clearly"},
                  {"type": "image_url", "image_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/clock.jpg", "detail": "auto"}}
              ]
          }
      ]
  )

  print(response.choices[0].message.content)
  # >>> {"url": "https://.../cropped.jpg", "label": "clock", "xywh": [0.2, 0.2, 0.6, 0.6]}
  ```

  ```python Python - Structured Outputs theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  from vlmrun.client import VLMRun
  from pydantic import BaseModel, Field

  # Define the response schema
  class ImageCropResponse(BaseModel):
      url: str = Field(..., description="URL of the cropped image")
      label: str = Field(..., description="Object label")
      xywh: tuple[float, float, float, float] = Field(..., description="Bounding box (x, y, w, h)")

  # Initialize the VLMRun client
  client = VLMRun(api_key="<VLMRUN_API_KEY>")

  # Crop image with structured output
  response = client.agent.completions.create(
      model="vlmrun-orion-1:auto",
      messages=[
          {
              "role": "user",
              "content": [
                  {"type": "text", "text": "Crop the clock to tell the time more clearly"},
                  {"type": "image_url", "image_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/clock.jpg", "detail": "auto"}}
              ]
          }
      ],
      response_format={"type": "json_schema", "schema": ImageCropResponse.model_json_schema()}
  )

  # Validate the response
  result = ImageCropResponse.model_validate_json(response.choices[0].message.content)
  ```

  ```typescript Node.js theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  import { VlmRun } from "vlmrun";

  const client = new VlmRun({
    apiKey: "<VLMRUN_API_KEY>",
    baseURL: "https://api.vlm.run/v1"
  });

  const response = await client.agent.completions.create({
    model: "vlmrun-orion-1:auto",
    messages: [
      {
        role: "user",
        content: [
          { type: "text", text: "Crop the clock to tell the time more clearly" },
          { type: "image_url", image_url: { url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/clock.jpg", detail: "auto" } }
        ]
      }
    ]
  });

  console.log(response.choices[0].message.content);
  ```

  ```typescript Node.js - Structured Outputs [expandable] theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  import { VlmRun } from "vlmrun";
  import { z } from "zod";
  import { zodToJsonSchema } from "zod-to-json-schema";

  // Define the response schema with Zod
  const ImageCropResponseSchema = z.object({
    url: z.string().describe("URL of the cropped image"),
    label: z.string().describe("Object label"),
    xywh: z.array(z.number()).describe("Bounding box (x, y, w, h)")
  });

  // Initialize the VLMRun client
  const client = new VlmRun({
    apiKey: "<VLMRUN_API_KEY>",
    baseURL: "https://api.vlm.run/v1"
  });

  // Crop image with structured output
  const response = await client.agent.completions.create({
    model: "vlmrun-orion-1:auto",
    messages: [
      {
        role: "user",
        content: [
          { type: "text", text: "Crop the clock to tell the time more clearly" },
          { type: "image_url", image_url: { url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/clock.jpg", detail: "auto" } }
        ]
      }
    ],
    response_format: {
      type: "json_schema",
      schema: zodToJsonSchema(ImageCropResponseSchema)
    }
  });

  const result = ImageCropResponseSchema.parse(JSON.parse(response.choices[0].message.content));
  ```
</CodeGroup>

While the demonstration uses a single crop, we also support cropping multiple regions at once.

### 2. Image Rotation

Correct image orientation or apply creative rotations for better composition.

```mdx wrap theme={"theme":{"light":"github-light","dark":"dark-plus"}}
Rotate the image 90 degrees clockwise to correct the orientation.
```

<Frame caption="Example of image rotation by 90 degrees clockwise.">
  <div style={{ display: 'flex', gap: '4px' }}>
    <div style={{ flex: '0 0 50%', maxWidth: '60%' }}>
      <img width="100%" src="https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.object-detection/cats.jpg" style={{ width: '100%', border: '1px solid #ddd', borderRadius: '4px' }} alt="Original image before rotation" />
    </div>

    <div style={{ flex: '0 0 50%', maxWidth: '40%' }}>
      <img width="80%" src="https://mintcdn.com/autonomiai/n0UzFHKSRrWx1Yji/agents/assets/image-tool-rotated.jpg?fit=max&auto=format&n=n0UzFHKSRrWx1Yji&q=85&s=fab78f4b913638834cba804b02822878" style={{ width: '80%', border: '1px solid #ddd', borderRadius: '4px' }} alt="Image after 90-degree rotation in the clockwise direction" data-path="agents/assets/image-tool-rotated.jpg" />
    </div>
  </div>
</Frame>

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  from vlmrun.client import VLMRun

  # Initialize the VLMRun client
  client = VLMRun(api_key="<VLMRUN_API_KEY>")

  # Rotate image to correct orientation
  response = client.agent.completions.create(
      model="vlmrun-orion-1:auto",
      messages=[
          {
              "role": "user",
              "content": [
                  {"type": "text", "text": "Rotate this image 90 degrees clockwise"},
                  {"type": "image_url", "image_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.object-detection/cats.jpg"}}
              ]
          }
      ]
  )

  print(response.choices[0].message.content)
  # >>> {"url": "https://.../rotated.jpg", "angle": 90}
  ```

  ```python Python - Structured Outputs theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  from vlmrun.client import VLMRun
  from pydantic import BaseModel, Field

  # Define the response schema
  class ImageRotationResponse(BaseModel):
      url: str = Field(..., description="URL of the rotated image")
      angle: int = Field(..., description="Rotation angle (0, 90, 180, 270) degrees clockwise")

  # Initialize the VLMRun client
  client = VLMRun(api_key="<VLMRUN_API_KEY>")

  # Rotate image with structured output
  response = client.agent.completions.create(
      model="vlmrun-orion-1:auto",
      messages=[
          {
              "role": "user",
              "content": [
                  {"type": "text", "text": "Rotate this image 90 degrees clockwise"},
                  {"type": "image_url", "image_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.object-detection/cats.jpg"}}
              ]
          }
      ],
      response_format={"type": "json_schema", "schema": ImageRotationResponse.model_json_schema()}
  )

  # Validate the response
  result = ImageRotationResponse.model_validate_json(response.choices[0].message.content)
  ```

  ```typescript Node.js theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  import { VlmRun } from "vlmrun";

  const client = new VlmRun({
    apiKey: "<VLMRUN_API_KEY>",
    baseURL: "https://api.vlm.run/v1"
  });

  const response = await client.agent.completions.create({
    model: "vlmrun-orion-1:auto",
    messages: [
      {
        role: "user",
        content: [
          { type: "text", text: "Rotate this image 90 degrees clockwise" },
          { type: "image_url", image_url: { url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.object-detection/cats.jpg" } }
        ]
      }
    ]
  });

  console.log(response.choices[0].message.content);
  ```

  ```typescript Node.js - Structured Outputs [expandable] theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  import { VlmRun } from "vlmrun";
  import { z } from "zod";
  import { zodToJsonSchema } from "zod-to-json-schema";

  // Define the response schema with Zod
  const ImageRotationResponseSchema = z.object({
    url: z.string().describe("URL of the rotated image"),
    angle: z.number().int().describe("Rotation angle (0, 90, 180, 270) degrees clockwise")
  });

  // Initialize the VLMRun client
  const client = new VlmRun({
    apiKey: "<VLMRUN_API_KEY>",
    baseURL: "https://api.vlm.run/v1"
  });

  // Rotate image with structured output
  const response = await client.agent.completions.create({
    model: "vlmrun-orion-1:auto",
    messages: [
      {
        role: "user",
        content: [
          { type: "text", text: "Rotate this image 90 degrees clockwise" },
          { type: "image_url", image_url: { url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.object-detection/cats.jpg" } }
        ]
      }
    ],
    response_format: {
      type: "json_schema",
      schema: zodToJsonSchema(ImageRotationResponseSchema)
    }
  });

  const result = ImageRotationResponseSchema.parse(JSON.parse(response.choices[0].message.content));
  ```
</CodeGroup>

### 3. Super-Resolution Enhancement

Upscale images while maintaining quality and adding realistic details.

```mdx wrap theme={"theme":{"light":"github-light","dark":"dark-plus"}}
Enhance this image using super-resolution to increase its resolution while preserving quality.
```

<Frame caption="Example of super-resolution enhancement showing 4x upscaling with quality preservation.">
  <div style={{ display: 'flex', gap: '20px', justifyContent: 'center', alignItems: 'center' }}>
    <img src="https://mintcdn.com/autonomiai/n0UzFHKSRrWx1Yji/agents/assets/vegetables-lo.jpg?fit=max&auto=format&n=n0UzFHKSRrWx1Yji&q=85&s=1db43698cae450a5e91809908c5eccc9" alt="Original low-resolution image" style={{ width: '100%', border: '1px solid #ddd', borderRadius: '4px' }} width="520" height="319" data-path="agents/assets/vegetables-lo.jpg" />
  </div>

  <div style={{ display: 'flex', gap: '20px', justifyContent: 'center' }}>
    <img src="https://mintcdn.com/autonomiai/n0UzFHKSRrWx1Yji/agents/assets/vegetables-hi.jpg?fit=max&auto=format&n=n0UzFHKSRrWx1Yji&q=85&s=f8fe7359e1edd1da2c9b85f4a2031829" alt="Enhanced high-resolution image" style={{ width: '90%', border: '1px solid #ddd', borderRadius: '4px' }} width="1280" height="800" data-path="agents/assets/vegetables-hi.jpg" />
  </div>
</Frame>

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  from vlmrun.client import VLMRun

  # Initialize the VLMRun client
  client = VLMRun(api_key="<VLMRUN_API_KEY>")

  # Apply super-resolution enhancement
  response = client.agent.completions.create(
      model="vlmrun-orion-1:auto",
      messages=[
          {
              "role": "user",
              "content": [
                  {"type": "text", "text": "Enhance this image using super-resolution to increase its resolution while preserving quality"},
                  {"type": "image_url", "image_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/vegetables-lo.jpg"}}
              ]
          }
      ]
  )

  print(response.choices[0].message.content)
  # >>> {"url": "https://.../enhanced.jpg"}
  ```

  ```python Python - Structured Outputs theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  from vlmrun.client import VLMRun
  from pydantic import BaseModel, Field

  # Define the response schema
  class SuperResolutionResponse(BaseModel):
      url: str = Field(..., description="URL of the enhanced image")

  # Initialize the VLMRun client
  client = VLMRun(api_key="<VLMRUN_API_KEY>")

  # Enhance image with structured output
  response = client.agent.completions.create(
      model="vlmrun-orion-1:auto",
      messages=[
          {
              "role": "user",
              "content": [
                  {"type": "text", "text": "Enhance this image using super-resolution to increase its resolution while preserving quality"},
                  {"type": "image_url", "image_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/vegetables-lo.jpg"}}
              ]
          }
      ],
      response_format={"type": "json_schema", "schema": SuperResolutionResponse.model_json_schema()}
  )

  # Validate the response
  result = SuperResolutionResponse.model_validate_json(response.choices[0].message.content)
  ```

  ```typescript Node.js theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  import { VlmRun } from "vlmrun";

  const client = new VlmRun({
    apiKey: "<VLMRUN_API_KEY>",
    baseURL: "https://api.vlm.run/v1"
  });

  const response = await client.agent.completions.create({
    model: "vlmrun-orion-1:auto",
    messages: [
      {
        role: "user",
        content: [
          { type: "text", text: "Enhance this image using super-resolution to increase its resolution while preserving quality" },
          { type: "image_url", image_url: { url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/vegetables-lo.jpg" } }
        ]
      }
    ]
  });

  console.log(response.choices[0].message.content);
  ```

  ```typescript Node.js - Structured Outputs [expandable] theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  import { VlmRun } from "vlmrun";
  import { z } from "zod";
  import { zodToJsonSchema } from "zod-to-json-schema";

  // Define the response schema with Zod
  const SuperResolutionResponseSchema = z.object({
    url: z.string().describe("URL of the enhanced image")
  });

  // Initialize the VLMRun client
  const client = new VlmRun({
    apiKey: "<VLMRUN_API_KEY>",
    baseURL: "https://api.vlm.run/v1"
  });

  // Enhance image with structured output
  const response = await client.agent.completions.create({
    model: "vlmrun-orion-1:auto",
    messages: [
      {
        role: "user",
        content: [
          { type: "text", text: "Enhance this image using super-resolution to increase its resolution while preserving quality" },
          { type: "image_url", image_url: { url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/vegetables-lo.jpg" } }
        ]
      }
    ],
    response_format: {
      type: "json_schema",
      schema: zodToJsonSchema(SuperResolutionResponseSchema)
    }
  });

  const result = SuperResolutionResponseSchema.parse(JSON.parse(response.choices[0].message.content));
  ```
</CodeGroup>

### 4. De-Oldify (Colorization)

Transform black and white or sepia images into vibrant color photos using AI.

```mdx wrap theme={"theme":{"light":"github-light","dark":"dark-plus"}}
De-oldify this image so that it's colorized and upsampled.
```

<Frame caption="Example of AI-powered colorization transforming a vintage black and white photo.">
  <div style={{ flex: '0 0 50%', maxWidth: '50%' }}>
    <img width="100%" src="https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/lunch-skyscraper.jpg" style={{ width: '100%', border: '1px solid #ddd', borderRadius: '4px' }} alt="Original black and white image" />
  </div>

  <div style={{ flex: '0 0 50%', maxWidth: '50%' }}>
    <img width="100%" src="https://mintcdn.com/autonomiai/n0UzFHKSRrWx1Yji/agents/assets/image-deoldify.jpg?fit=max&auto=format&n=n0UzFHKSRrWx1Yji&q=85&s=1a29359ab6bac654ba76509d2f3e91fb" style={{ width: '100%', border: '1px solid #ddd', borderRadius: '4px' }} alt="Colorized image with realistic colors" data-path="agents/assets/image-deoldify.jpg" />
  </div>
</Frame>

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  from vlmrun.client import VLMRun

  # Initialize the VLMRun client
  client = VLMRun(api_key="<VLMRUN_API_KEY>")

  # Colorize black and white image
  response = client.agent.completions.create(
      model="vlmrun-orion-1:auto",
      messages=[
          {
              "role": "user",
              "content": [
                  {"type": "text", "text": "De-oldify this image so that it's colorized and upsampled"},
                  {"type": "image_url", "image_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/lunch-skyscraper.jpg"}}
              ]
          }
      ]
  )

  print(response.choices[0].message.content)
  # >>> {"url": "https://.../colorized.jpg"}
  ```

  ```python Python - Structured Outputs theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  from vlmrun.client import VLMRun
  from pydantic import BaseModel, Field

  # Define the response schema
  class DeOldifyResponse(BaseModel):
      url: str = Field(..., description="URL of the colorized image")

  # Initialize the VLMRun client
  client = VLMRun(api_key="<VLMRUN_API_KEY>")

  # Colorize image with structured output
  response = client.agent.completions.create(
      model="vlmrun-orion-1:auto",
      messages=[
          {
              "role": "user",
              "content": [
                  {"type": "text", "text": "De-oldify this image so that it's colorized and upsampled"},
                  {"type": "image_url", "image_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/lunch-skyscraper.jpg"}}
              ]
          }
      ],
      response_format={"type": "json_schema", "schema": DeOldifyResponse.model_json_schema()}
  )

  # Validate the response
  result = DeOldifyResponse.model_validate_json(response.choices[0].message.content)
  ```

  ```typescript Node.js theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  import { VlmRun } from "vlmrun";

  const client = new VlmRun({
    apiKey: "<VLMRUN_API_KEY>",
    baseURL: "https://api.vlm.run/v1"
  });

  const response = await client.agent.completions.create({
    model: "vlmrun-orion-1:auto",
    messages: [
      {
        role: "user",
        content: [
          { type: "text", text: "De-oldify this image so that it's colorized and upsampled" },
          { type: "image_url", image_url: { url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/lunch-skyscraper.jpg" } }
        ]
      }
    ]
  });

  console.log(response.choices[0].message.content);
  ```

  ```typescript Node.js - Structured Outputs [expandable] theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  import { VlmRun } from "vlmrun";
  import { z } from "zod";
  import { zodToJsonSchema } from "zod-to-json-schema";

  // Define the response schema with Zod
  const DeOldifyResponseSchema = z.object({
    url: z.string().describe("URL of the colorized image")
  });

  // Initialize the VLMRun client
  const client = new VlmRun({
    apiKey: "<VLMRUN_API_KEY>",
    baseURL: "https://api.vlm.run/v1"
  });

  // Colorize image with structured output
  const response = await client.agent.completions.create({
    model: "vlmrun-orion-1:auto",
    messages: [
      {
        role: "user",
        content: [
          { type: "text", text: "De-oldify this image so that it's colorized and upsampled" },
          { type: "image_url", image_url: { url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/lunch-skyscraper.jpg" } }
        ]
      }
    ],
    response_format: {
      type: "json_schema",
      schema: zodToJsonSchema(DeOldifyResponseSchema)
    }
  });

  const result = DeOldifyResponseSchema.parse(JSON.parse(response.choices[0].message.content));
  ```
</CodeGroup>

## FAQ

<AccordionGroup>
  <Accordion title="What image formats are supported for editing?" icon="image">
    * **JPEG/JPG**: Most common format with excellent compatibility
    * **PNG**: Lossless format with transparency support
    * **TIFF**: High-quality format for professional editing
    * **WebP**: Modern format with superior compression
    * **BMP**: Uncompressed bitmap format
    * **Quality Preservation**: Maintains original image quality in all transformations
  </Accordion>

  <Accordion title="What are the best practices for image cropping?" icon="scissors">
    * **Rule of Thirds**: Align subjects with intersection points for better composition
    * **Aspect Ratio**: Maintain consistent aspect ratios for professional results
    * **Subject Focus**: Keep the main subject centered or following composition rules
    * **Background Removal**: Remove distracting elements while preserving context
  </Accordion>

  <Accordion title="How accurate is the super-resolution enhancement?" icon="zoom-in">
    * **AI-Powered**: Uses advanced neural networks for realistic detail generation
    * **Multiple Scales**: Supports 2x, 4x, and 8x upscaling with quality preservation
    * **Detail Enhancement**: Intelligently adds realistic textures and patterns
    * **Quality Metrics**: Provides confidence scores for enhancement quality
  </Accordion>

  <Accordion title="How realistic is the de-oldify colorization?" icon="palette">
    * **Historical Accuracy**: Uses context-aware AI to suggest period-appropriate colors
    * **Natural Colors**: Generates realistic skin tones, clothing, and environmental colors
    * **Confidence Scoring**: Provides confidence levels for color accuracy
    * **Region Analysis**: Identifies and colors different regions with appropriate palettes
  </Accordion>
</AccordionGroup>