Image Tools - VLM Run

VLM Run’s Orion agents can leverage various image-editing tools such as cropping, rotating, super-resolution, and de-oldify. These tools are designed to help you enhance image quality, extract specific regions, correct orientation, and restore historical photos with modern AI techniques.

For best results, we recommend using the Structured Outputs API to get responses in a structured and validated data format.

1. Image Cropping

Extract specific regions or focus on particular subjects within an image.

Crop the clock to tell the time more clearly.

from vlmrun.client import VLMRun

# Initialize the VLMRun client
client = VLMRun(api_key="<VLMRUN_API_KEY>")

# Crop image to focus on main subject
response = client.agent.completions.create(
    model="vlmrun-orion-1:auto",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Crop the clock to tell the time more clearly"},
                {"type": "image_url", "image_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/clock.jpg", "detail": "auto"}}
            ]
        }
    ]
)

print(response.choices[0].message.content)
# >>> {"url": "https://.../cropped.jpg", "label": "clock", "xywh": [0.2, 0.2, 0.6, 0.6]}

from vlmrun.client import VLMRun
from pydantic import BaseModel, Field

# Define the response schema
class ImageCropResponse(BaseModel):
    url: str = Field(..., description="URL of the cropped image")
    label: str = Field(..., description="Object label")
    xywh: tuple[float, float, float, float] = Field(..., description="Bounding box (x, y, w, h)")

# Initialize the VLMRun client
client = VLMRun(api_key="<VLMRUN_API_KEY>")

# Crop image with structured output
response = client.agent.completions.create(
    model="vlmrun-orion-1:auto",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Crop the clock to tell the time more clearly"},
                {"type": "image_url", "image_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/clock.jpg", "detail": "auto"}}
            ]
        }
    ],
    response_format={"type": "json_schema", "schema": ImageCropResponse.model_json_schema()}
)

# Validate the response
result = ImageCropResponse.model_validate_json(response.choices[0].message.content)

import { VlmRun } from "vlmrun";

const client = new VlmRun({
  apiKey: "<VLMRUN_API_KEY>",
  baseURL: "https://api.vlm.run/v1"
});

const response = await client.agent.completions.create({
  model: "vlmrun-orion-1:auto",
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "Crop the clock to tell the time more clearly" },
        { type: "image_url", image_url: { url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/clock.jpg", detail: "auto" } }
      ]
    }
  ]
});

console.log(response.choices[0].message.content);

import { VlmRun } from "vlmrun";
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";

// Define the response schema with Zod
const ImageCropResponseSchema = z.object({
  url: z.string().describe("URL of the cropped image"),
  label: z.string().describe("Object label"),
  xywh: z.array(z.number()).describe("Bounding box (x, y, w, h)")
});

// Initialize the VLMRun client
const client = new VlmRun({
  apiKey: "<VLMRUN_API_KEY>",
  baseURL: "https://api.vlm.run/v1"
});

// Crop image with structured output
const response = await client.agent.completions.create({
  model: "vlmrun-orion-1:auto",
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "Crop the clock to tell the time more clearly" },
        { type: "image_url", image_url: { url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/clock.jpg", detail: "auto" } }
      ]
    }
  ],
  response_format: {
    type: "json_schema",
    schema: zodToJsonSchema(ImageCropResponseSchema)
  }
});

const result = ImageCropResponseSchema.parse(JSON.parse(response.choices[0].message.content));

While the demonstration uses a single crop, we also support cropping multiple regions at once.

2. Image Rotation

Correct image orientation or apply creative rotations for better composition.

Rotate the image 90 degrees clockwise to correct the orientation.

Image after 90-degree rotation in the clockwise direction

from vlmrun.client import VLMRun

# Initialize the VLMRun client
client = VLMRun(api_key="<VLMRUN_API_KEY>")

# Rotate image to correct orientation
response = client.agent.completions.create(
    model="vlmrun-orion-1:auto",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Rotate this image 90 degrees clockwise"},
                {"type": "image_url", "image_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.object-detection/cats.jpg"}}
            ]
        }
    ]
)

print(response.choices[0].message.content)
# >>> {"url": "https://.../rotated.jpg", "angle": 90}

from vlmrun.client import VLMRun
from pydantic import BaseModel, Field

# Define the response schema
class ImageRotationResponse(BaseModel):
    url: str = Field(..., description="URL of the rotated image")
    angle: int = Field(..., description="Rotation angle (0, 90, 180, 270) degrees clockwise")

# Initialize the VLMRun client
client = VLMRun(api_key="<VLMRUN_API_KEY>")

# Rotate image with structured output
response = client.agent.completions.create(
    model="vlmrun-orion-1:auto",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Rotate this image 90 degrees clockwise"},
                {"type": "image_url", "image_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.object-detection/cats.jpg"}}
            ]
        }
    ],
    response_format={"type": "json_schema", "schema": ImageRotationResponse.model_json_schema()}
)

# Validate the response
result = ImageRotationResponse.model_validate_json(response.choices[0].message.content)

import { VlmRun } from "vlmrun";

const client = new VlmRun({
  apiKey: "<VLMRUN_API_KEY>",
  baseURL: "https://api.vlm.run/v1"
});

const response = await client.agent.completions.create({
  model: "vlmrun-orion-1:auto",
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "Rotate this image 90 degrees clockwise" },
        { type: "image_url", image_url: { url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.object-detection/cats.jpg" } }
      ]
    }
  ]
});

console.log(response.choices[0].message.content);

import { VlmRun } from "vlmrun";
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";

// Define the response schema with Zod
const ImageRotationResponseSchema = z.object({
  url: z.string().describe("URL of the rotated image"),
  angle: z.number().int().describe("Rotation angle (0, 90, 180, 270) degrees clockwise")
});

// Initialize the VLMRun client
const client = new VlmRun({
  apiKey: "<VLMRUN_API_KEY>",
  baseURL: "https://api.vlm.run/v1"
});

// Rotate image with structured output
const response = await client.agent.completions.create({
  model: "vlmrun-orion-1:auto",
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "Rotate this image 90 degrees clockwise" },
        { type: "image_url", image_url: { url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.object-detection/cats.jpg" } }
      ]
    }
  ],
  response_format: {
    type: "json_schema",
    schema: zodToJsonSchema(ImageRotationResponseSchema)
  }
});

const result = ImageRotationResponseSchema.parse(JSON.parse(response.choices[0].message.content));

3. Super-Resolution Enhancement

Upscale images while maintaining quality and adding realistic details.

Enhance this image using super-resolution to increase its resolution while preserving quality.

from vlmrun.client import VLMRun

# Initialize the VLMRun client
client = VLMRun(api_key="<VLMRUN_API_KEY>")

# Apply super-resolution enhancement
response = client.agent.completions.create(
    model="vlmrun-orion-1:auto",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Enhance this image using super-resolution to increase its resolution while preserving quality"},
                {"type": "image_url", "image_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/vegetables-lo.jpg"}}
            ]
        }
    ]
)

print(response.choices[0].message.content)
# >>> {"url": "https://.../enhanced.jpg"}

from vlmrun.client import VLMRun
from pydantic import BaseModel, Field

# Define the response schema
class SuperResolutionResponse(BaseModel):
    url: str = Field(..., description="URL of the enhanced image")

# Initialize the VLMRun client
client = VLMRun(api_key="<VLMRUN_API_KEY>")

# Enhance image with structured output
response = client.agent.completions.create(
    model="vlmrun-orion-1:auto",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Enhance this image using super-resolution to increase its resolution while preserving quality"},
                {"type": "image_url", "image_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/vegetables-lo.jpg"}}
            ]
        }
    ],
    response_format={"type": "json_schema", "schema": SuperResolutionResponse.model_json_schema()}
)

# Validate the response
result = SuperResolutionResponse.model_validate_json(response.choices[0].message.content)

import { VlmRun } from "vlmrun";

const client = new VlmRun({
  apiKey: "<VLMRUN_API_KEY>",
  baseURL: "https://api.vlm.run/v1"
});

const response = await client.agent.completions.create({
  model: "vlmrun-orion-1:auto",
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "Enhance this image using super-resolution to increase its resolution while preserving quality" },
        { type: "image_url", image_url: { url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/vegetables-lo.jpg" } }
      ]
    }
  ]
});

console.log(response.choices[0].message.content);

import { VlmRun } from "vlmrun";
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";

// Define the response schema with Zod
const SuperResolutionResponseSchema = z.object({
  url: z.string().describe("URL of the enhanced image")
});

// Initialize the VLMRun client
const client = new VlmRun({
  apiKey: "<VLMRUN_API_KEY>",
  baseURL: "https://api.vlm.run/v1"
});

// Enhance image with structured output
const response = await client.agent.completions.create({
  model: "vlmrun-orion-1:auto",
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "Enhance this image using super-resolution to increase its resolution while preserving quality" },
        { type: "image_url", image_url: { url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/vegetables-lo.jpg" } }
      ]
    }
  ],
  response_format: {
    type: "json_schema",
    schema: zodToJsonSchema(SuperResolutionResponseSchema)
  }
});

const result = SuperResolutionResponseSchema.parse(JSON.parse(response.choices[0].message.content));

4. De-Oldify (Colorization)

Transform black and white or sepia images into vibrant color photos using AI.

De-oldify this image so that it's colorized and upsampled.

from vlmrun.client import VLMRun

# Initialize the VLMRun client
client = VLMRun(api_key="<VLMRUN_API_KEY>")

# Colorize black and white image
response = client.agent.completions.create(
    model="vlmrun-orion-1:auto",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "De-oldify this image so that it's colorized and upsampled"},
                {"type": "image_url", "image_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/lunch-skyscraper.jpg"}}
            ]
        }
    ]
)

print(response.choices[0].message.content)
# >>> {"url": "https://.../colorized.jpg"}

from vlmrun.client import VLMRun
from pydantic import BaseModel, Field

# Define the response schema
class DeOldifyResponse(BaseModel):
    url: str = Field(..., description="URL of the colorized image")

# Initialize the VLMRun client
client = VLMRun(api_key="<VLMRUN_API_KEY>")

# Colorize image with structured output
response = client.agent.completions.create(
    model="vlmrun-orion-1:auto",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "De-oldify this image so that it's colorized and upsampled"},
                {"type": "image_url", "image_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/lunch-skyscraper.jpg"}}
            ]
        }
    ],
    response_format={"type": "json_schema", "schema": DeOldifyResponse.model_json_schema()}
)

# Validate the response
result = DeOldifyResponse.model_validate_json(response.choices[0].message.content)

import { VlmRun } from "vlmrun";

const client = new VlmRun({
  apiKey: "<VLMRUN_API_KEY>",
  baseURL: "https://api.vlm.run/v1"
});

const response = await client.agent.completions.create({
  model: "vlmrun-orion-1:auto",
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "De-oldify this image so that it's colorized and upsampled" },
        { type: "image_url", image_url: { url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/lunch-skyscraper.jpg" } }
      ]
    }
  ]
});

console.log(response.choices[0].message.content);

import { VlmRun } from "vlmrun";
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";

// Define the response schema with Zod
const DeOldifyResponseSchema = z.object({
  url: z.string().describe("URL of the colorized image")
});

// Initialize the VLMRun client
const client = new VlmRun({
  apiKey: "<VLMRUN_API_KEY>",
  baseURL: "https://api.vlm.run/v1"
});

// Colorize image with structured output
const response = await client.agent.completions.create({
  model: "vlmrun-orion-1:auto",
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "De-oldify this image so that it's colorized and upsampled" },
        { type: "image_url", image_url: { url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.agent/lunch-skyscraper.jpg" } }
      ]
    }
  ],
  response_format: {
    type: "json_schema",
    schema: zodToJsonSchema(DeOldifyResponseSchema)
  }
});

const result = DeOldifyResponseSchema.parse(JSON.parse(response.choices[0].message.content));

FAQ

What image formats are supported for editing?

JPEG/JPG: Most common format with excellent compatibility
PNG: Lossless format with transparency support
TIFF: High-quality format for professional editing
WebP: Modern format with superior compression
BMP: Uncompressed bitmap format
Quality Preservation: Maintains original image quality in all transformations

What are the best practices for image cropping?

Rule of Thirds: Align subjects with intersection points for better composition
Aspect Ratio: Maintain consistent aspect ratios for professional results
Subject Focus: Keep the main subject centered or following composition rules
Background Removal: Remove distracting elements while preserving context

How accurate is the super-resolution enhancement?

AI-Powered: Uses advanced neural networks for realistic detail generation
Multiple Scales: Supports 2x, 4x, and 8x upscaling with quality preservation
Detail Enhancement: Intelligently adds realistic textures and patterns
Quality Metrics: Provides confidence scores for enhancement quality

How realistic is the de-oldify colorization?

Historical Accuracy: Uses context-aware AI to suggest period-appropriate colors
Natural Colors: Generates realistic skin tones, clothing, and environmental colors
Confidence Scoring: Provides confidence levels for color accuracy
Region Analysis: Identifies and colors different regions with appropriate palettes

​1. Image Cropping

​2. Image Rotation

​3. Super-Resolution Enhancement

​4. De-Oldify (Colorization)

​FAQ

1. Image Cropping

2. Image Rotation

3. Super-Resolution Enhancement

4. De-Oldify (Colorization)

FAQ