Client Reference

This guide provides detailed examples for using the VLM Run client, organized from basic to advanced usage.

Basic Client Setup

Initialization

from vlmrun.client import VLMRun

# Using environment variable (recommended)
client = VLMRun()

# Or with explicit API key
client = VLMRun(api_key="your-api-key")

# With custom configuration
client = VLMRun(
    api_key="your-api-key",
    base_url="https://custom-endpoint.com/v1",
    timeout=60.0,
    max_retries=3
)

For production use, set your API key in the VLMRUN_API_KEY environment variable rather than hardcoding it.

Configuration Options

Parameter	Type	Default	Description
`api_key`	`str`	`None`	Your API key (falls back to `VLMRUN_API_KEY` env var)
`base_url`	`str`	`"https://api.vlm.run/v1"`	API endpoint URL
`timeout`	`float`	`120.0`	Request timeout in seconds
`max_retries`	`int`	`5`	Maximum retry attempts for failed requests

Media Processing

Image Processing

# Process an image file
from PIL import Image
image = Image.open("invoice.jpg")

response = client.image.generate(
    images=[image],
    domain="document.invoice"
)

# Process from URL
response = client.image.generate(
    urls=["https://example.com/invoice.jpg"],
    domain="document.invoice"
)

Document Processing

# Process a document file
response = client.document.generate(
    file="document.pdf",
    domain="document.receipt"
)

# Process from URL
response = client.document.generate(
    url="https://example.com/document.pdf",
    domain="document.receipt"
)

Audio Processing

# Process an audio file
response = client.audio.generate(
    file="recording.mp3",
    domain="audio.transcription"
)

# Process from URL
response = client.audio.generate(
    url="https://example.com/recording.mp3",
    domain="audio.transcription"
)

Video Processing

# Process a video file
response = client.video.generate(
    file="clip.mp4",
    domain="video.transcription"
)

Working with Predictions

Retrieving Predictions

# Get a specific prediction by ID
prediction = client.predictions.get("pred_abc123")

# List recent predictions (paginated)
predictions = client.predictions.list(limit=10)
for pred in predictions:
    print(f"ID: {pred.id}, Status: {pred.status}")

Waiting for Completion

# Initial prediction request
response = client.document.generate(
    file="large-document.pdf",
    domain="document.receipt"
)

# Wait for completion if still processing
if response.status != "completed":
    response = client.predictions.wait(
        response.id,
        timeout=300,  # 5 minutes max
        sleep=2       # Check every 2 seconds
    )

# Or with a callback function
def on_complete(prediction):
    print(f"Prediction {prediction.id} completed!")

client.predictions.wait(response.id, callback=on_complete)

Working with Response Data

# Check prediction status
if prediction.status == "completed":
    # Access structured data
    result = prediction.response

    # Access metadata
    print(f"Created: {prediction.created_at}")
    print(f"Completed: {prediction.completed_at}")

    # Access usage information
    print(f"Elements processed: {prediction.usage.elements_processed}")
    print(f"Credits used: {prediction.usage.credits_used}")
elif prediction.status == "failed":
    print("Prediction failed")
else:
    print(f"Status: {prediction.status}")

File Management

Uploading Files

# Upload from a file path
file = client.files.upload("document.pdf")

# Upload with purpose tag
file = client.files.upload(
    file="image.jpg",
    purpose="document-processing"
)

# Upload from an open file handle
with open("document.pdf", "rb") as f:
    file = client.files.upload(file=f, filename="document.pdf")

Managing Files

# List files (paginated)
files = client.files.list(limit=20)
for file in files:
    print(f"ID: {file.id}, Name: {file.filename}")

# Get file details
file = client.files.get("file_abc123")
print(f"URL: {file.url}")
print(f"Size: {file.bytes} bytes")

# Delete a file
client.files.delete("file_abc123")

Domain and Schema Management

Working with Domains

# List available domains
domains = client.hub.list_domains()
for domain in domains:
    print(f"Domain: {domain.name}, Version: {domain.version}")

# Filter by type
doc_domains = [d for d in domains if d.name.startswith("document.")]

Working with Schemas

# Get schema information for a domain
schema = client.hub.get_schema("document.invoice")

# Get version information
print(f"Schema version: {schema.schema_version}")

# Get JSON schema definition
json_schema = schema.model_json_schema()

# Get Pydantic model for a domain
InvoiceModel = client.hub.get_pydantic_model("document.invoice")

# Create a model instance
invoice = InvoiceModel(
    invoice_number="INV-123",
    total_amount=1250.00
)

Custom Schemas

Defining Custom Schemas

from pydantic import BaseModel, Field

# Define a custom schema with Pydantic
class ProductSchema(BaseModel):
    name: str = Field(..., description="Product name")
    price: float = Field(..., description="Product price")
    sku: str = Field(..., description="Stock keeping unit")
    in_stock: bool = Field(..., description="Whether item is in stock")

# Use in generation request
response = client.image.generate(
    images=[product_image],
    domain="image.product",
    config={"json_schema": ProductSchema.model_json_schema()}
)

Using Auto-casting

# With autocast=True, the response is converted to a Pydantic model
response = client.document.generate(
    file="invoice.pdf",
    domain="document.invoice",
    autocast=True
)

# Now response.response is a domain-specific model
invoice = response.response
total_with_tax = invoice.total_amount * 1.1  # Type-safe operation

Common Workflows

Upload and Process

# 1. Upload file
file_response = client.files.upload("invoice.jpg")

# 2. Process the file
prediction = client.image.generate(
    urls=[file_response.url],
    domain="document.invoice"
)

# 3. Wait if needed
if prediction.status != "completed":
    prediction = client.predictions.wait(prediction.id)

# 4. Process results
invoice_data = prediction.response
print(f"Invoice number: {invoice_data.invoice_number}")

Batch Processing

import glob
from pathlib import Path

# 1. Get all files to process
files = glob.glob("./invoices/*.pdf")
results = []

# 2. Process each file
for file_path in files:
    response = client.document.generate(
        file=file_path,
        domain="document.invoice",
        batch=True  # Enable batch processing
    )
    results.append(response.id)

# 3. Wait for all results
completed_results = []
for prediction_id in results:
    result = client.predictions.wait(prediction_id)
    completed_results.append(result)

Error Handling

Handling Common Errors

from vlmrun.client.exceptions import (
    ApiError,
    AuthenticationError,
    RateLimitError,
    ServerError,
    InvalidRequestError
)

try:
    response = client.image.generate(
        images=[image],
        domain="document.invoice"
    )
except AuthenticationError:
    print("Authentication failed. Check your API key.")
except RateLimitError as e:
    print(f"Rate limit exceeded. Try again in {e.retry_after} seconds.")
except ServerError:
    print("Server error. Please try again later.")
except InvalidRequestError as e:
    print(f"Invalid request: {e.message}")
except ApiError as e:
    print(f"API error ({e.status_code}): {e.message}")

Implementing Retries

import time
from vlmrun.client.exceptions import RateLimitError, ServerError

def process_with_retry(image_path, max_retries=5):
    for attempt in range(max_retries):
        try:
            return client.image.generate(
                images=[image_path],
                domain="document.invoice"
            )
        except (RateLimitError, ServerError) as e:
            if attempt == max_retries - 1:
                raise

            # Exponential backoff
            wait_time = 2 ** attempt
            print(f"Retrying in {wait_time}s (attempt {attempt+1}/{max_retries})")
            time.sleep(wait_time)

Advanced Features

Custom Timeouts

# Global timeout setting
client = VLMRun(timeout=180.0)  # 3 minutes default

# Component-specific timeouts
client.document._requestor._timeout = 300.0  # 5 minutes for documents
client.video._requestor._timeout = 600.0     # 10 minutes for videos

Request Metadata

from vlmrun.client.types import RequestMetadata

# Add metadata to track requests
response = client.image.generate(
    images=[image],
    domain="document.invoice",
    metadata=RequestMetadata(
        user_id="user123",
        session_id="session456",
        tags=["invoice", "processing"]
    )
)

Best Practices

Client Lifecycle

Create a single client instance and reuse it across your application:

# At application startup
client = VLMRun()

# Reuse throughout your application
def process_image(image_path):
    return client.image.generate(images=[image_path], domain="document.invoice")

def process_document(document_path):
    return client.document.generate(file=document_path, domain="document.invoice")

API Key Management

Keep your API key secure:

Use environment variables instead of hardcoding
Use a secrets manager for production applications
Regularly rotate API keys for security

Resource Cleanup

Clean up resources when they’re no longer needed:

# Upload temporary file
file_response = client.files.upload("temp.jpg")

# Use the file
prediction = client.image.generate(urls=[file_response.url])

# Delete when done
client.files.delete(file_response.id)

Performance Optimization

For high-volume applications:

Reuse the client instance
Use batch processing for multiple files
Implement exponential backoff for retries
Use async processing for non-blocking operations

Python SDK

Documentation Index

​Client Reference

​Basic Client Setup

​Initialization

​Configuration Options

​Media Processing

​Image Processing

​Document Processing

​Audio Processing

​Video Processing

​Working with Predictions

​Retrieving Predictions

​Waiting for Completion

​Working with Response Data

​File Management

​Uploading Files

​Managing Files

​Domain and Schema Management

​Working with Domains

​Working with Schemas

​Custom Schemas

​Defining Custom Schemas

​Using Auto-casting

​Common Workflows

​Upload and Process

​Batch Processing

​Error Handling

​Handling Common Errors

​Implementing Retries

​Advanced Features

​Custom Timeouts

​Request Metadata

​Best Practices

​Client Lifecycle

​API Key Management

​Resource Cleanup

​Performance Optimization

Client Reference

Basic Client Setup

Initialization

Configuration Options

Media Processing

Image Processing

Document Processing

Audio Processing

Video Processing

Working with Predictions

Retrieving Predictions

Waiting for Completion

Working with Response Data

File Management

Uploading Files

Managing Files

Domain and Schema Management

Working with Domains

Working with Schemas

Custom Schemas

Defining Custom Schemas

Using Auto-casting

Common Workflows

Upload and Process

Batch Processing

Error Handling

Handling Common Errors

Implementing Retries

Advanced Features

Custom Timeouts

Request Metadata

Best Practices

Client Lifecycle

API Key Management

Resource Cleanup

Performance Optimization