Client Reference

Basic Client Setup

Initialization

from vlmrun.client import VLMRun

# Using environment variable (recommended)
client = VLMRun()

# Or with explicit API key
client = VLMRun(api_key="your-api-key")

# With custom configuration
client = VLMRun(
    api_key="your-api-key",
    base_url="https://custom-endpoint.com/v1",
    timeout=60.0,
    max_retries=3
)

In production, set VLMRUN_API_KEY instead of hardcoding the API key.

Configuration Options

Parameter	Type	Default	Description
`api_key`	`str`	`None`	Your API key (falls back to `VLMRUN_API_KEY` env var)
`base_url`	`str`	`"https://api.vlm.run/v1"`	API endpoint URL (falls back to `VLMRUN_BASE_URL` env var)
`timeout`	`float`	`120.0`	Request timeout in seconds
`max_retries`	`int`	`5`	Maximum retry attempts for failed requests
`require_api_key`	`bool`	`True`	Raise `ConfigurationError` without an API key, and validate the key with a `/health` call. Set `False` for deployments that do not authenticate

Media Processing

Image Processing

# Process an image file
from PIL import Image
image = Image.open("invoice.jpg")

response = client.image.generate(
    images=[image],
    domain="document.invoice"
)

# Process from URL
response = client.image.generate(
    urls=["https://example.com/invoice.jpg"],
    domain="document.invoice"
)

Document Processing

# Process a document file
response = client.document.generate(
    file="document.pdf",
    domain="document.receipt"
)

# Process from URL
response = client.document.generate(
    url="https://example.com/document.pdf",
    domain="document.receipt"
)

Audio Processing

# Process an audio file
response = client.audio.generate(
    file="recording.mp3",
    domain="audio.transcription"
)

# Process from URL
response = client.audio.generate(
    url="https://example.com/recording.mp3",
    domain="audio.transcription"
)

Video Processing

# Process a video file
response = client.video.generate(
    file="clip.mp4",
    domain="video.transcription"
)

Working with Predictions

Retrieving Predictions

# Get a specific prediction by ID
prediction = client.predictions.get("pred_abc123")

# List recent predictions (paginated)
predictions = client.predictions.list(limit=10)
for pred in predictions:
    print(f"ID: {pred.id}, Status: {pred.status}")

Waiting for Completion

# Initial prediction request
response = client.document.generate(
    file="large-document.pdf",
    domain="document.receipt"
)

# Wait for completion if still processing
if response.status != "completed":
    response = client.predictions.wait(
        response.id,
        timeout=300,  # 5 minutes max
        sleep=2       # Check every 2 seconds
    )

# Or with a callback function
def on_complete(prediction):
    print(f"Prediction {prediction.id} completed!")

client.predictions.wait(response.id, callback=on_complete)

Working with Response Data

# Check prediction status
if prediction.status == "completed":
    # Access structured data
    result = prediction.response

    # Access metadata
    print(f"Created: {prediction.created_at}")
    print(f"Completed: {prediction.completed_at}")

    # Access usage information
    print(f"Elements processed: {prediction.usage.elements_processed}")
    print(f"Credits used: {prediction.usage.credits_used}")
elif prediction.status == "failed":
    print("Prediction failed")
else:
    print(f"Status: {prediction.status}")

File Management

Uploading Files

# Upload from a file path
file = client.files.upload("document.pdf")

# Upload with purpose tag
file = client.files.upload(
    file="image.jpg",
    purpose="document-processing"
)

# Upload from an open file handle
with open("document.pdf", "rb") as f:
    file = client.files.upload(file=f, filename="document.pdf")

Managing Files

# List files (paginated)
files = client.files.list(limit=20)
for file in files:
    print(f"ID: {file.id}, Name: {file.filename}")

# Get file details
file = client.files.get("file_abc123")
print(f"ID: {file.id}")
print(f"Size: {file.bytes} bytes")

# Delete a file
client.files.delete("file_abc123")

Domain and Schema Management

Working with Domains

# List available domains
domains = client.hub.list_domains()
for domain in domains:
    print(f"Domain: {domain.domain}")

# Filter by type
doc_domains = [d for d in domains if d.name.startswith("document.")]

Working with Schemas

# Get schema information for a domain
schema = client.hub.get_schema("document.invoice")

# Get version information
print(f"Schema version: {schema.schema_version}")

# Get JSON schema definition
json_schema = schema.model_json_schema()

# Get Pydantic model for a domain
InvoiceModel = client.hub.get_pydantic_model("document.invoice")

# Create a model instance
invoice = InvoiceModel(
    invoice_number="INV-123",
    total_amount=1250.00
)

Custom Schemas

Defining Custom Schemas

from pydantic import BaseModel, Field

# Define a custom schema with Pydantic
class ProductSchema(BaseModel):
    name: str = Field(..., description="Product name")
    price: float = Field(..., description="Product price")
    sku: str = Field(..., description="Stock keeping unit")
    in_stock: bool = Field(..., description="Whether item is in stock")

# Use in generation request
response = client.image.generate(
    images=[product_image],
    domain="image.product",
    config={"json_schema": ProductSchema.model_json_schema()}
)

Using Auto-casting

# With autocast=True, the response is converted to a Pydantic model
response = client.document.generate(
    file="invoice.pdf",
    domain="document.invoice",
    autocast=True
)

# Now response.response is a domain-specific model
invoice = response.response
total_with_tax = invoice.total_amount * 1.1  # Type-safe operation

Common Workflows

Upload and Process

# 1. Upload file
file_response = client.files.upload("invoice.jpg")

# 2. Process the file
prediction = client.image.generate(
    urls=[file_response.url],
    domain="document.invoice"
)

# 3. Wait if needed
if prediction.status != "completed":
    prediction = client.predictions.wait(prediction.id)

# 4. Process results
invoice_data = prediction.response
print(f"Invoice number: {invoice_data.invoice_number}")

Batch Processing

import glob
from pathlib import Path

# 1. Get all files to process
files = glob.glob("./invoices/*.pdf")
results = []

# 2. Process each file
for file_path in files:
    response = client.document.generate(
        file=file_path,
        domain="document.invoice",
        batch=True  # Enable batch processing
    )
    results.append(response.id)

# 3. Wait for all results
completed_results = []
for prediction_id in results:
    result = client.predictions.wait(prediction_id)
    completed_results.append(result)

Error Handling

Handling Common Errors

from vlmrun.client.exceptions import (
    APIError,
    AuthenticationError,
    RateLimitError,
    ServerError,
    ValidationError
)

client = VLMRun()

try:
    response = client.image.generate(
        images=[image],
        domain="document.invoice"
    )
except AuthenticationError:
    print("Authentication failed. Check your API key.")
except RateLimitError as e:
    print(f"Rate limit exceeded. Try again in {e.retry_after} seconds.")
except ServerError:
    print("Server error. Please try again later.")
except InvalidRequestError as e:
    print(f"Invalid request: {e.message}")
except ApiError as e:
    print(f"API error ({e.status_code}): {e.message}")

Implementing Retries

import time
from vlmrun.client.exceptions import RateLimitError, ServerError

def process_with_retry(image_path, max_retries=5):
    for attempt in range(max_retries):
        try:
            return client.image.generate(
                images=[image_path],
                domain="document.invoice"
            )
        except (RateLimitError, ServerError) as e:
            if attempt == max_retries - 1:
                raise

            # Exponential backoff
            wait_time = 2 ** attempt
            print(f"Retrying in {wait_time}s (attempt {attempt+1}/{max_retries})")
            time.sleep(wait_time)

Advanced Features

Custom Timeouts

# Global timeout setting
client = VLMRun(timeout=180.0)  # 3 minutes default

# Component-specific timeouts
client.document._requestor._timeout = 300.0  # 5 minutes for documents
client.video._requestor._timeout = 600.0     # 10 minutes for videos

Request Metadata

from vlmrun.client.types import RequestMetadata

# Add metadata to track requests
response = client.image.generate(
    images=[image],
    domain="document.invoice",
    metadata=RequestMetadata(
        user_id="user123",
        session_id="session456",
        tags=["invoice", "processing"]
    )
)

Best Practices

Client Lifecycle

Create one VLMRun client and reuse it.

# At application startup
client = VLMRun()

# Reuse throughout your application
def process_image(image_path):
    return client.image.generate(images=[image_path], domain="document.invoice")

def process_document(document_path):
    return client.document.generate(file=document_path, domain="document.invoice")

API Key Management

Set VLMRUN_API_KEY instead of hardcoding the key.
Use a secrets manager in production.
Rotate API keys.

Resource Cleanup

# Upload temporary file
file_response = client.files.upload("temp.jpg")

# Use the file
prediction = client.image.generate(urls=[file_response.url])

# Delete when done
client.files.delete(file_response.id)

Performance Optimization

Reuse the client instance.
Use batch processing for multiple files.
Implement exponential backoff for retries.
Use async processing for non-blocking operations.

​Client Reference

​Basic Client Setup

​Initialization

​Configuration Options

​Media Processing

​Image Processing

​Document Processing

​Audio Processing

​Video Processing

​Working with Predictions

​Retrieving Predictions

​Waiting for Completion

​Working with Response Data

​File Management

​Uploading Files

​Managing Files

​Domain and Schema Management

​Working with Domains

​Working with Schemas

​Custom Schemas

​Defining Custom Schemas

​Using Auto-casting

​Common Workflows

​Upload and Process

​Batch Processing

​Error Handling

​Handling Common Errors

​Implementing Retries

​Advanced Features

​Custom Timeouts

​Request Metadata

​Best Practices

​Client Lifecycle

​API Key Management

​Resource Cleanup

​Performance Optimization

Client Reference

Basic Client Setup

Initialization

Configuration Options

Media Processing

Image Processing

Document Processing

Audio Processing

Video Processing

Working with Predictions

Retrieving Predictions

Waiting for Completion

Working with Response Data

File Management

Uploading Files

Managing Files

Domain and Schema Management

Working with Domains

Working with Schemas

Custom Schemas

Defining Custom Schemas

Using Auto-casting

Common Workflows

Upload and Process

Batch Processing

Error Handling

Handling Common Errors

Implementing Retries

Advanced Features

Custom Timeouts

Request Metadata

Best Practices

Client Lifecycle

API Key Management

Resource Cleanup

Performance Optimization