.domain-column {
    width: 240px;
    display: inline-block;
  }

.allowed-inputs-column {
  width: 160px;
  display: inline-block;
}

.description-column {
  width: 100%;
  display: inline-block;
}

.tag {
  padding: 2px 8px;
  border-radius: 12px;
  font-size: 0.9em;
  font-weight: 500;
  display: inline-block;
  margin-right: 8px;
  margin-bottom: 4px;
}

.tag-document {
  background: #FFF3E0;
  color: #E65100;
}

.tag-image {
  background: #E6FFDD;
  color: #438F2A;
}

.tag-video {
  background: #EAE0FF;
  color: #784BCE;
}

.tag-audio {
  background: #DEE9FF;
  color: #4478D7;
}

.card {
  background-color: #f9f9f9;
}

.card-demo {
    color: #4478D7;
    background-color: #dae6ff;
}


VLM Run

Extract JSON from images, videos, and documents with a unified API.

Introduction

Extract JSON from images, videos, and documents with type-safety.

Structured Responses

Pre-built schemas and domain definitions for common data extraction tasks.

Supported Domains

Define custom schemas for visual extraction purposes.

Custom Schemas

Ground extracted data with location (bounding box) coordinates and confidence scores.

Visual Grounding

Ground extracted data with start/end times for audio/video segments and speaker identification.

Temporal Grounding

Support for long-output contexts for domains like audio/video transcription, exceeding 8K token limits.

Long-context Outputs

Query a subset of schema fields to improve efficiency for querying and document ETL.

GraphQL

Extract structured data from long documents and reports.

Parsing Documents

Learn how to classify documents into categories like invoices, bank statements, and utility bills.

Classifying Documents

Learn how to classify images into categories like animals, landscapes, and objects using AI.

Classifying Images

Learn how to generate captions, tags and descriptions for images.

Cataloging Images

Learn how to transcribe and analyze hours-long video content using our Video Transcription API.

Transcribing Videos

Learn how to transcribe and analyze long-form audio.

Transcribing Audio

Rate limits to consider when using the API.

Rate Limits

List of error codes that you may encounter when using the API

Error Codes

Changelog

Health

Get models

List domains

Get the JSON schema for a given domain (document.invoice, document.receipt, etc).

Get schema for domain

Generate structured prediction for the given image.

Image -> JSON

Generate structured prediction for the given document.

Doc -> JSON

Generate structured prediction for the given document using a custom agent.

Doc Agent -> JSON

Generate structured prediction for the given audio file.

Audio -> JSON

Generate structured prediction for the given video file.

Video -> JSON

Get Predictions by ID

Get all predictions uploaded by the user with pagination.

Get Predictions

Upload File

Get File by ID

Get all files uploaded by the user with pagination.

List Files

Create chat completion

List all models

Getting Started with the VLM Run Python SDK

Getting Started

Overview

client

client.files

client.models

client.hub

client.image

client.document

client.audio

client.video

Predictions

CLI Overview

Learn how to install and use the VLM Run Node.js SDK

VLM Run Node.js SDK Client Configuration and Usage

Learn how to upload and manage files with the VLM Run Node.js SDK

Learn how to work with models in the VLM Run Node.js SDK

Learn how to process images with the VLM Run Node.js SDK

Learn how to process documents with the VLM Run Node.js SDK

Learn how to process audio files with the VLM Run Node.js SDK

Extract JSON from images, videos, and documents with custom schemas.

Run VLM-1 with the OpenAI Python SDK with just 2 lines of code change.

OpenAI Compatibility

Run `vlm-1` with the Instructor Python SDK with minimal code changes.

Instructor Compatibility

The VLM Run Hub is a collection of pre-defined domains and schemas for structured data extraction.

Get Started

Capabilities

Guides - Doc AI

Guides - Image AI

Guides - Video/Audio AI

Misc

FAQ