Structured Responses

Domain schemas are in the hub.

vlm-1 returns JSON that matches a domain schema, including nested objects and lists, instead of a free-form caption.

Extract Structured Data

Pass a pre-defined domain to document.generate. The response is validated against that schema.

from pathlib import Path
from vlmrun.client import VLMRun
from vlmrun.client.types import PredictionResponse

# Initialize the client
client = VLMRun(api_key="<your-api-key>")

# Process the file or image with the predefined schema
path: Path = Path("path/to/invoice.pdf")
prediction: PredictionResponse = client.document.generate(
    file=path,
    domain="document.invoice",
)
response_dict = prediction.response.model_dump()

import { VlmRun } from "vlmrun";

// Initialize the client
const client = new VlmRun({ apiKey: '<your-api-key>' });

// Process the file or image with the predefined schema
const fileResponse = await client.files.upload(filePath: "<path/to/invoice.pdf>")
const prediction = await client.document.generate({
  fileId: fileResponse.file_id,
  domain: "document.invoice"
});
console.log(prediction);

Illustrative Examples

Response

{
  "invoice_id": "79BBD516-0005",
  "period_start": null,
  "period_end": null,
  "invoice_issue_date": "2024-01-10",
  "invoice_due_date": "2024-02-09",
  "order_id": null,
  "customer_id": null,
  "issuer": "Typographic",
  "issuer_address": {
      "street": "1 Grand Canal St Lower",
      "city": "Dublin",
      "state": "Co. Dublin",
      "postal_code": "D04 Y7R5",
      "country": "Ireland"
  },
  "customer": "French Customer",
  "customer_email": null,
  "customer_phone": "+33 1 23 45 67 89",
  "customer_billing_address": {
      "street": "5 Avenue Anatole France",
      "city": "Champ de Mars",
      "state": "Paris",
      "postal_code": "75007",
      "country": "France"
  },
  "customer_shipping_address": null,
  "items": [
      {
      "description": "Line Item 1",
      "quantity": 1,
      "currency": "EUR",
      "unit_price": 10.0,
      "total_price": 10.0
      },
      {
      "description": "Line Item 2",
      "quantity": 1,
      "currency": "EUR",
      "unit_price": 5.0,
      "total_price": 5.0
      }
  ],
  "subtotal": 15.0,
  "tax": 0.0,
  "total": 15.0,
  "currency": "EUR",
  "notes": "[1] Tax to be paid on reverse charge basis",
  "others": {
      "due_amount": 15.0,
      "vat_number": "FRAB123456789",
      "support_email": "support@typographic.com",
      "contact_phone": "+353123456789"
  }
}

Custom Schemas

vlm-1 supports custom schemas and pre-defined schemas.

Try our Document -> JSON API today

Open Document -> JSON on VLM Run. Sign up on the platform.

Overview

SDKs

CLI

CLI - Gateway

CLI - Orion

Structured Responses

Structured Responses

Extract Structured Data

Illustrative Examples

Custom Schemas

Try our Document -> JSON API today

Structured Responses

​Extract Structured Data

​Illustrative Examples

​Custom Schemas

​Try our Document -> JSON API today

Extract Structured Data

Illustrative Examples

Custom Schemas

Try our Document -> JSON API today