Supported Domains

The VLM Run Hub is a collection of pre-defined domains and schemas for structured data extraction.

Document Domains

Domain	Allowed Inputs	Description
document.bank-statement	documentimage	Extract account details, balances, and transactions.
document.classification	documentimage	Classify a document into one or more categories from content, visual features, and metadata.
document.invoice	documentimage	Extract invoice metadata, customer details, line items, and totals.
document.markdown	documentimage	Convert pages to content descriptions, including tables and charts.
document.q-and-a	documentimage	Convert pages to content descriptions, including tables and charts.
document.receipt	documentimage	Extract transaction details, merchant information, and totals.
document.resume	documentimage	Extract contact details, education, work experience, skills, and other sections.
document.us-drivers-license	documentimage	Extract name, address, date of birth, and license details.
document.utility-bill	documentimage	Extract account details, billing period, charges, and payment information.

Domain	Allowed Inputs	Description
image.classification	image	Classify the image into one of these categories: [category1, category2, category3, …]
image.caption	image	Extract the caption from the image.
image.tv-news	image	Extract the TV news segment from the image.
image.q-and-a	image	Answer questions about an image.

Domain	Allowed Inputs	Description
audio.transcription	audio	Real-time audio analytics: trending topics, public sentiment, and influential quotes.

Three types:

Whole-video Summary (summary): Summarize the whole video.
Segmented Summary (segmented-summary): Summarize the video as segments (key moments, scenes, highlights). Prompts and cues guide the split. Set the segment count (for example, “Find 5 key moments in this video where the CEO mentions “AI”).
Segmented Analysis (segmented-analysis): Extract fields per segment with a custom segment model (json_schema). Segments are detected from audio and visual cues (silence, a new scene).

Domain	Allowed Inputs	Type	Description
video.transcription	video	`segmented-analysis`	Transcribe the video: timestamps, visual descriptions, audio transcriptions, summaries, and topics.
video.transcription-summary	video	`segmented-summary`	Split the video into 5-minute segments. Each segment includes timestamps, visual descriptions, audio transcriptions, summaries, and topics.
video.product-demo-summary	video	`segmented-summary`	Split the video into 5-minute segments. Each segment includes timestamps, visual descriptions, audio transcriptions, summaries, and topics.
video.conferencing-summary	video	`segmented-summary`	Extract segments, topics, presenters, and key events from conferencing videos.
video.podcast-summary	video	`segmented-summary`	Extract segments, topics, presenters, and key events from podcast videos.
video.summary	video	`summary`	Summarize the video in 2–3 sentences. Return at most 5 topics. Use the provided topics enum when possible.
video.dashcam-analytics	imagevideo	`summary`	Identify events, objects, traffic conditions, incidents, and environmental factors in dashcam footage.

Domain	Allowed Inputs	Description
aerospace.remote-sensing	imagevideo	Identify geographical features, infrastructure, and environmental elements in aerial imagery.
healthcare.patient-consent	document	Fill the consent model. Leave a field blank when unsure.
healthcare.patient-identification	documentimage	Fill the identification model. Leave a field blank when unsure.
healthcare.patient-insurance-card	documentimage	Fill the insurance card model. Leave a field blank when unsure.
healthcare.patient-intake	document	Fill the intake form model. Leave a field blank when unsure.
healthcare.patient-medical-history	document	Fill the medical history model. Leave a field blank when unsure.
healthcare.patient-referral	document	Fill the patient referral record model. Leave a field blank when unsure.
retail.ecommerce-product-caption	image	Extract a visual description, product details, and delivery information.
retail.product-catalog	image	Real-time product catalog analytics: trending topics, public sentiment, and influential quotes.