Define custom schemas for visual extraction purposes.
Navigate over to the schema generator on our dashboard to build custom schemas from your own data.
In addition to the pre-defined domains, vlm-1
also supports custom schemas that allows you to define your own schema for a specific domain or use-case. This feature allows you to extract structured data that conforms to your specific needs and requirements, while still leveraging all the vision-based reasoning capabilities of vlm-1
(see Capabilities section for more details).
Custom schemas define the structure and validation rules for the data you want to extract from visual content. For example, you can use Pydantic or Zod models to specify exactly which fields you need, their types, and validation rules - passing the schema to the API will ensure our VLM will extract the data in exactly the format you defined.
Type-based data validation for LLMs have been popularized by tools like Instructor and LangChain, however we take it a step further by instrumenting new capabilities on top of your schemas such as visual grounding, confidence scores, GQL querying and much more.
VLM Run has first-class support for Pydantic and Zod, which allows you to define your schema using rich, strongly-typed Pydantic models. Here’s an example of a custom schema for classifying and captioning images:
Once you have defined your custom schema, you can use it with the VLM Run API to extract structured data that conforms to this schema. The extracted data will be validated against the schema you defined.
Here’s how to use a custom schema with the Python SDK:
Since we’ve defined the schema using Pydantic, you can validate and use the extracted data as a strongly-typed object:
If you’re interested in building your own schema for a specific domain or use-case, take a look at our schema best practices guide.
Head over to our Image -> JSON to start building your own document processing pipeline with VLM Run. Sign-up for access on our platform.
Define custom schemas for visual extraction purposes.
Navigate over to the schema generator on our dashboard to build custom schemas from your own data.
In addition to the pre-defined domains, vlm-1
also supports custom schemas that allows you to define your own schema for a specific domain or use-case. This feature allows you to extract structured data that conforms to your specific needs and requirements, while still leveraging all the vision-based reasoning capabilities of vlm-1
(see Capabilities section for more details).
Custom schemas define the structure and validation rules for the data you want to extract from visual content. For example, you can use Pydantic or Zod models to specify exactly which fields you need, their types, and validation rules - passing the schema to the API will ensure our VLM will extract the data in exactly the format you defined.
Type-based data validation for LLMs have been popularized by tools like Instructor and LangChain, however we take it a step further by instrumenting new capabilities on top of your schemas such as visual grounding, confidence scores, GQL querying and much more.
VLM Run has first-class support for Pydantic and Zod, which allows you to define your schema using rich, strongly-typed Pydantic models. Here’s an example of a custom schema for classifying and captioning images:
Once you have defined your custom schema, you can use it with the VLM Run API to extract structured data that conforms to this schema. The extracted data will be validated against the schema you defined.
Here’s how to use a custom schema with the Python SDK:
Since we’ve defined the schema using Pydantic, you can validate and use the extracted data as a strongly-typed object:
If you’re interested in building your own schema for a specific domain or use-case, take a look at our schema best practices guide.
Head over to our Image -> JSON to start building your own document processing pipeline with VLM Run. Sign-up for access on our platform.