Let’s say we want to classify images into one of three categories: tv, document, or other. You can define a custom schema as follows, and pass it to the json_schema parameter:
Copy
from typing import Literalfrom pydantic import BaseModel, Fieldfrom vlmrun.client.types import GenerationConfigclass ImagePrediction(BaseModel): label: Literal["tv", "document", "other"] = Field(..., title="Class label for the image.") caption: str = Field(..., title="Caption for the image.")# Initialize the clientclient = VLMRun()# Load the image, and process it with the custom schemaimage: Image.Image = Image.open("path/to/image.jpg")response: PredictionResponse = client.image.generate( images=[image], domain="image.classification", config=GenerationConfig( json_schema=ImagePrediction.model_json_schema() ))