> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vlm.run/llms.txt
> Use this file to discover all available pages before exploring further.

# Caption & Tag

> Generate detailed captions and tags for images using advanced vision models.

Generate comprehensive, contextual captions for images using state-of-the-art vision-language models. Perfect for accessibility, content management, and automated image analysis workflows.

<Frame caption="Example image to be captioned.">
  <img className="block dark:hidden" src="https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.caption/car.jpg" alt="Image captioning example showing detailed scene description" style={{ width: '80%', border: '1px solid #ddd', borderRadius: '4px' }} />
</Frame>

## Example Response

This is an example of the response from the `Chat Completions API` example (using the image shown above):

<CodeGroup>
  ```mdx Chat Completions wrap theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  A classic, light turquoise Volkswagen Beetle with chrome accents is parked on a cobblestone street, set against a warm yellow stucco wall with rustic brown wooden doors and windows.
  Tags: car, volkswagen, beetle, street, cobblestone, wooden, doors, windows
  ```

  ```json Structured Outputs wrap theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  {
    "caption": "A classic, light turquoise Volkswagen Beetle with chrome accents is parked on a cobblestone street, set against a warm yellow stucco wall with rustic brown wooden doors and windows.",
    "tags": ["car", "volkswagen", "beetle", "street", "cobblestone", "wooden", "doors", "windows"]
  }
  ```
</CodeGroup>

## Usage Example

<Tip>
  For best results, we recommend using the [Structured Outputs API](/agents/structured-responses) to get responses in a structured and validated data format.
</Tip>

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  from vlmrun.client import VLMRun

  # Initialize the VLMRun client
  client = VLMRun(api_key="<VLMRUN_API_KEY>")

  # Caption the image
  response = client.agent.completions.create(
      model="vlmrun-orion-1:auto",
      messages=[
          {"role": "user",
          "content": [
            {"type": "text", "text": "Generate a detailed caption for this image"},
            {"type": "image_url", "image_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.caption/car.jpg", "detail": "auto"}}
          ]
        }
      ],
  )

  # Print the response
  print(response.choices[0].message.content)
  # >> "A classic, light turquoise Volkswagen Beetle..."
  ```

  ```python Python - Structured Outputs theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  from vlmrun.client import VLMRun
  from pydantic import BaseModel, Field

  # Define the response schema
  class ImageCaption(BaseModel):
      caption: str = Field(..., description="Detailed caption of the scene")
      tags: list[str] = Field(..., description="Tags that describe the image")

  # Initialize the VLMRun client
  client = VLMRun(api_key="<VLMRUN_API_KEY>")

  # Caption the image with structured output
  response = client.agent.completions.create(
      model="vlmrun-orion-1:auto",
      messages=[
          {"role": "user",
          "content": [
            {"type": "text", "text": "Generate a detailed caption for this image"},
            {"type": "image_url", "image_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.caption/car.jpg", "detail": "auto"}}
          ]
        }
      ],
      response_format={"type": "json_schema", "schema": ImageCaption.model_json_schema()}
  )

  # Validate the response
  result = ImageCaption.model_validate_json(response.choices[0].message.content)
  # >>> ImageCaption(caption="...", tags=[...])
  ```

  ```typescript Node.js theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  import { VlmRun } from "vlmrun";

  // Initialize the VLMRun client
  const client = new VlmRun({
    apiKey: "<VLMRUN_API_KEY>",
    baseURL: "https://api.vlm.run/v1"
  });

  // Caption the image
  const response = await client.agent.completions.create({
    model: "vlmrun-orion-1:auto",
    messages: [
      {
        role: "user",
        content: [
          { type: "text", text: "Generate a detailed caption for this image" },
          { type: "image_url", image_url: { url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.caption/car.jpg", detail: "auto" } }
        ]
      }
    ]
  });

  console.log(response.choices[0].message.content);
  ```

  ```typescript Node.js - Structured Outputs [expandable] theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  import { VlmRun } from "vlmrun";
  import { z } from "zod";
  import { zodToJsonSchema } from "zod-to-json-schema";

  // Define the response schema with Zod
  const ImageCaptionSchema = z.object({
    caption: z.string().describe("Detailed caption of the scene"),
    tags: z.array(z.string()).describe("Tags that describe the image")
  });

  // Initialize the VLMRun client
  const client = new VlmRun({
    apiKey: "<VLMRUN_API_KEY>",
    baseURL: "https://api.vlm.run/v1"
  });

  // Caption the image with structured output
  const response = await client.agent.completions.create({
    model: "vlmrun-orion-1:auto",
    messages: [
      {
        role: "user",
        content: [
          { type: "text", text: "Generate a detailed caption for this image" },
          { type: "image_url", image_url: { url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.caption/car.jpg", detail: "auto" } }
        ]
      }
    ],
    response_format: {
      type: "json_schema",
      schema: zodToJsonSchema(ImageCaptionSchema)
    }
  });

  const result = ImageCaptionSchema.parse(JSON.parse(response.choices[0].message.content));
  ```
</CodeGroup>

## FAQ

<AccordionGroup>
  <Accordion title="How do I ask the model for more detailed captions?" icon="chat">
    You can ask simply ask for a more detailed caption by providing a more detailed prompt. In most cases, you can provide the number of words you want the caption to be, and the model will generate a more detailed caption.
  </Accordion>

  <Accordion title="What tags are supported?" icon="tags">
    * **Common Objects**: person, car, truck, bus, bicycle, motorcycle
    * **Scenes**: street, building, park, forest, beach, etc.
    * **Time-of-Day**: morning, afternoon, evening, night
    * **Weather**: sunny, cloudy, rainy, snowing, etc.
  </Accordion>

  <Accordion title="What format do the tags come in?" icon="crop">
    The tags come in the format of a list of strings.
  </Accordion>
</AccordionGroup>