> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vlm.run/llms.txt
> Use this file to discover all available pages before exploring further.

# microsoft/florence-2-base-ft

> Multi-task image captioning, detection, and OCR.

Florence-2 multi-task vision model (caption / detection / OCR). Accepts exactly one
`image_url`; no text-only, video, or `document_url` input. Default method: `caption`.
The caption and `ocr` methods emit a Markdown string; the region methods emit a
region container.

## Output by method

| Method                  | Payload kind | `content.object`                          | Payload                                   |
| ----------------------- | ------------ | ----------------------------------------- | ----------------------------------------- |
| `caption` (default)     | markdown     | none                                      | Short image caption                       |
| `detailed_caption`      | markdown     | none                                      | Longer caption                            |
| `more_detailed_caption` | markdown     | none                                      | Longest caption                           |
| `ocr`                   | markdown     | none                                      | All text on the image                     |
| `od`                    | json         | `florence_2.od.regions`                   | Region items with `bbox_xywh` and `label` |
| `dense_region_caption`  | json         | `florence_2.dense_region_caption.regions` | Region items with `bbox_xywh` and `label` |
| `region_proposal`       | json         | `florence_2.region_proposal.regions`      | Region items with `bbox_xywh`, no `label` |
| `ocr_with_region`       | json         | `florence_2.ocr_with_region.regions`      | Region items with `bbox_xywh` and `label` |

Each method maps to one Florence-2 task token. The task token itself
(`<CAPTION>`, `<OD>`, ...) is an implementation detail and is not part of the
reply: a caption is the string, and a region method is the container.

Boxes are normalized 0-1 against the image at `precision` decimals (default 4),
and a region method with no hits returns `"items": []`.

## Request

<CodeGroup>
  ```python Python [expandable] theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  from openai import OpenAI

  client = OpenAI(
      base_url="https://gateway.vlm.run/v1/openai",
      api_key="<VLMRUN_API_KEY>",
  )

  response = client.chat.completions.create(
      model="microsoft/florence-2-base-ft",
      messages=[
          {
              "role": "user",
              "content": [
                  {
                      "type": "image_url",
                      "image_url": {
                          "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.caption/car.jpg"
                      },
                  },
              ],
          }
      ],
      extra_body={"method": "caption"},
  )

  print(response.choices[0].message.content)
  ```

  ```bash cURL [expandable] theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  curl https://gateway.vlm.run/v1/openai/chat/completions \
    -X POST \
    -H "Authorization: Bearer $VLMRUN_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "microsoft/florence-2-base-ft",
      "method": "caption",
      "messages": [
        {
          "role": "user",
          "content": [
            {
              "type": "image_url",
              "image_url": {
                "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.caption/car.jpg"
              }
            }
          ]
        }
      ]
    }'
  ```
</CodeGroup>

## Response

<Tabs>
  <Tab title="Text mode (caption)">
    The caption string alone, with no wrapper:

    ```text theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    A cat sitting on a windowsill.
    ```
  </Tab>

  <Tab title="JSON mode (caption)">
    One image entry; `content` is the string:

    ```json theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    {
      "model": "microsoft/Florence-2-base-ft",
      "method": "caption",
      "data": [
        {
          "object": "image",
          "image_hash": "sha256:...",
          "image_width": 640,
          "image_height": 480,
          "content": "A cat sitting on a windowsill."
        }
      ]
    }
    ```
  </Tab>

  <Tab title="JSON mode (od)">
    For a detection method (`method="od"`), `content` is the region container:

    ```json theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    {
      "model": "microsoft/Florence-2-base-ft",
      "method": "od",
      "data": [
        {
          "object": "image",
          "image_hash": "sha256:...",
          "image_width": 640,
          "image_height": 480,
          "content": {
            "object": "florence_2.od.regions",
            "items": [
              {"bbox_xywh": [0.0531, 0.0417, 0.275, 0.3333], "label": "cat"},
              {"bbox_xywh": [0.0781, 0.125, 0.1094, 0.1458], "label": "dog"}
            ]
          }
        }
      ]
    }
    ```
  </Tab>
</Tabs>