Methods & Response Format

Two request fields decide what a VLM Run Gateway model returns:

method selects what the model computes (ocr, detect, parse_layout, chat, …).
response_format selects how that result is serialized (a text rendering, or a JSON object).

They are orthogonal: any method combines with any format, and the two renderings carry the same information. This page is the canonical reference for both. For copy-pasteable per-model examples, see Models.

The `method` field

Most models expose one or more method values. Pass method (and optional method_params) at the top level of the request body, or via extra_body in the OpenAI Python SDK. If you omit method, the model’s default is applied: markdown on zai-org/glm-ocr and rednote-hilab/dots.mocr, ocr on paddleocr/pp-ocrv6, detect on paddlepaddle/pp-doclayoutv3, caption on microsoft/florence-2-base-ft, and chat on the chat VLMs.

Request body
OpenAI SDK

{
  "model": "paddleocr/pp-ocrv6",
  "method": "ocr",
  "messages": [ ... ]
}

from openai import OpenAI

client = OpenAI(...)
response = client.chat.completions.create(
    model="paddleocr/pp-ocrv6",
    messages=[{...}],
    extra_body={"method": "ocr"},
)

Every model’s live methods, default_method, and extra_body_help are also available on GET /v1/openai/models. A method a model does not advertise is a 400.

The `response_format` field

response_format mirrors the OpenAI object, so existing clients work unchanged:

`response_format`	Reply
omitted or `{"type":"text"}`	The text rendering.
`{"type":"json_object"}`	One JSON object.

A JSON response_format is always served non-streamed: the stream flag is ignored, because a single valid JSON object cannot be assembled from SSE deltas. In text mode, stream: true is honored and the stream is byte-identical to the non-streaming reply: the <document> open tag, then one chunk per <page> block, then the close tag. See Streaming.

The `precision` field

precision (int, range 1-8, default 4) sets the number of decimal places on normalized coordinates (bbox_xywh / poly_xy / point_xy) and score. It applies to both formats and is ignored for markdown payloads. The response shape is unchanged.

Text mode

A text reply is a sequence of top-level blocks, one per input medium. Each block body is either a json block or a markdown block, and nothing else. Json is the default. A method renders a markdown block only when it emits genuinely free-form text: markdown, chat, caption, text, and rednote-hilab/dots.mocr’s ocr. Which one applies is a property of the (model, method) pair, and it is echoed on the wire as format=. See Methods by model for the mapping.

Document input

Each PDF is one <document> block wrapping one <page> block per rasterized page. Multiple document_url parts yield one <document> block each, never merged.

<document file_name="invoice.pdf" file_hash="sha256:1b7f…" file_bytes="48213" mimetype="application/pdf" num_pages="3" dpi="150">
<page page_index="0" format="json" page_width="1275" page_height="1650">
{"object": "document.page.blocks", "items": [{"index": 0, "bbox_xywh": [0.0332, 0.0138, 0.1719, 0.0331], "text": "Invoice", "score": 0.998}]}
</page>
<page page_index="1" format="markdown" page_width="1275" page_height="1650">
## Line items

| SKU | Qty |
| --- | --- |
</page>
<page page_index="2" format="json" page_width="1275" page_height="1650" status="error"/>
</document>

Element	Attributes
`<document>`	`file_name`?, `file_hash`?, `file_bytes`?, `mimetype`, `num_pages`, `dpi`, `language`?, in that order. `file_name` is omitted for `data:` URI uploads. `language` is a comma-separated list of ISO-639 codes in confidence order, and it is present only when the read reports one.
`<page>`	`page_index` (zero-based), `format` (`json` or `markdown`), `page_width`, `page_height`, `status`?.

A failed page is self-closing, with status="error" and no body, so page numbering stays intact. format is the same on every page of a request, and it is emitted on every page.

Single image input

The block alone, with no wrapper:

{"object": "pp_ocrv6.ocr.regions", "items": [{"bbox_xywh": [0.0332, 0.0138, 0.1719, 0.0331], "text": "Invoice", "score": 0.998}]}

or, for a markdown-kind method:

# Annual Report 2024

## Overview

This is the one form that carries no image_hash / image_width / image_height. Use JSON mode when you need that metadata.

Multi-image input

Repeated <image> blocks, one per image:

<image image_hash="sha256:3f2a…" format="json" image_width="1024" image_height="1448">
{"object": "pp_ocrv6.ocr.regions", "items": [{"bbox_xywh": [0.0332, 0.0138, 0.1719, 0.0331], "text": "Invoice", "score": 0.998}]}
</image>
<image image_hash="sha256:9c4e…" format="json" image_width="512" image_height="724">
{"object": "pp_ocrv6.ocr.regions", "items": [{"bbox_xywh": [0.0117, 0.0203, 0.2422, 0.0414], "text": "Receipt", "score": 0.994}]}
</image>

Structured models (OCR, detection, and layout: everything except the chat VLMs) run inference per image and emit one block each, and usage is summed. Chat VLMs reason over all the images together and emit a single block.

Chat VLMs pass through

The chat VLMs (qwen/qwen3.5-0.8b and the rest of the chat family) return their reply verbatim in every case: text, one image, or several. No wrapper, no tags, byte-identical to the model’s own output. They do not accept document_url: a PDF request is a 400 capability error.

JSON mode

With response_format={"type":"json_object"} the reply is always the same envelope. The chat VLMs are the one exemption; see Chat VLMs are not enveloped.

{ "model": "paddleocr/pp-ocrv6", "method": "ocr", "data": [ <entry>, ... ] }

model and method name what produced the payload. Both keys are JSON-mode only. data holds one self-describing entry per input medium. A single image or PDF is a list of one; multiple images or documents (up to the model’s max_images) are a list of many. Clients iterate the same way every time.

Image entry
Document entry

{
  "object": "image",
  "image_hash": "sha256:…",
  "image_width": 1024,
  "image_height": 1448,
  "content": "<payload>"
}

object is the input medium: image, video, or document. It says which fields to read; the payload type lives on the content container’s own object. The hash and dimensions are omitted only when the image cannot be decoded.

{
  "object": "document",
  "file_name": "invoice.pdf",
  "file_hash": "sha256:…",
  "file_bytes": 48213,
  "mimetype": "application/pdf",
  "num_pages": 3,
  "dpi": 150,
  "pages": [
    {
      "object": "document.page",
      "page_index": 0,
      "page_width": 1275,
      "page_height": 1650,
      "content": { "object": "document.page.blocks", "items": [ <block>, ... ] }
    },
    {
      "object": "document.page",
      "page_index": 1,
      "page_width": 1275,
      "page_height": 1650,
      "status": "error"
    }
  ]
}

A page record carries object, page_index, page_width, page_height, content, and optional status. A failed page carries "status": "error" and no content. file_name is omitted for inline (data:) uploads, and the optional language is document-level.

`content` per medium

Medium	`content`
Document page	Always `{"object": "document.page.blocks", "items": [ <block>, ... ]}`, whatever method read the page.
Image, json kind	`{"object": "<group>.<method>.regions", "items": [ <region record>, ... ]}`. `items` is `[]` when nothing is found.
Image, markdown kind	The string itself, not wrapped.

A json payload is never a bare array, and a markdown payload is never wrapped. An image content has exactly one shape per (model, method).

{
  "model": "paddlepaddle/pp-doclayoutv3",
  "method": "detect",
  "data": [
    {
      "object": "image",
      "image_hash": "sha256:…",
      "image_width": 1700,
      "image_height": 2200,
      "content": {
        "object": "pp_doclayoutv3.detect.regions",
        "items": [
          {
            "bbox_xywh": [0.08, 0.0485, 0.84, 0.0364],
            "label": "doc_title",
            "order": 1,
            "score": 0.9813
          }
        ]
      }
    }
  ]
}

{
  "model": "zai-org/GLM-OCR",
  "method": "markdown",
  "data": [
    {
      "object": "image",
      "image_hash": "sha256:…",
      "image_width": 1700,
      "image_height": 2200,
      "content": "# Annual Report 2024\n\n## Overview"
    }
  ]
}

{
  "object": "document.page.blocks",
  "items": [
    {
      "index": 0,
      "text": "# Annual Report 2024\n\n## Overview"
    }
  ]
}

The region record

One entry of an image content.items, that is, what one detector found in one pass.

Key	Type	Notes
`bbox_xywh`	`[x, y, w, h]`	The single universal box: normalized 0-1, `precision` dp. There is no `bbox`, no `bbox_norm`, and no pixel or xyxy variant.
`poly_xy`	`[[x, y], ...]`	Normalized polygon, only when the model emits one (`paddleocr/pp-ocrv6`, `paddlepaddle/pp-doclayoutv3`).
`point_xy`	`[x, y]`	Normalized point, only when the method returns points rather than boxes.
`label`	`string`	Class or category. A layout category is folded to lower `snake_case`, so `doc_title` and `section_header` read the same whichever layout model produced them.
`text`	`string`	Recognized text.
`score`	`float`	Confidence 0-1.
`order`	`int`	Reading order, the detector’s own value, 1-based on `paddlepaddle/pp-doclayoutv3`.
`keypoints`	`[[x, y], ...]`	Normalized joints, only on `usyd-community/vitpose-plus-small`.

Unset optional keys are omitted. Which keys a given (model, method) emits is fixed, and an unknown key is a validation error, not a silently ignored extra.

The document block record

One entry of document.page.blocks.items. Every key is optional, and a model fills the subset it can, so one type covers every document method.

Key	Type	Notes
`index`	`int`	The block’s position on the page, zero-based like `page_index`.
`bbox_xywh`	`[x, y, w, h]`	Normalized 0-1 box. Absent on a whole-page read.
`poly_xy`	`[[x, y], ...]`	Normalized polygon, from a quad or polygon detector.
`label`	`string`	Layout category, folded to lower `snake_case`.
`text`	`string`	Recognized text. Absent when the block was not read, for example a `picture` region.
`text_format`	`"text"`, `"markdown"`, `"html"`, `"latex"`	Format of `text`. Omitted means `markdown`.
`score`	`float`	Detector confidence 0-1.
`parent`	`int`	The `index` of the block that contains this one.
`children`	`[int]`	The `index` of every block inside this one.
`attributes`	`object`	Metadata from a customized pipeline. Absent on the stock pipeline.

A whole-page read is one block that carries text and no geometry:

{
  "object": "document.page.blocks",
  "items": [
    { "index": 0, "text": "# Annual Report 2024\n\n## Overview" }
  ]
}

A layout read is one block per region:

{
  "object": "document.page.blocks",
  "items": [
    {
      "index": 0,
      "bbox_xywh": [0.08, 0.0485, 0.84, 0.0364],
      "label": "title",
      "text": "Annual Report 2024",
      "score": 0.98
    }
  ]
}

paddlepaddle/pp-doclayoutv3 numbers reading order from 1 on an image. On the document path that becomes the zero-based index.

Content object tags

On a document page the tag is always document.page.blocks, because the payload is a property of the medium. On an image the tag is <group>.<method>.regions, where the group is the model’s contract family. Aliases resolve first, so pp-ocrv6 and paddleocr/pp-ocrv6 share one contract.

Model	Method	`content.object`
`paddleocr/pp-ocrv6`	`ocr`	`pp_ocrv6.ocr.regions`
`paddleocr/pp-ocrv6`	`detect`	`pp_ocrv6.detect.regions`
`paddlepaddle/pp-doclayoutv3`	`detect`	`pp_doclayoutv3.detect.regions`
`rednote-hilab/dots.mocr`	`parse_layout`	`dots_mocr.parse_layout.regions`
`rednote-hilab/dots.mocr`	`parse_layout_only`	`dots_mocr.parse_layout_only.regions`
`microsoft/florence-2-base-ft`	`od`, `dense_region_caption`, `region_proposal`, `ocr_with_region`	`florence_2.<method>.regions`

A markdown-kind method on an image has no container and therefore no tag.

Chat VLMs are not enveloped

A chat VLM reply is passed through in JSON mode too: the body is the model’s own JSON, with no data wrapper and no object tag.

{ "animal": "golden retriever", "setting": "beach at sunset" }

This is what OpenAI’s response_format={"type":"json_object"} means: the model emits the JSON.

Text and JSON modes carry the same data

A text rendering decodes into exactly the JSON-mode data list: parse the body as JSON when format="json", and take it verbatim when format="markdown". One type therefore validates both renderings, and you can switch formats without a second parser. The single exception is the bare single-image form, which carries no image metadata: decoding recovers content but not image_hash, image_width, or image_height.

Methods by model

Output kind is a function of the model and the method. Default method in bold. Imgs is the maximum images per request, and Doc marks the models that accept document_url PDFs.

Model	Methods to output kind	Imgs	Doc
`qwen/qwen3.5-0.8b`	`chat` to passthrough	64	–
`microsoft/florence-2-base-ft`	`caption`, `detailed_caption`, `more_detailed_caption`, `ocr` to markdown · `od`, `dense_region_caption`, `region_proposal`, `ocr_with_region` to json	1	–
`paddleocr/pp-ocrv6`	`ocr`, `detect` to json · `text` to markdown	8	✓
`zai-org/glm-ocr`	`markdown` to markdown	1	✓
`rednote-hilab/dots.mocr`	`parse_layout`, `parse_layout_only` to json · `ocr`, `markdown` to markdown	1	✓
`paddlepaddle/pp-doclayoutv3`	`detect` to json	1	✓

The kind above is the image kind, which is also the page format in text mode. In JSON mode a document page is always document.page.blocks. paddlepaddle/pp-doclayoutv3 is a layout detector with no OCR path, so detect is its only method.

Method Parameters

method_params is an optional object passed alongside method at the top level of the request body, or via extra_body in the OpenAI Python SDK. Keys are model- and method-specific; the table below lists every key.

Request body
OpenAI SDK

{
  "model": "paddleocr/pp-ocrv6",
  "method": "ocr",
  "method_params": {
    "lang": "en",
    "score_threshold": 0.5
  },
  "messages": [ ... ]
}

response = client.chat.completions.create(
    model="paddleocr/pp-ocrv6",
    messages=[{...}],
    extra_body={
        "method": "ocr",
        "method_params": {"lang": "en", "score_threshold": 0.5},
    },
)

Method-specific keys

Key	Model	Method(s)	Description
`lang`	`paddleocr/pp-ocrv6`	`ocr`, `detect`	Recognition language (`en`, `ch`, `japan`, …). Default `en`.
`score_threshold`	`paddleocr/pp-ocrv6`	`ocr`	Drop text regions below this recognition confidence (0.0-1.0). Default `0.5`.
`score_threshold`	`paddlepaddle/pp-doclayoutv3`	`detect`	Drop layout regions below this detection confidence (0.0-1.0). Default `0.5`. Also binarizes the masks polygons are traced from, so lower values loosen `poly_xy`.

Models

Per-model dropdowns with request and response examples for every method.

Multimodal Inputs

Content part types and document input limits.

Flexible Document OCR

End-to-end recipe from model selection to response parsing.

Chat Completions

Full request and response schema.

Overview

Guides

API Reference

Misc

Methods & Response Format

The `method` field

The `response_format` field

The `precision` field

Text mode

Document input

Single image input

Multi-image input

Chat VLMs pass through

JSON mode

`content` per medium

The region record

The document block record

Content object tags

Chat VLMs are not enveloped

Text and JSON modes carry the same data

Methods by model

Method Parameters

Method-specific keys

Models

Multimodal Inputs

Flexible Document OCR

Chat Completions

​The method field

​The response_format field

​The precision field

​Text mode

​Document input

​Single image input

​Multi-image input

​Chat VLMs pass through

​JSON mode

​content per medium

​The region record

​The document block record

​Content object tags

​Chat VLMs are not enveloped

​Text and JSON modes carry the same data

​Methods by model

​Method Parameters

​Method-specific keys

​Related

Models

Multimodal Inputs

Flexible Document OCR

Chat Completions

The `method` field

The `response_format` field

The `precision` field

Text mode

Document input

Single image input

Multi-image input

Chat VLMs pass through

JSON mode

`content` per medium

The region record

The document block record

Content object tags

Chat VLMs are not enveloped

Text and JSON modes carry the same data

Methods by model

Method Parameters

Method-specific keys

Related