Flexible Document OCR

This guide walks through running a document (PDF) through the VLM Run Gateway, from picking a model to handling the response. Document requests fan out one page at a time and default to ocr for pp-ocrv6. For a quick copy-paste example, see Quickstart. For the cost/latency/accuracy rationale behind picking a model, method, and DPI, see Why the Gateway.

1. Pick a model

Start with paddleocr/pp-ocrv6 for general-purpose PDF OCR. See Models for the full catalog and Methods if you need a specific operation like text detection (detect) instead of full OCR. zai-org/glm-ocr serves markdown. rednote-hilab/dots.mocr adds layout parsing via parse_layout and parse_layout_only, plus ocr and markdown. For layout without text recognition, use paddlepaddle/pp-doclayoutv3: it returns ordered layout regions (labels, boxes, reading order) per page and no transcribed text, which makes it the layout stage of a layout-then-OCR pipeline. detect is its only method and its default, so you can omit method entirely. markdown is the default on zai-org/glm-ocr and rednote-hilab/dots.mocr. paddleocr/pp-ocrv6 defaults to ocr and offers text for its plain string form.

2. Send the document

Pass the PDF as a document_url content part. Prefer a hosted URL over a base64 data URI. See Multimodal Inputs for size limits and Method Parameters for document_dpi and other knobs.

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

response = client.chat.completions.create(
    model="paddleocr/pp-ocrv6",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "document_url",
                    "document_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/document.invoice/sample-invoice.pdf"},
                }
            ],
        }
    ],
    extra_body={"method": "ocr", "document_dpi": 72},
)

Request knobs

Field	Default	Notes
`document_dpi`	`72`	Rasterization DPI per page. See Method Parameters.
`document_max_pages`	`128`	Pages one request may read. A longer PDF is a `400` naming the ranges that cover it. See Multimodal Inputs.
`document_pages`	all	Which 0-indexed pages to read: page indices and/or `[start, stop]` half-open ranges. Only those pages are rasterized, and they are re-numbered from 0.
`method`	model default	Per-page backend operation. It sets each page’s payload kind (`format="markdown"` or `format="json"`), not the page wrapper. See Methods.
`response_format`	omitted	Omitted (or `{"type":"text"}`) returns the `<document>` / `<page>` text blocks; `{"type":"json_object"}` returns the JSON object. See Parse the response.
`precision`	`4`	Decimal places on normalized `bbox_xywh` / `poly_xy` / `point_xy` and `score`. Ignored for markdown payloads.
`stream`	`false`	When true, emits ordered SSE page blocks. Ignored when a JSON `response_format` is requested. See Streaming vs non-streaming.

Document rasterization

When the request includes a document_url part, document_dpi controls the rasterization DPI for each page before OCR. Set it at the top level or inside method_params (either location works; if both are present, method_params wins).

DPI	Tradeoff
`72` (default)	Fast and cheap; fine for clean, standard-size text.
`150`	Good balance of legibility and per-page inference cost for denser pages.
`300`+	Preserves fine print and small text, at higher inference cost and latency per page.

Start at the default and only raise document_dpi if you see missed text on dense or small-font pages. Page count and file-size limits are documented under Multimodal Inputs.

3. Streaming vs non-streaming

In text mode, stream: true is honored for every chat request, and the streamed result is byte-identical to the non-streaming one. Document requests emit one SSE chunk per completed page block, in ascending page order. Other chat requests (regular chat, single-image OCR) buffer the full reply and re-chunk it into the same OpenAI SSE contract (no time-to-first-token benefit yet). A JSON response_format is always served non-streamed, since a single JSON object cannot be assembled from per-page deltas.

Mode	When to use
Non-streaming (default)	Short documents, batch pipelines, or when you need the full result before continuing.
Streaming (`stream: true`)	Long documents where you want to render or process pages as they finish.

Set stream: true to receive each page over SSE as it completes, in ascending page order, instead of waiting for the entire document:

stream = client.chat.completions.create(
    model="paddleocr/pp-ocrv6",
    stream=True,
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "document_url", "document_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf"}}
            ],
        }
    ],
    extra_body={"method": "ocr", "document_dpi": 72},
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="")

Each delta.content value is one component of the document structure: the <document> open tag, then one <page> block per page, then the close tag. Each <page> carries its own format attribute, so a streamed page is self-describing before the client has buffered the header.

4. Parse the response

response_format picks text mode or JSON mode, and method picks each page’s payload kind. Both modes carry the same per-page content; JSON mode is easiest to consume programmatically. See Methods & Response Format for the full contract.

Text mode (default)

Every document method returns the same wrapper: one <document> element per input PDF, carrying its metadata as attributes, wrapping one <page> block per rasterized page. method decides only the page body, and each page declares it in format: markdown for a Markdown body (markdown, text, and dots.mocr’s ocr) and json for a region payload (detect, parse_layout, parse_layout_only).

<document file_name="invoice.pdf" file_hash="sha256:1b7f…" file_bytes="120000" mimetype="application/pdf" num_pages="2" dpi="72">
<page page_index="0" format="markdown" page_width="612" page_height="792">
# ACME Corp, Invoice #4471

| Item | Qty | Price |
| ---- | --- | ------ |
| Widget | 3 | $12.00 |
</page>
<page page_index="1" format="markdown" page_width="612" page_height="792" status="error"/>
</document>

A failed page is self-closing, with status="error" and no body, so page numbering stays intact.
A single image (not a PDF) returns the payload block alone, with no wrapper.
Several images return one <image> block each.

JSON mode

JSON mode is the same for every method. Add response_format={"type":"json_object"} to get one object, naming what produced it. data holds one document entry per input PDF; each entry lists its pages, and a failed page carries "status": "error" with no content:

{
  "model": "paddleocr/pp-ocrv6",
  "method": "ocr",
  "data": [
    {
      "object": "document",
      "file_name": "invoice.pdf",
      "file_hash": "sha256:…",
      "file_bytes": 120000,
      "mimetype": "application/pdf",
      "num_pages": 2,
      "dpi": 72,
      "pages": [
        {
          "object": "document.page",
          "page_index": 0,
          "page_width": 612,
          "page_height": 792,
          "content": {
            "object": "document.page.blocks",
            "items": [
              { "index": 0, "bbox_xywh": [0.0332, 0.0138, 0.1719, 0.0331], "text": "Invoice", "score": 0.998 }
            ]
          }
        },
        {
          "object": "document.page",
          "page_index": 1,
          "page_width": 612,
          "page_height": 792,
          "status": "error"
        }
      ]
    }
  ]
}

Every page’s content is document.page.blocks, whatever method read it. A whole-page method (markdown, text, dots.mocr’s ocr) fills it with one block that carries text and no geometry:

{
  "object": "document.page.blocks",
  "items": [
    { "index": 0, "text": "# ACME Corp, Invoice #4471\n\n| Item | Qty | Price |" }
  ]
}

A region method (ocr, detect, parse_layout, parse_layout_only) fills it with one block per region, each with bbox_xywh and, where the method reads the region, text. items is [] when nothing is found, and it is never a bare array. See The document block record for every key, and the per-model pages on Models for exact examples.

This is the only JSON shape the Gateway returns, so you cannot extract a custom document schema here. For schema-driven extraction, use the VLM Run API’s structured responses and custom schemas.

Multiple documents in one request

When a request includes more than one document_url (or file_url) content part, each PDF gets its own entry: they are never merged. In text mode that is one <document> block per input, concatenated in request order; in JSON mode data is a list of document entries in the same order.

<document file_name="a.pdf" file_hash="sha256:…" file_bytes="120000" mimetype="application/pdf" num_pages="1" dpi="72">
<page page_index="0" format="markdown" page_width="612" page_height="792">
Page one text
</page>
</document>
<document file_name="b.pdf" file_hash="sha256:…" file_bytes="240000" mimetype="application/pdf" num_pages="2" dpi="72">
<page page_index="0" format="markdown" page_width="612" page_height="792">
Section A
</page>
<page page_index="1" format="markdown" page_width="612" page_height="792">
Section B
</page>
</document>

Page indices restart at 0 for each document, in both modes.

Element attributes

Element	Attribute	Description
`<document>`	`file_name`	Original file name of the source document. Omitted for `data:` URI inputs.
`<document>`	`file_hash`	SHA-256 of the source bytes, for correlation.
`<document>`	`file_bytes`	Size of the source document in bytes.
`<document>`	`mimetype`	MIME type of the source (e.g. `application/pdf`).
`<document>`	`num_pages`	Number of pages in the document.
`<document>`	`dpi`	Rasterization DPI applied to every page.
`<document>`	`language`	Comma-separated ISO-639 codes in confidence order. Present only when the read reports one.
`<page>`	`page_index`	Zero-based page index within its parent document.
`<page>`	`format`	Payload kind of this page’s body: `markdown` or `json`.
`<page>`	`page_width` / `page_height`	Rasterized page dimensions in pixels at the document’s DPI.
`<page>`	`status`	`error` on a page that failed. The element is then self-closing.

The JSON-mode keys carry the same names, so the two modes map key for key.

Consuming each shape

Non-streaming: the blocks are concatenated in order inside choices[0].message.content. Streaming: the VLM Run Gateway emits OpenAI-style SSE chunks in document order, byte-identical to the non-streaming reply: the <document> open tag, one chunk per <page> block, then the matching </document> close tag. The next document follows after. To consume text mode, split on the <document> / <page> open and close tags, then read each page body according to its format. To skip parsing entirely, ask for JSON mode.

5. Track cost and handle errors

Non-streaming: read response.usage.cost for per-request metering during the alpha. Streaming: pass stream_options={"include_usage": true} so the terminal SSE event carries usage (including usage.cost) on the chunk instead of delta.content. Check for it before treating every chunk as page content:

stream = client.chat.completions.create(
    model="paddleocr/pp-ocrv6",
    stream=True,
    stream_options={"include_usage": True},
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "document_url", "document_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf"}}
            ],
        }
    ],
    extra_body={"method": "ocr", "document_dpi": 72},
)

usage = None
pages: list[str] = []

for chunk in stream:
    choice = chunk.choices[0]
    if choice.delta.content:
        pages.append(choice.delta.content)
    if chunk.usage:
        usage = chunk.usage

print(f"Cost: {usage.cost if usage else 'n/a'}")

For both modes:

Retry 429/500 with backoff; do not retry 400 capability violations. See Error Codes.
Keep the x-request-id response header if you need to contact support.

Multimodal Inputs

Content part types, document limits, and URL vs base64 tradeoffs.

Methods

Per-model method and method_params reference.

Error Codes

Status codes, error bodies, and retry guidance.

Models

Document and image OCR model catalog.

Overview

Guides

API Reference

Misc

Flexible Document OCR

1. Pick a model

2. Send the document

Request knobs

Document rasterization

3. Streaming vs non-streaming

4. Parse the response

Text mode (default)

JSON mode

Multiple documents in one request

Element attributes

Consuming each shape

5. Track cost and handle errors

Multimodal Inputs

Methods

Error Codes

Models

​1. Pick a model

​2. Send the document

​Request knobs

​Document rasterization

​3. Streaming vs non-streaming

​4. Parse the response

​Text mode (default)

​JSON mode

​Multiple documents in one request

​Element attributes

​Consuming each shape

​5. Track cost and handle errors

​Related

Multimodal Inputs

Methods

Error Codes

Models

1. Pick a model

2. Send the document

Request knobs

Document rasterization

3. Streaming vs non-streaming

4. Parse the response

Text mode (default)

JSON mode

Multiple documents in one request

Element attributes

Consuming each shape

5. Track cost and handle errors

Related