Word Boxes

paddleocr/pp-ocrv6 returns a box for every word, grouped under its line, when you set method_params.word_boxes to true. Boxes are normalized to the page (0 to 1), so they map onto the original PDF or image at any resolution.

Word boxes are opt-in and add nothing to the price: a request bills the same with or without them.

Request

Send an image or a PDF with method: "ocr". JSON mode is the easiest to parse; text mode carries the same words.

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

response = client.chat.completions.create(
    model="paddleocr/pp-ocrv6",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "document_url",
                    "document_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/document.invoice/sample-invoice.pdf"},
                }
            ],
        }
    ],
    response_format={"type": "json_object"},
    extra_body={"method": "ocr", "method_params": {"word_boxes": True}},
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.vlm.run/v1/openai",
  apiKey: process.env.VLMRUN_API_KEY,
});

const response = await client.chat.completions.create({
  model: "paddleocr/pp-ocrv6",
  messages: [
    {
      role: "user",
      content: [
        {
          type: "document_url",
          document_url: {
            url: "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/document.invoice/sample-invoice.pdf",
          },
        },
      ],
    },
  ],
  response_format: { type: "json_object" },
  method: "ocr",
  method_params: { word_boxes: true },
});

console.log(response.choices[0].message.content);

curl https://gateway.vlm.run/v1/openai/chat/completions \
  -X POST \
  -H "Authorization: Bearer $VLMRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "paddleocr/pp-ocrv6",
    "method": "ocr",
    "method_params": {"word_boxes": true},
    "response_format": {"type": "json_object"},
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "document_url",
            "document_url": {
              "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/document.invoice/sample-invoice.pdf"
            }
          }
        ]
      }
    ]
  }'

Response

PDF

Each page’s content.items is a flat list of blocks. A line block lists its words in child_ids. Each word block follows its line and names it in parent_id.

{"block_id": 7, "label": "line", "text": "Im Bruch 3, 63897 Miltenberg", "score": 0.9999,
 "bbox_xywh": [0.6083, 0.1202, 0.17, 0.0116], "child_ids": [8, 9, 10, 11, 12]}
{"block_id": 8, "label": "word", "text": "Im", "bbox_xywh": [0.6134, 0.1202, 0.0076, 0.0116], "parent_id": 7}
{"block_id": 9, "label": "word", "text": "Bruch", "bbox_xywh": [0.6297, 0.1202, 0.0302, 0.0116], "parent_id": 7}
{"block_id": 10, "label": "word", "text": "3,", "bbox_xywh": [0.6662, 0.1202, 0.0076, 0.0116], "parent_id": 7}

To read lines only, keep the blocks with no parent_id.

Image

Each line in content.items carries a words list, in reading order.

{
  "text": "123 Main Street",
  "score": 0.9998,
  "bbox_xywh": [0.093, 0.1712, 0.1151, 0.0123],
  "words": [
    {"text": "123", "bbox_xywh": [0.0955, 0.1712, 0.0245, 0.0123]},
    {"text": "Main", "bbox_xywh": [0.1261, 0.1712, 0.0318, 0.0123]},
    {"text": "Street", "bbox_xywh": [0.1652, 0.1712, 0.0404, 0.0123]}
  ]
}

bbox_xywh is [x, y, width, height] from the top-left corner. See Response Formats.

Highlight a phrase

Match the phrase against consecutive words on a page, then scale each box by the page size. This example uses PyMuPDF and continues from the Python request above.

import json
import pymupdf

reply = json.loads(response.choices[0].message.content)
doc = pymupdf.open("sample-invoice.pdf")  # the same PDF you sent
phrase = "63897 Miltenberg".lower().split()

for page in reply["pages"]:
    words = [b for b in (page.get("content") or {}).get("items", []) if b.get("label") == "word"]
    texts = [w["text"].lower().strip(".,:;()") for w in words]
    pdf_page = doc[page["page_id"]]
    W, H = pdf_page.rect.width, pdf_page.rect.height
    for i in range(len(words) - len(phrase) + 1):
        if texts[i : i + len(phrase)] == phrase:
            for w in words[i : i + len(phrase)]:
                x, y, bw, bh = w["bbox_xywh"]
                pdf_page.add_highlight_annot(pymupdf.Rect(x * W, y * H, (x + bw) * W, (y + bh) * H))

doc.save("highlighted.pdf")

A phrase can wrap onto the next line. Matching over the page’s word list, not one line, finds it.

Behavior


Word	Split on whitespace. Punctuation stays with its word (`3,`, `$99.00`). Chinese text is one word per character.
Box	Axis-aligned and the height of its line.
Confidence	Per line (`score`), not per word.
`detect`	With `word_boxes`, one box per word instead of per line, without text.
`text`	Ignores `word_boxes`.
Invalid value	`word_boxes` takes `true` or `false`. Anything else is a `400` with `param: "method_params.word_boxes"`.
Small print	Raise `document_dpi` for small type in a PDF.

Document OCR

Pick a model and read a PDF page by page.

Method parameters

Every method_params key, by model.

Gateway

Documents

Images

Video and audio

Request

Response

PDF

Image

Highlight a phrase

Behavior

Document OCR

Method parameters

​Request

​Response

​PDF

​Image

​Highlight a phrase

​Behavior

​Related

Document OCR

Method parameters

Request

Response

PDF

Image

Highlight a phrase

Behavior

Related