Document Classification

Classification is a routing decision: given a page or a document, pick one label from a fixed set so the right pipeline runs next. This guide covers both ways to do it on the Gateway, and when each is the right tool.

1. Pick an approach

	System One	Chat Completions
Surface	`POST /typesafe/v1/systemone`	`POST /v1/openai/chat/completions`
Output	One label from your set, always	Generated text you parse
Off-schema answer	Impossible	Possible, needs handling
Probability per label	Yes	No
Confidence to threshold on	Yes	No
Takes a PDF directly	Yes, first 8 pages, as a `file` part	Only on the OCR models
Free-form reasoning	No	Yes

Default to System One: a fixed label set with a probability attached is exactly what a routing step needs, and the answer can never come back as something outside your set. Reach for chat completions when the label set is not fixed, or when you need an explanation alongside the label.

2. Classify a page image

One choice question with an entry per label, plus any yes/no checks the pipeline needs. Descriptions are optional; null is fine when a label speaks for itself.

QUESTIONS = {
    "kind": Choice(
        instructions="What kind of document is this page from?",
        criteria={
            "invoice": "A bill with amounts due",
            "receipt": "Proof of a completed payment",
            "contract": None,
            "form": None,
            "letter": None,
            "report": None,
            "other": None,
        },
    ),
    "handwriting": Noul(instructions="Does the page contain handwriting?"),
}

Send it with an image_url part. The System One quickstart has the same call in Python, Node.js, cURL and the CLI, and the shape of the reply. Designing the label set is the part that decides your accuracy:

Do	Why
Cover the space, including `other`	A read must pick one of your labels, so a missing label becomes a confident wrong answer.
Keep labels mutually exclusive	Overlapping labels split probability mass and depress `confidence` on both.
Describe only the ambiguous ones	A description costs prompt tokens; `null` is right for a label whose name is self-evident.
Stay under ~20 labels	Beyond that the distribution flattens and thresholds stop separating.

Classification rarely needs full page detail, but the default gives it to you: an input with no detail is read at 280 vision tokens. Set "detail": "low" to read at 70 instead, as the examples here do, for 4x fewer billed image tokens at the same latency. See Inputs.

3. Classify a PDF

Pass the PDF straight through as a file part. The route rasterises its first 8 pages for you at 96 DPI, so this is one call, not an OCR pass followed by a classification pass. detail stays low here: naming a document kind does not need small print.

import base64, pathlib
from typesafe_sdk import TypeSafeClient

client = TypeSafeClient(
    api_key="<VLMRUN_API_KEY>",
    base_url="https://gateway.vlm.run/typesafe",
)

pdf = base64.b64encode(pathlib.Path("invoice.pdf").read_bytes()).decode()

result = client.system_one(  # QUESTIONS as defined in step 2
    "Answer about the document.",
    QUESTIONS,
    extra_body={
        "content": [
            {
                "type": "file",
                "file": {
                    "filename": "invoice.pdf",
                    "file_data": f"data:application/pdf;base64,{pdf}",
                    "detail": "low",
                },
            }
        ]
    },
    model="google/diffusiongemma-26b-a4b-it",
)

print(result.choices["kind"].choice)

import { readFileSync } from "node:fs";
import { TypeSafeClient } from "@typesafe-ai/sdk";

const client = new TypeSafeClient({
  apiKey: process.env.VLMRUN_API_KEY,
  baseURL: "https://gateway.vlm.run/typesafe",
  defaultModel: "google/diffusiongemma-26b-a4b-it",
});

const pdf = readFileSync("invoice.pdf").toString("base64");

const request = {
  state: "Answer about the document.",
  questions: QUESTIONS, // as defined in step 2
  content: [
    { type: "file",
      file: { filename: "invoice.pdf",
              file_data: `data:application/pdf;base64,${pdf}`,
              detail: "low" } },
  ],
};

const { answers } = await client.systemOne(request);
console.log(answers.kind.choice);

vlmrun gw s1 -s "Answer about the document." invoice.pdf --detail low \
  -m google/diffusiongemma-26b-a4b-it \
  --choice kind="invoice|receipt|contract|other"

PDF=$(base64 -w0 invoice.pdf)

curl https://gateway.vlm.run/typesafe/v1/systemone \
  -H "Authorization: Bearer $VLMRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d @- <<JSON
{
  "model": "google/diffusiongemma-26b-a4b-it",
  "state": "Answer about the document.",
  "content": [
    {"type": "file",
     "file": {"filename": "invoice.pdf",
              "file_data": "data:application/pdf;base64,$PDF",
              "detail": "low"}}
  ],
  "questions": {
    "kind": {
      "type": "choice",
      "instructions": "What kind of document is this?",
      "criteria": {"invoice": null, "receipt": null, "contract": null, "other": null}
    }
  }
}
JSON

A request carries images or one document, never both: the document’s pages are the read’s images. A longer PDF is answered from its opening pages, which is what naming a document kind needs. When you genuinely need every page, fan out per page on chat completions instead.

Act on the confidence

The point of a calibrated answer is that you can decide when not to trust it. confidence runs from 1 (all mass on one label) to 0 (uniform).

kind = result.choices["kind"]
top = kind.probabilities[kind.choice]

if kind.confidence >= 0.5 and top >= 0.7:
    route_to(kind.choice)                # confident, run the pipeline
elif top >= 0.4:
    route_to(kind.choice, review=True)   # plausible, flag for a spot-check
else:
    route_to("manual_triage")            # guessing, do not pretend otherwise

const kind = answers.kind;
const top = kind.probabilities[kind.choice];

if (kind.confidence >= 0.5 && top >= 0.7) {
  routeTo(kind.choice);                  // confident, run the pipeline
} else if (top >= 0.4) {
  routeTo(kind.choice, { review: true });// plausible, flag for a spot-check
} else {
  routeTo("manual_triage");              // guessing, do not pretend otherwise
}

curl -s https://gateway.vlm.run/typesafe/v1/systemone \
  -H "Authorization: Bearer $VLMRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d @request.json \
  | jq '.answers.kind | {choice, confidence, top: .probabilities[.choice]}'

Thresholds are model-specific. Tune them on a labelled sample of your own documents, and re-tune whenever you change models.

System One

Question types, images, confidence, and cost levers.

Document OCR

Reading documents to markdown or structured blocks.

System One API

Request and response reference.

Models

Catalog, capabilities, and accepted inputs per model.

Get Started

OpenAI-Compatible API

TypeSafe-Compatible API

Guides

API Reference

Tools

Platform

Document Classification

1. Pick an approach

2. Classify a page image

3. Classify a PDF

Act on the confidence

System One

Document OCR

System One API

Models

​1. Pick an approach

​2. Classify a page image

​3. Classify a PDF

​ Act on the confidence

​Related

System One

Document OCR

System One API

Models

1. Pick an approach

2. Classify a page image

3. Classify a PDF

Act on the confidence

Related