Document Classification

1. Pick an approach

	System One	Chat Completions
Surface	`POST /typesafe/v1/systemone`	`POST /v1/openai/chat/completions`
Output	One label from your set, always	Generated text you parse
Off-schema answer	Impossible	Possible, needs handling
Probability per label	Yes	No
Confidence to threshold on	Yes	No
Takes a PDF directly	Yes, first 8 pages, as a `file` part	Only on the OCR models
Free-form reasoning	No	Yes

Default to System One for a fixed label set. Use chat completions when the label set is not fixed. Use them when you need an explanation with the label.

2. Classify a page image

Add any yes/no checks the pipeline needs. Send the questions with an image_url part. The System One quickstart has the same call in Python, Node.js, cURL, and the CLI. It shows the shape of the reply.

QUESTIONS = {
    "kind": Choice(
        instructions="What kind of document is this page from?",
        criteria={
            "invoice": "A bill with amounts due",
            "receipt": "Proof of a completed payment",
            "contract": None,
            "form": None,
            "letter": None,
            "report": None,
            "other": None,
        },
    ),
    "handwriting": Noul(instructions="Does the page contain handwriting?"),
}

Do	Why
Cover the space, including `other`	A read must pick one of your labels, so a missing label becomes a confident wrong answer.
Keep labels mutually exclusive	Overlapping labels split probability mass and depress `confidence` on both.
Describe only the ambiguous ones	A description costs prompt tokens; `null` is right for a label whose name is self-evident.
Stay under ~20 labels	Beyond that the distribution flattens and thresholds stop separating.

An input with no detail is read at 280 vision tokens. Classification rarely needs full page detail. Set "detail": "low" to read at 70, as the examples here do. That bills 4x fewer image tokens. Latency stays the same. See Inputs.

3. Classify a PDF

Pass the PDF as a file part. The route rasterises the first 8 pages at 96 DPI. That is one call, with no OCR pass before it. Keep detail at low. Document kind does not need small print.

import base64, pathlib
from typesafe_sdk import TypeSafeClient

client = TypeSafeClient(
    api_key="<VLMRUN_API_KEY>",
    base_url="https://gateway.vlm.run/typesafe",
)

pdf = base64.b64encode(pathlib.Path("invoice.pdf").read_bytes()).decode()

result = client.system_one(  # QUESTIONS as defined in step 2
    "Answer about the document.",
    QUESTIONS,
    extra_body={
        "content": [
            {
                "type": "file",
                "file": {
                    "filename": "invoice.pdf",
                    "file_data": f"data:application/pdf;base64,{pdf}",
                    "detail": "low",
                },
            }
        ]
    },
    model="google/diffusiongemma-26b-a4b-it",
)

print(result.choices["kind"].choice)

import { readFileSync } from "node:fs";
import { TypeSafeClient } from "@typesafe-ai/sdk";

const client = new TypeSafeClient({
  apiKey: process.env.VLMRUN_API_KEY,
  baseURL: "https://gateway.vlm.run/typesafe",
  defaultModel: "google/diffusiongemma-26b-a4b-it",
});

const pdf = readFileSync("invoice.pdf").toString("base64");

const request = {
  state: "Answer about the document.",
  questions: QUESTIONS, // as defined in step 2
  content: [
    { type: "file",
      file: { filename: "invoice.pdf",
              file_data: `data:application/pdf;base64,${pdf}`,
              detail: "low" } },
  ],
};

const { answers } = await client.systemOne(request);
console.log(answers.kind.choice);

vlmrun gw s1 -s "Answer about the document." invoice.pdf --detail low \
  -m google/diffusiongemma-26b-a4b-it \
  --choice kind="invoice|receipt|contract|other"

PDF=$(base64 -w0 invoice.pdf)

curl https://gateway.vlm.run/typesafe/v1/systemone \
  -H "Authorization: Bearer $VLMRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d @- <<JSON
{
  "model": "google/diffusiongemma-26b-a4b-it",
  "state": "Answer about the document.",
  "content": [
    {"type": "file",
     "file": {"filename": "invoice.pdf",
              "file_data": "data:application/pdf;base64,$PDF",
              "detail": "low"}}
  ],
  "questions": {
    "kind": {
      "type": "choice",
      "instructions": "What kind of document is this?",
      "criteria": {"invoice": null, "receipt": null, "contract": null, "other": null}
    }
  }
}
JSON

A request carries images or one document, never both. The document’s pages are the read’s images. A longer PDF is answered from its opening pages. To read every page, fan out per page on chat completions.

Act on the confidence

confidence runs from 1 (all mass on one label) to 0 (uniform).

kind = result.choices["kind"]
top = kind.probabilities[kind.choice]

if kind.confidence >= 0.5 and top >= 0.7:
    route_to(kind.choice)                # confident, run the pipeline
elif top >= 0.4:
    route_to(kind.choice, review=True)   # plausible, flag for a spot-check
else:
    route_to("manual_triage")            # guessing, do not pretend otherwise

const kind = answers.kind;
const top = kind.probabilities[kind.choice];

if (kind.confidence >= 0.5 && top >= 0.7) {
  routeTo(kind.choice);                  // confident, run the pipeline
} else if (top >= 0.4) {
  routeTo(kind.choice, { review: true });// plausible, flag for a spot-check
} else {
  routeTo("manual_triage");              // guessing, do not pretend otherwise
}

curl -s https://gateway.vlm.run/typesafe/v1/systemone \
  -H "Authorization: Bearer $VLMRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d @request.json \
  | jq '.answers.kind | {choice, confidence, top: .probabilities[.choice]}'

Thresholds are model-specific. Tune them on a labelled sample of your own documents, and re-tune whenever you change models.

System One

Question types, images, confidence, and cost levers.

Document OCR

Reading documents to markdown or structured blocks.

System One API

Request and response reference.

Models

Catalog, capabilities, and accepted inputs per model.

Gateway

Documents

Images

Video and audio

Document Classification

1. Pick an approach

2. Classify a page image

3. Classify a PDF

Act on the confidence

System One

Document OCR

System One API

Models

​1. Pick an approach

​2. Classify a page image

​3. Classify a PDF

​ Act on the confidence

​Related

System One

Document OCR

System One API

Models

1. Pick an approach

2. Classify a page image

3. Classify a PDF

Act on the confidence

Related