Structured Table Extraction with Layout Preservation

​

The Challenge: Beyond Simple Extraction

• Nested Headers: Hierarchical column relationships are lost.
• Merged Cells: Spanning information is discarded, breaking row/column alignment.
• Semantic Grouping: Visual cues indicating related data are ignored.

​

MarkdownTable: A Schema for Structure and Semantics

from typing import List, Optional, Literal
from pydantic import BaseModel, Field

# Simplified representation for documentation
# Actual implementation includes more detailed fields
class MarkdownTable(BaseModel):
    """A table with layout information."""

    class MarkdownTableMetadata(BaseModel):
        title: str | None = Field(None, description="Title of the table.")
        caption: str | None = Field(None, description="Caption of the table.")

    class MarkdownTableHeader(BaseModel):
        id: str = Field(..., description="Unique, sanitized identifier (e.g., 'col_subcol').")
        column: int = Field(..., description="0-indexed column index.")
        name: str = Field(..., description="Hierarchical name (e.g., 'Header > Subheader').")
        dtype: str | None = Field(None, description="Inferred data type.")

    metadata: MarkdownTableMetadata = Field(default_factory=MarkdownTableMetadata)
    content: str | None = Field(
        None,
        description="Table data in Markdown format, optimized for structure preservation."
    )
    headers: List[MarkdownTableHeader] = Field(..., description="List of structured header objects.")

​

Key Design Benefits:

1. Hierarchical Headers (headers):
   • The name field uses > to represent nesting (e.g., Performance > Max Value).
   • The unique id provides a stable reference for each column, crucial for programmatic access and comparison.
   • column index and dtype add essential metadata for data validation and processing.
2. Layout-Preserving Markdown (content):
   • The table is rendered as GitHub-flavored markdown.
   • Crucially, spanned cells are handled by repeating the cell value across the spanned rows/columns. This ensures the markdown table has a regular grid structure, directly loadable into structures like Pandas DataFrames without complex parsing or reconstruction.
   • The first row of the markdown only contains the unique ids from the headers list, providing a clean mapping for data ingestion.
3. Rich Context (metadata): Captures titles, captions, and notes often surrounding tables, providing essential context that might be lost otherwise.
4. Downstream Interoperability: The combination of structured headers and the regularized markdown content facilitates seamless conversion to Pandas DataFrames, database schemas, or input for further LLM analysis.

​

Extracting Structured Tables via SDK

import time
from pathlib import Path
from typing import List, Optional, Literal
from pydantic import BaseModel, Field
from vlmrun.client import VLMRun
from vlmrun.client.types import PredictionResponse, GenerationConfig
from vlmrun.dtypes.base import MarkdownPage

# Initialize the client
client = VLMRun(api_key="<VLMRUN_API_KEY>")

# Extract tables from a document using `document.table-markdown`
# that is optimized for structured table extraction
path = Path("path/to/technical_document.pdf")
prediction: PredictionResponse = client.document.generate(
    file=path,
    domain="document.table-markdown"
)

# Access the structured response
page: MarkdownPage = prediction.response
print(page.model_dump_json(indent=2))

​

Example: Structured Output

Extracting Dense Tables in a Technical Document

{
  "metadata": {
    "title": "Sensor Performance Characteristics",
    "caption": "Table 1: Key performance metrics at 25°C",
    "notes": "Typical values unless otherwise noted."
  },
  "headers": [
    {
      "id": "parameter",
      "column": 0,
      "name": "Parameter",
      "dtype": "str"
    },
    {
      "id": "conditions",
      "column": 1,
      "name": "Conditions",
      "dtype": "str"
    },
    {
      "id": "performance_min",
      "column": 2,
      "name": "Performance > Min",
      "dtype": "float"
    },
    {
      "id": "performance_typ",
      "column": 3,
      "name": "Performance > Typ",
      "dtype": "float"
    },
    {
      "id": "performance_max",
      "column": 4,
      "name": "Performance > Max",
      "dtype": "float"
    },
    {
      "id": "units",
      "column": 5,
      "name": "Units",
      "dtype": "str"
    }
  ],
  "content": "| parameter | conditions | performance_min | performance_typ | performance_max | units |\n|---|---|---|---|---|---|\n| Sensitivity | Vcc = 5V | 0.9 | 1.0 | 1.1 | mV/G |\n| Zero Field Output | Vcc = 5V | 2450 | 2500 | 2550 | mV |\n| Linearity | Full Range | -0.5 | 0.1 | 0.5 | % |\n| Noise | 1Hz to 1kHz | - | 150 | - | uVrms |"
}

​

Example: Rendered Output

for table in page.tables:
    print(f"Table [title={table.metadata.title}, caption={table.metadata.caption}]")
    print(table.render()) # renders the table as markdown format

​

Benefits Demonstrated:

• Nested Headers: Performance > Min, Performance > Typ, Performance > Max clearly show the hierarchy under Performance.
• Markdown Ready: The render method returns a string that is valid GitHub-flavored markdown:
  Copy

  | parameter | conditions | performance_min | performance_typ | performance_max | units |
  |---|---|---|---|---|---|
  | Sensitivity | Vcc = 5V | 0.9 | 1.0 | 1.1 | mV/G |
  | Zero Field Output | Vcc = 5V | 2450 | 2500 | 2550 | mV |
  | Linearity | Full Range | -0.5 | 0.1 | 0.5 | % |
  | Noise | 1Hz to 1kHz | - | 150 | - | uVrms |
  

• Pandas Integration: The data field in the MarkdownTable object can be easily read into a Pandas DataFrame, with appropriate header metadata such as unique id, column index, name, and dtype. We provide a convenience method to_dataframe to convert the MarkdownTable object to a Pandas DataFrame.
  Copy

  import pandas as pd
  
  # Convert the extracted markdown table to a pandas dataframe
  df: pd.DataFrame = table.to_dataframe(header="id")
  print(df)
  

​

Fine-tuning for Domain Specificity

​

Try our Document -> JSON API today