> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vlm.run/llms.txt
> Use this file to discover all available pages before exploring further.

# Structured Table Extraction with Layout Preservation

> Leverage vlm-1 with the TableWithLayout schema to extract complex tables, preserving hierarchical structure and enabling robust downstream processing.

Extracting data from tables within documents presents a significant challenge, especially with dense, complex layouts involving nested headers, merged cells, and implicit structural hierarchies. Simple text extraction or basic table parsing often fails to capture the full semantic and structural integrity required for reliable downstream analysis and processing. `vlm-1`, coupled with the specialized `TableWithLayout` schema, provides a robust solution designed for experts who demand high fidelity and structural preservation.

## The Challenge: Beyond Simple Extraction

Traditional methods often flatten table structures, losing critical layout information:

* **Nested Headers:** Hierarchical column relationships are lost.
* **Merged Cells:** Spanning information is discarded, breaking row/column alignment.
* **Semantic Grouping:** Visual cues indicating related data are ignored.

This loss of information hinders accurate data analysis, comparison across documents, and seamless integration into data pipelines (e.g., Pandas DataFrames).

## `MarkdownTable`: A Schema for Structure and Semantics

To address these challenges, we introduce a new schema called `MarkdownTable`, meticulously designed to capture not just the cell content but also the table's intrinsic structure and metadata.

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  from typing import List, Optional, Literal
  from pydantic import BaseModel, Field

  # Simplified representation for documentation
  # Actual implementation includes more detailed fields
  class MarkdownTable(BaseModel):
      """A table with layout information."""

      class MarkdownTableMetadata(BaseModel):
          title: str | None = Field(None, description="Title of the table.")
          caption: str | None = Field(None, description="Caption of the table.")

      class MarkdownTableHeader(BaseModel):
          id: str = Field(..., description="Unique, sanitized identifier (e.g., 'col_subcol').")
          column: int = Field(..., description="0-indexed column index.")
          name: str = Field(..., description="Hierarchical name (e.g., 'Header > Subheader').")
          dtype: str | None = Field(None, description="Inferred data type.")

      metadata: MarkdownTableMetadata = Field(default_factory=MarkdownTableMetadata)
      content: str | None = Field(
          None,
          description="Table data in Markdown format, optimized for structure preservation."
      )
      headers: List[MarkdownTableHeader] = Field(..., description="List of structured header objects.")
  ```
</CodeGroup>

### Key Design Benefits:

1. **Hierarchical Headers (`headers`):**
   * The `name` field uses `>` to represent nesting (e.g., `Performance > Max Value`).
   * The unique `id` provides a stable reference for each column, crucial for programmatic access and comparison.
   * `column` index and `dtype` add essential metadata for data validation and processing.
2. **Layout-Preserving Markdown (`content`):**
   * The table is rendered as GitHub-flavored markdown.
   * **Crucially, spanned cells are handled by repeating the cell value across the spanned rows/columns.** This ensures the markdown table has a regular grid structure, directly loadable into structures like Pandas DataFrames without complex parsing or reconstruction.
   * The first row of the markdown *only* contains the unique `id`s from the `headers` list, providing a clean mapping for data ingestion.
3. **Rich Context (`metadata`):** Captures titles, captions, and notes often surrounding tables, providing essential context that might be lost otherwise.
4. **Downstream Interoperability:** The combination of structured `headers` and the regularized markdown `content` facilitates seamless conversion to Pandas DataFrames, database schemas, or input for further LLM analysis.

## Extracting Structured Tables via SDK

You can leverage `vlm-1` with the `TableWithLayout` schema using the Python SDK. Specify the schema in the `GenerationConfig`.

<CodeGroup>
  ```python Python theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  import time
  from pathlib import Path
  from typing import List, Optional, Literal
  from pydantic import BaseModel, Field
  from vlmrun.client import VLMRun
  from vlmrun.client.types import PredictionResponse, GenerationConfig
  from vlmrun.dtypes.base import MarkdownPage

  # Initialize the client
  client = VLMRun(api_key="<VLMRUN_API_KEY>")

  # Extract tables from a document using `document.table-markdown`
  # that is optimized for structured table extraction
  path = Path("path/to/technical_document.pdf")
  prediction: PredictionResponse = client.document.generate(
      file=path,
      domain="document.table-markdown"
  )

  # Access the structured response
  page: MarkdownPage = prediction.response
  print(page.model_dump_json(indent=2))
  ```
</CodeGroup>

## Example: Structured Output

Consider the following table with nested headers and merged cells.

<Frame caption="Extracting Dense Tables in a Technical Document">
  <img src="https://mintcdn.com/autonomiai/hv1ZFyEZ1wMYWx0b/guides/doc-ai/images/dense-table-extraction.jpg?fit=max&auto=format&n=hv1ZFyEZ1wMYWx0b&q=85&s=b6d965e572360b9bb59e18ed3bd1f836" width="80%" data-path="guides/doc-ai/images/dense-table-extraction.jpg" />
</Frame>

The `MarkdownTable` output captures this complexity:

```json theme={"theme":{"light":"github-light","dark":"dark-plus"}}
{
  "metadata": {
    "title": "Sensor Performance Characteristics",
    "caption": "Table 1: Key performance metrics at 25°C",
    "notes": "Typical values unless otherwise noted."
  },
  "headers": [
    {
      "id": "parameter",
      "column": 0,
      "name": "Parameter",
      "dtype": "str"
    },
    {
      "id": "conditions",
      "column": 1,
      "name": "Conditions",
      "dtype": "str"
    },
    {
      "id": "performance_min",
      "column": 2,
      "name": "Performance > Min",
      "dtype": "float"
    },
    {
      "id": "performance_typ",
      "column": 3,
      "name": "Performance > Typ",
      "dtype": "float"
    },
    {
      "id": "performance_max",
      "column": 4,
      "name": "Performance > Max",
      "dtype": "float"
    },
    {
      "id": "units",
      "column": 5,
      "name": "Units",
      "dtype": "str"
    }
  ],
  "content": "| parameter | conditions | performance_min | performance_typ | performance_max | units |\n|---|---|---|---|---|---|\n| Sensitivity | Vcc = 5V | 0.9 | 1.0 | 1.1 | mV/G |\n| Zero Field Output | Vcc = 5V | 2450 | 2500 | 2550 | mV |\n| Linearity | Full Range | -0.5 | 0.1 | 0.5 | % |\n| Noise | 1Hz to 1kHz | - | 150 | - | uVrms |"
}
```

## Example: Rendered Output

The `MarkdownTable` object contained in the `MarkdownPage` schema also includes a `render` method that renders the table as a markdown string.

```python theme={"theme":{"light":"github-light","dark":"dark-plus"}}
for table in page.tables:
    print(f"Table [title={table.metadata.title}, caption={table.metadata.caption}]")
    print(table.render()) # renders the table as markdown format
```

### Benefits Demonstrated:

* **Nested Headers:** `Performance > Min`, `Performance > Typ`, `Performance > Max` clearly show the hierarchy under `Performance`.
* **Markdown Ready:** The `render` method returns a string that is valid GitHub-flavored markdown:
  ```markdown theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  | parameter | conditions | performance_min | performance_typ | performance_max | units |
  |---|---|---|---|---|---|
  | Sensitivity | Vcc = 5V | 0.9 | 1.0 | 1.1 | mV/G |
  | Zero Field Output | Vcc = 5V | 2450 | 2500 | 2550 | mV |
  | Linearity | Full Range | -0.5 | 0.1 | 0.5 | % |
  | Noise | 1Hz to 1kHz | - | 150 | - | uVrms |
  ```
* **Pandas Integration:** The `data` field in the `MarkdownTable` object can be easily read into a Pandas DataFrame, with appropriate header metadata such as unique `id`, `column` index, `name`, and `dtype`. We provide a convenience method `to_dataframe` to convert the `MarkdownTable` object to a Pandas DataFrame.
  ```python theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  import pandas as pd

  # Convert the extracted markdown table to a pandas dataframe
  df: pd.DataFrame = table.to_dataframe(header="id")
  print(df)
  ```

## Fine-tuning for Domain Specificity

While `vlm-1` offers strong general table extraction capabilities, optimal performance on highly specialized or uniquely formatted tables (e.g., specific financial reports, legacy scientific documents) can be achieved through fine-tuning. Consult our [fine-tuning guides](/guides/fine-tune) to adapt the model to your specific table structures and document types, maximizing accuracy and structural fidelity using the `TableWithLayout` schema.

## Try our Document -> JSON API today

Head over to our [Document -> JSON](/api-reference/v1/post-document-generate) to start building your own document processing pipeline with [VLM Run](https://vlm.run). Sign-up for access on our [platform](https://app.vlm.run).