> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vlm.run/llms.txt
> Use this file to discover all available pages before exploring further.

# geopavlakos/hamer

> 3D hand mesh recovery: 21 hand keypoints in both the image plane and 3D.

Transformer-based 3D hand mesh recovery, with ViTPose-H detection finding the
hands first. Accepts one `image_url`; no text-only input, no `document_url`,
and no video. `pose` is the only method.

Each detected hand returns a `left_hand` or `right_hand` label, a detection
score, a normalized bounding box, and 21 MANO keypoints twice over: `kpts_xy`
in the image plane and `kpts_xyz` in 3D. That is why the payload is tagged
`world.pose.kpts` rather than `img.pose.kpts`, which is what the 2D-only
[ViTPose](/gateway/models/usyd-community-vitpose-plus-large) returns.

## Method parameters

| Parameter        | Default | Description                                                      |
| ---------------- | ------: | ---------------------------------------------------------------- |
| `body_conf`      |   `0.5` | Minimum confidence for the body detection that locates the hands |
| `hand_conf`      |   `0.5` | Minimum confidence for a hand to be reported                     |
| `rescale_factor` |   `2.0` | How far the detected hand box is expanded before the mesh is fit |
| `include_mesh`   | `false` | Return the full MANO mesh alongside the keypoints                |

## Output by method

| Method           | Payload kind | Image `content.object` | Payload                                                                                    |
| ---------------- | ------------ | ---------------------- | ------------------------------------------------------------------------------------------ |
| `pose` (default) | json         | `world.pose.kpts`      | One entry per detected hand, with `bbox_xywh`, `label`, `score`, `kpts_xy`, and `kpts_xyz` |

## Request

<CodeGroup>
  ```python Python [expandable] theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  from openai import OpenAI

  client = OpenAI(
      base_url="https://gateway.vlm.run/v1/openai",
      api_key="<VLMRUN_API_KEY>",
  )

  response = client.chat.completions.create(
      model="geopavlakos/hamer",
      messages=[
          {
              "role": "user",
              "content": [
                  {
                      "type": "image_url",
                      "image_url": {
                          "url": "http://images.cocodataset.org/val2017/000000000785.jpg"
                      },
                  },
              ],
          }
      ],
      extra_body={"method": "pose", "method_params": {"hand_conf": 0.5}},
  )

  print(response.choices[0].message.content)
  ```

  ```typescript Node.js [expandable] theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  import OpenAI from "openai";

  const client = new OpenAI({
    baseURL: "https://gateway.vlm.run/v1/openai",
    apiKey: process.env.VLMRUN_API_KEY,
  });

  const response = await client.chat.completions.create({
    model: "geopavlakos/hamer",
    messages: [
      {
        role: "user",
        content: [
          {
            type: "image_url",
            image_url: {
              url: "http://images.cocodataset.org/val2017/000000000785.jpg",
            },
          },
        ],
      },
    ],
    method: "pose",
    method_params: { hand_conf: 0.5 },
  });

  console.log(response.choices[0].message.content);
  ```

  ```bash CLI theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  vlmrun gw chat http://images.cocodataset.org/val2017/000000000785.jpg \
    -m geopavlakos/hamer --method pose
  ```

  ```bash cURL [expandable] theme={"theme":{"light":"github-light","dark":"dark-plus"}}
  curl https://gateway.vlm.run/v1/openai/chat/completions \
    -X POST \
    -H "Authorization: Bearer $VLMRUN_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "geopavlakos/hamer",
      "method": "pose",
      "messages": [
        {
          "role": "user",
          "content": [
            {
              "type": "image_url",
              "image_url": {
                "url": "http://images.cocodataset.org/val2017/000000000785.jpg"
              }
            }
          ]
        }
      ]
    }'
  ```
</CodeGroup>

## Response

<Tabs>
  <Tab title="Text mode">
    A single image returns the json block alone, with no wrapper:

    ```json theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    {
      "object": "world.pose.kpts",
      "items": [
        {
          "bbox_xywh": [0.6735, 0.3556, 0.0623, 0.0938],
          "label": "left_hand",
          "score": 1.0,
          "kpts_xy": [[0.695, 0.3822], [0.6881, 0.3928], "..."],
          "kpts_xyz": [[-0.0957, 0.0064, 0.0062], [-0.1146, 0.0257, -0.0211], "..."]
        }
      ],
      "kpts_labels": ["wrist", "thumb_mcp", "thumb_pip", "thumb_dip", "thumb_tip", "..."]
    }
    ```
  </Tab>

  <Tab title="JSON mode">
    The flat reply, with the image's details beside the payload:

    ```json theme={"theme":{"light":"github-light","dark":"dark-plus"}}
    {
      "model": "geopavlakos/hamer",
      "method": "pose",
      "image_hash": "sha256:1433c58c...",
      "image_width": 640,
      "image_height": 425,
      "content": {
        "object": "world.pose.kpts",
        "items": [
          {
            "bbox_xywh": [0.6735, 0.3556, 0.0623, 0.0938],
            "label": "left_hand",
            "score": 1.0,
            "kpts_xy": [[0.695, 0.3822], [0.6881, 0.3928], "..."],
            "kpts_xyz": [[-0.0957, 0.0064, 0.0062], [-0.1146, 0.0257, -0.0211], "..."]
          }
        ],
        "kpts_labels": ["wrist", "thumb_mcp", "thumb_pip", "thumb_dip", "thumb_tip", "..."]
      }
    }
    ```
  </Tab>
</Tabs>

`label` is the hand's laterality, `left_hand` or `right_hand`, and `score` is
the detection confidence.

`kpts_xy` is normalized against the source image, so multiply by `image_width`
and `image_height` for pixels. `kpts_xyz` is metric 3D in the model's own
hand-centred frame and is not tied to image coordinates; it is the reason the
payload is tagged `world.pose.kpts` rather than the `img.pose.kpts` that
[ViTPose](/gateway/models/usyd-community-vitpose-plus-large) returns. Both lists
hold the same 21 MANO joints in the same order, named once on the container by
`kpts_labels`: the wrist, then the thumb, index, middle, ring and pinky
fingers, four joints each from base to tip (`_mcp`, `_pip`, `_dip`, `_tip`).

`method_params.include_mesh` adds the full MANO mesh beside the keypoints.