geopavlakos/hamer

Transformer-based 3D hand mesh recovery, with ViTPose-H detection finding the hands first. Accepts one image_url; no text-only input, no document_url, and no video. pose is the only method. Each detected hand returns a left_hand or right_hand label, a detection score, a normalized bounding box, and 21 MANO keypoints twice over: kpts_xy in the image plane and kpts_xyz in 3D. That is why the payload is tagged world.pose.kpts rather than img.pose.kpts, which is what the 2D-only ViTPose returns.

Method parameters

Parameter	Default	Description
`body_conf`	`0.5`	Minimum confidence for the body detection that locates the hands
`hand_conf`	`0.5`	Minimum confidence for a hand to be reported
`rescale_factor`	`2.0`	How far the detected hand box is expanded before the mesh is fit
`include_mesh`	`false`	Return the full MANO mesh alongside the keypoints

Output by method

Method	Payload kind	Image `content.object`	Payload
`pose` (default)	json	`world.pose.kpts`	One entry per detected hand, with `bbox_xywh`, `label`, `score`, `kpts_xy`, and `kpts_xyz`

Request

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

response = client.chat.completions.create(
    model="geopavlakos/hamer",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "http://images.cocodataset.org/val2017/000000000785.jpg"
                    },
                },
            ],
        }
    ],
    extra_body={"method": "pose", "method_params": {"hand_conf": 0.5}},
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.vlm.run/v1/openai",
  apiKey: process.env.VLMRUN_API_KEY,
});

const response = await client.chat.completions.create({
  model: "geopavlakos/hamer",
  messages: [
    {
      role: "user",
      content: [
        {
          type: "image_url",
          image_url: {
            url: "http://images.cocodataset.org/val2017/000000000785.jpg",
          },
        },
      ],
    },
  ],
  method: "pose",
  method_params: { hand_conf: 0.5 },
});

console.log(response.choices[0].message.content);

vlmrun gw chat http://images.cocodataset.org/val2017/000000000785.jpg \
  -m geopavlakos/hamer --method pose

curl https://gateway.vlm.run/v1/openai/chat/completions \
  -X POST \
  -H "Authorization: Bearer $VLMRUN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "geopavlakos/hamer",
    "method": "pose",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "image_url",
            "image_url": {
              "url": "http://images.cocodataset.org/val2017/000000000785.jpg"
            }
          }
        ]
      }
    ]
  }'

Response

Text mode
JSON mode

A single image returns the json block alone, with no wrapper:

{
  "object": "world.pose.kpts",
  "items": [
    {
      "bbox_xywh": [0.6735, 0.3556, 0.0623, 0.0938],
      "label": "left_hand",
      "score": 1.0,
      "kpts_xy": [[0.695, 0.3822], [0.6881, 0.3928], "..."],
      "kpts_xyz": [[-0.0957, 0.0064, 0.0062], [-0.1146, 0.0257, -0.0211], "..."]
    }
  ],
  "kpts_labels": ["wrist", "thumb_mcp", "thumb_pip", "thumb_dip", "thumb_tip", "..."]
}

The flat reply, with the image’s details beside the payload:

{
  "model": "geopavlakos/hamer",
  "method": "pose",
  "image_hash": "sha256:1433c58c...",
  "image_width": 640,
  "image_height": 425,
  "content": {
    "object": "world.pose.kpts",
    "items": [
      {
        "bbox_xywh": [0.6735, 0.3556, 0.0623, 0.0938],
        "label": "left_hand",
        "score": 1.0,
        "kpts_xy": [[0.695, 0.3822], [0.6881, 0.3928], "..."],
        "kpts_xyz": [[-0.0957, 0.0064, 0.0062], [-0.1146, 0.0257, -0.0211], "..."]
      }
    ],
    "kpts_labels": ["wrist", "thumb_mcp", "thumb_pip", "thumb_dip", "thumb_tip", "..."]
  }
}

label is the hand’s laterality, left_hand or right_hand, and score is the detection confidence. kpts_xy is normalized against the source image, so multiply by image_width and image_height for pixels. kpts_xyz is metric 3D in the model’s own hand-centred frame and is not tied to image coordinates; it is the reason the payload is tagged world.pose.kpts rather than the img.pose.kpts that ViTPose returns. Both lists hold the same 21 MANO joints in the same order, named once on the container by kpts_labels: the wrist, then the thumb, index, middle, ring and pinky fingers, four joints each from base to tip (_mcp, _pip, _dip, _tip). method_params.include_mesh adds the full MANO mesh beside the keypoints.

Get Started

OpenAI-Compatible API

Guides

API Reference

Tools

Platform

geopavlakos/hamer

Method parameters

Output by method

Request

Response

​Method parameters

​Output by method

​Request

​Response

Method parameters

Output by method

Request

Response