Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.vlm.run/llms.txt

Use this file to discover all available pages before exploring further.

Detect and localize keypoints of objects, people or faces in images with precise coordinate mapping. Ideal for counting, localization and salience detection.
Keypoint prediction example showing object keypoints
Object Localization
Object keypoint detection
Person Localization
Person keypoint detection
Face Localization
Face keypoint detection

​
Usage Example

For best results, we recommend using the Structured Outputs API to get responses in a structured and validated data format.
from vlmrun.client import VLMRun

# Initialize the VLMRun client
client = VLMRun(api_key="<VLMRUN_API_KEY>")

# Predict the keypoints in the image
response = client.agent.completions.create(
    model="vlmrun-orion-1:auto",
    messages=[
        {
          "role": "user",
          "content": [
            {"type": "text", "text": "Point to all the cars and doors in this image"},
            {"type": "image_url", "image_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.caption/car.jpg", "detail": "auto"}}
          ]
        }
    ],
)

print(response.choices[0].message.content)

​
FAQ

The keypoints come in the format of a list of objects with their keypoints. The keypoints are in the format of normalized xy, where x and y are the top-left corner of the keypoint. All values are between 0 and 1, and normalized by the image size. x and y are normalized by the image width and height respectively.
You can extract the following tags for each keypoint:
  • Object Name: The name of the object that the keypoint belongs to. For example, “car”, “door”, “person”, “face”, etc.
  • Confidence Score: The confidence score of the keypoint detection.