Skip to main content
Create precise pixel-level segmentation masks for objects, regions, and features in images. Perfect for medical imaging, autonomous driving, photo editing, and augmented reality applications.

​
Object Segmentation

Segmentation of cars with their masks overlaid

​
Face Segmentation

Segmentation of individual faces with their masks overlaid

​
Usage Example

For best results, we recommend using the Structured Outputs API to get responses in a structured and validated data format.
from vlmrun.client import VLMRun

# Initialize the VLMRun client
client = VLMRun(api_key="<VLMRUN_API_KEY>")

# Segment objects in the image
response = client.agent.completions.create(
    model="vlmrun-orion-1:auto",
    messages=[
        {
          "role": "user",
          "content": [
            {"type": "text", "text": "Segment all the cars in this image"},
            {"type": "image_url", "image_url": {"url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.object-detection/nascar.jpg", "detail": "auto"}}
          ]
        }
    ],
)

print(response.choices[0].message.content)

​
FAQ

  • Instance Segmentation: Segment individual objects with unique masks
  • Semantic Segmentation: Classify pixels by category or class
  • Panoptic Segmentation: Combine instance and semantic segmentation
The segmentation masks come in the format of a list of objects with their masks. The masks are in PNG format that can be retrieved as a pre-signed URL, per object instance.
Orion supports promptable segmentation, where you can describe in natural language the object(s) you want to segment (e.g., “segment the writing-related items in this image”). For best results, clearly describe the object(s) of interest. Below are some common examples of classes that can be segmented:Common Objects
  • People: person, face, hand, foot
  • Vehicles: car, truck, bus, motorcycle, bicycle
  • Animals: dog, cat, bird, horse, cow, sheep
  • Furniture: chair, table, bed, sofa, desk
  • Electronics: laptop, phone, tv, keyboard, mouse
Specialized Categories
  • Medical: organ, tissue, lesion, bone
  • Nature: tree, grass, sky, water, mountain
  • Indoor: wall, floor, ceiling, door, window
  • Outdoor: road, sidewalk, building, sign, traffic_light
PNG Masks
  • Binary or grayscale images where each pixel value represents a segment ID
  • Compatible with most image editing software
  • Small file size for simple segmentations
Other Formats (Coming soon!): JSON Polygons and COCO Format