Use Case Overview
Template matching is a powerful computer vision technique used to find small patches of an image (templates) in a larger image. In this workflow, the user provides an image of a logo and asks the agent to find that logo in three other news articles and photos, then return the images with the logo’s location highlighted.Workflow Steps
Try it out yourself with the following prompt (in Claude Desktop or Web):Given a template image (https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTC_ps_PWPSsQ0ZeX7Zsqvtu_30qfYpdmW-0g&s) check which of the following images contains this template in the provided 3 reference images below.
- https://newsvitran.com/wp-content/uploads/2025/01/image144.jpg
- https://s.wsj.net/public/resources/images/OB-YZ407_0920ip_M_20130920103617.jpg
- https://www.apple.com/newsroom/images/2024/09/apple-introduces-iphone-16-and-iphone-16-plus/article/Apple-iPhone-16-Apple-Intelligence-240909_big.jpg.large.jpg. Once the matches are found localize the object in the image and return the image with a bounding box drawn on the image. Finally, share a link to the images with bounding boxes overlayed of the template in each image where it is present.”*
1
Load all images
Use the Response: Each call returns a unique image ID.
put_image_url
tool to load the template image and each of the three reference images into the system. This is done four times.2
Find template in reference images
Use the Response: The tool returns a JSON object detailing the matches, confidence scores, and bounding box coordinates for each reference image.
find_template
tool, providing the ID of the template and a list of the reference image IDs. A text prompt can be included to help guide the search.3
Visualize bounding boxes
For each image where a match was found, use the Response: Creates a new image with the bounding boxes rendered.
visualize_bboxes
tool with the coordinates from the find_template
response to draw the red boxes on the images.4
Generate preview links
Use the Response:
preview_object_ref
tool on the newly created annotated images to get shareable links.Example Result
The final output clearly summarizes the findings and provides links to the annotated images, showing exactly where the Apple logo was found.Build Your Own Visual Search Workflow
This template matching capability can be adapted for many powerful applications that require finding specific visual information within larger sets of data.- 🏷️ Brand Management: Scan social media and news sites to find every instance of your company’s logo.
- 🛍️ E-commerce: Allow users to upload a picture of a product they like and find visually similar items in your inventory.
- 🏭 Industrial Automation: Use a template of a perfect product to automatically spot defects or missing components on an assembly line.
- 📚 Digital Archiving: Find all documents or pages that contain a specific seal, stamp, or signature.