Temporal Grounding

from pathlib import Path
from vlmrun.client import VLMRun
from vlmrun.client.types import GenerationConfig

client = VLMRun(api_key="...")
response = client.video.generate(
    file=Path("path/to/episode.mp4"),
    domain="video.transcription",
)

{
  "metadata": {
    "duration": 248.67   // total duration of the video in seconds
  },
  "segments": [
    {
      "start_time": 0,   // start time of the first segment in seconds
      "end_time": 21.33, // end time of the first segment in seconds
      "audio": {
        "content": " The Keys Rocks is a rough and tumble area just outside Pittsburgh. My name is Scott Baker. My family has been connected with this community for generations. In 1941, my grandfather opened his bakery here, and he called it Jenny Lee. We used to always go to Jenny Lee's after church. If you were good in church,"
      },
      "video": {
        "content": "In this image we can see many buildings and trees. There is a bridge in the image. There are many towers in the background of the image and the sky is in white color."
      }
    },
    {
      "start_time": 21.33, // start time of the second segment in seconds
      "end_time": 42.5,    // end time of the second segment in seconds
      "audio": {
        "content": " oh, egg custard pot. Gooden Church. Oh, I ate custard pies, homemade bread that was still warm. Years later, I worked in the store. I did wedding cakes. My father, Bernie, took over after my grandfather retired. I am a baker. took over after my grandfather retired. I am a baker by name, baker by trade. I started coming in to the bakery with my dad when I was seven or eight years old."
      },
      "video": {
        "content": "In this image we can see a man standing on the floor. We can also see a group of people standing beside a table containing some food items in a cupboard. On the backside we can a wall with some photo frames and a roof with some ceiling lights."
      }
    },
    ...
    {
      "start_time": 230.67, // start time of the last segment in seconds
      "end_time": 248.67,   // end time of the last segment in seconds
      "audio": {
        "content": " I made that. I'm proud to say that. You know, You know, You know, Thank you."
      },
      "video": {
        "content": "In this image we can see a person holding a food item in his hand. In front of him there is a table. In the background of the image there are some objects."
      }
    }
  ]
}