Video -> JSON

POST

video

generate

!pip install vlmrun

from pathlib import Path
from vlmrun.client import VLMRun

client = VLMRun(api_key="<VLMRUN_API_KEY>")
response = client.video.generate(
    file=Path("<path>.mp4"),
    domain="video.transcription",
    batch=True
)

{
  "usage": {
    "elements_processed": 123,
    "element_type": "image",
    "credits_used": 123
  },
  "id": "<string>",
  "created_at": "2023-11-07T05:31:56Z",
  "completed_at": "2023-11-07T05:31:56Z",
  "response": "<see JSON response example>",
  "status": "enqueued"
}

!pip install vlmrun

from pathlib import Path
from vlmrun.client import VLMRun

client = VLMRun(api_key="<VLMRUN_API_KEY>")
response = client.video.generate(
    file=Path("<path>.mp4"),
    domain="video.transcription",
    batch=True
)

{
  "usage": {
    "elements_processed": 123,
    "element_type": "image",
    "credits_used": 123
  },
  "id": "<string>",
  "created_at": "2023-11-07T05:31:56Z",
  "completed_at": "2023-11-07T05:31:56Z",
  "response": "<see JSON response example>",
  "status": "enqueued"
}

Try our Colab Cookbook Example

Try our Colab Cookbook example for long-form video transcription.

  {
      "metadata": {
          "description": null,
          "topics": null,
          "duration": 488.56
      },
      "segments": [
          {
              "start_time": 0.0,
              "end_time": 25.8,
              "audio": {
                  "content": " Like the only way to find these opportunities to learn about them is to find weirdos on the internet that are also into this thing. Yes. And they're figuring it out too. And you can kind of compare notes. Yes. And this is how new industries are created. Literally. By weirdos on the internet. Like literally. Literally. This is Dalton, plus Michael, and today we're going to talk about why AI is going to create more successful founders in the world."
              },
              "video": {
                  "content": "Two men are sitting at a table in a brightly lit room with large windows in the background. The man on the left is wearing a light gray button-up shirt and has curly hair. He is gesturing with his hands as he speaks. The man on the right is wearing a blue button-up shirt and glasses. He is smiling and listening attentively. They appear to be engaged in a conversation."
              }
          },
          ...
          {
              "start_time": 469.08,
              "end_time": 488.56,
              "audio": {
                  "content": " oh, maybe you won't need it, you won't be needed anymore, as opposed to here's a set of tools. You could do things that people couldn't think of doing affordably before. Like you could be your own boss. You don't even need to be inside of a company to create value. Yeah. So anyways, hopefully that's inspiring. Good shot. Thanks. good shot thanks"
              },
              "video": {
                  "content": "Two men are sitting at a table in an office setting. The man on the left is wearing a light gray button-up shirt and has curly hair. The man on the right is wearing a blue jacket over a black shirt and has a beard. They are engaged in a conversation, with the man in the blue jacket gesturing with his hands as he speaks. The background features large windows with blinds, allowing natural light to filter into the room."
              }
          }
      ]
  }

Authorizations

Authorization

string

header

required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json

Request to the Video API (i.e. structured prediction).

domain

enum<string>

required

The domain identifier for the model (e.g. video.transcription).

Available options:

video.transcription,

video.transcription-summary,

video.tv-news-summary,

video.dashcam

metadata

object

Optional metadata to pass to the model.

config

object

The VLM generation config to be used for /<dtype>/generate.

url

string | null

The URL of the file (provide either file_id or url).

file_id

string | null

The ID of the uploaded file (provide either file_id or url).

string

Unique identifier of the request.

created_at

string

Date and time when the request was created (in UTC timezone)

callback_url

string | null

The URL to call when the request is completed.

Minimum length: 1

model

string

default:vlm-1

The model to use for generating the response.

Allowed value: "vlm-1"

batch

boolean

default:true

Whether to process the document in batch mode (async).

Response

200

application/json

Successful Response

The response is of type any.

Audio -> JSON Get Predictions by ID

!pip install vlmrun

from pathlib import Path
from vlmrun.client import VLMRun

client = VLMRun(api_key="<VLMRUN_API_KEY>")
response = client.video.generate(
    file=Path("<path>.mp4"),
    domain="video.transcription",
    batch=True
)

{
  "usage": {
    "elements_processed": 123,
    "element_type": "image",
    "credits_used": 123
  },
  "id": "<string>",
  "created_at": "2023-11-07T05:31:56Z",
  "completed_at": "2023-11-07T05:31:56Z",
  "response": "<see JSON response example>",
  "status": "enqueued"
}

Models

Hub

Generate

Predictions

Files

OpenAI Compatibility

Try our Colab Cookbook Example

Authorizations

Body

Response