from pathlib import Path
from vlmrun.client import VLMRun
client = VLMRun(api_key="<VLMRUN_API_KEY>")
response = client.audio.generate(
file=Path("<path>.mp3"),
domain="audio.transcription",
batch=True
)
from pathlib import Path
from vlmrun.client import VLMRun
from vlmrun.client.types import GenerationConfig, AgentSkill
client = VLMRun(api_key="<VLMRUN_API_KEY>")
response = client.audio.generate(
file=Path("<path>.mp3"),
domain="audio.transcription",
batch=True,
config=GenerationConfig(
skills=[AgentSkill(skill_name="<skill-name>")]
)
)
import { VlmRun } from "vlmrun";
const client = new VlmRun({apiKey: "<VLMRUN_API_KEY>"});
const fileResponse = await client.files.upload(
filePath: "<path>.mp3"
);
const response = await client.audio.generate({
fileId: fileResponse.id,
domain: "audio.transcription",
});
import { VlmRun } from "vlmrun";
const client = new VlmRun({apiKey: "<VLMRUN_API_KEY>"});
const fileResponse = await client.files.upload(
filePath: "<path>.mp3"
);
const response = await client.audio.generate({
fileId: fileResponse.id,
batch: true,
config: {
skills: [{ skillName: "<skill-name>" }],
},
});
{
"usage": {
"elements_processed": 123,
"element_type": "image",
"credits_used": 123
},
"id": "<string>",
"created_at": "2023-11-07T05:31:56Z",
"completed_at": "2023-11-07T05:31:56Z",
"response": "<see JSON response example>",
"status": "enqueued"
}
Audio → JSON
Generate structured prediction for the given audio file.
from pathlib import Path
from vlmrun.client import VLMRun
client = VLMRun(api_key="<VLMRUN_API_KEY>")
response = client.audio.generate(
file=Path("<path>.mp3"),
domain="audio.transcription",
batch=True
)
from pathlib import Path
from vlmrun.client import VLMRun
from vlmrun.client.types import GenerationConfig, AgentSkill
client = VLMRun(api_key="<VLMRUN_API_KEY>")
response = client.audio.generate(
file=Path("<path>.mp3"),
domain="audio.transcription",
batch=True,
config=GenerationConfig(
skills=[AgentSkill(skill_name="<skill-name>")]
)
)
import { VlmRun } from "vlmrun";
const client = new VlmRun({apiKey: "<VLMRUN_API_KEY>"});
const fileResponse = await client.files.upload(
filePath: "<path>.mp3"
);
const response = await client.audio.generate({
fileId: fileResponse.id,
domain: "audio.transcription",
});
import { VlmRun } from "vlmrun";
const client = new VlmRun({apiKey: "<VLMRUN_API_KEY>"});
const fileResponse = await client.files.upload(
filePath: "<path>.mp3"
);
const response = await client.audio.generate({
fileId: fileResponse.id,
batch: true,
config: {
skills: [{ skillName: "<skill-name>" }],
},
});
{
"usage": {
"elements_processed": 123,
"element_type": "image",
"credits_used": 123
},
"id": "<string>",
"created_at": "2023-11-07T05:31:56Z",
"completed_at": "2023-11-07T05:31:56Z",
"response": "<see JSON response example>",
"status": "enqueued"
}
audio domains, see the Hub Catalog.
from pathlib import Path
from vlmrun.client import VLMRun
client = VLMRun(api_key="<VLMRUN_API_KEY>")
response = client.audio.generate(
file=Path("<path>.mp3"),
domain="audio.transcription",
batch=True
)
from pathlib import Path
from vlmrun.client import VLMRun
from vlmrun.client.types import GenerationConfig, AgentSkill
client = VLMRun(api_key="<VLMRUN_API_KEY>")
response = client.audio.generate(
file=Path("<path>.mp3"),
domain="audio.transcription",
batch=True,
config=GenerationConfig(
skills=[AgentSkill(skill_name="<skill-name>")]
)
)
import { VlmRun } from "vlmrun";
const client = new VlmRun({apiKey: "<VLMRUN_API_KEY>"});
const fileResponse = await client.files.upload(
filePath: "<path>.mp3"
);
const response = await client.audio.generate({
fileId: fileResponse.id,
domain: "audio.transcription",
});
import { VlmRun } from "vlmrun";
const client = new VlmRun({apiKey: "<VLMRUN_API_KEY>"});
const fileResponse = await client.files.upload(
filePath: "<path>.mp3"
);
const response = await client.audio.generate({
fileId: fileResponse.id,
batch: true,
config: {
skills: [{ skillName: "<skill-name>" }],
},
});
Example Output
{
"metadata": {
"duration": 146.94
},
"segments": [
{
"start_time": 0,
"end_time": 24.88,
"content": " After reading tons of productivity books, I came across so many rules, like the two-year rule, the five-minute rule, the five-second rule. No, not that five second rule. The problem is that these rules were meant for companies or entrepreneurs, but I was able to adapt them to my studies during med school and drastically cut down to my procrastination. So I'm going to share with you two different two minute rules for the next two minutes. The first two minute rule comes from"
},
{
"start_time": 24.88,
"end_time": 45.5,
"content": " getting things done by David Allen. He says if it takes two minutes to do, get it done right now. For example, if I need to take out the trash today, it takes two minutes to do. So if I'm thinking about it now, might as well just do it now. Instead of writing it down on a to-do list or probably forgetting about it or having to come back to it later, which takes more than two minutes. That's how I see it."
},
{
"start_time": 45.5,
"end_time": 67.86,
"content": " So here's a list of things that might take two minutes throughout the day, like organizing your desk or watering your plants or clipping those nasty nails. I just do it when I notice it, but these little things start to add up, so this rule biases my brain towards taking action and away from procrastination. The second two-minute rule comes from atomic habits by James Clear. He says, when you're trying to do something you don't really want to do, simplify the"
},
{
"start_time": 67.86,
"end_time": 91.27,
"content": " task down to two minutes or less. So doing your entire reading assignment becomes just reading one paragraph or memorizing the entire periodic table becomes memorizing just 10 flashcards. Now, some of you might think, yeah, this is just a Jedi mind trick. Like, why would I fall for it? How is this at all sustainable? And to that, he says, when you're starting out, limit yourself to only two minutes."
},
{
"start_time": 91.27,
"end_time": 117.33,
"content": " So back in med school, I wanted to build a habit of studying for one hour every day before dinner. So I tried this trick, but I limited myself to just two minutes. I'd sit down, open my laptop, study for two minutes, and then close my laptop and went to do something else. It seems unproductive at first, right? It seems stupid. But staying consistent with this two-minute routine day after day meant that I was becoming the type of person who studies daily."
},
{
"start_time": 117.33,
"end_time": 137.99,
"content": " I was mastering the habit of just showing up because a habit needs to be established before it can be expanded upon. If I can't become a person who studies for just two minutes a day, I'd never be able to become the person that studies for an hour a day. You've got to start somewhere, but starting small is easier. There's a lot of other useful tips from books."
},
{
"start_time": 138.15,
"end_time": 146.94,
"content": " I cover more here in this video on three books and three minutes. Check it out. And if you guys like these types of videos, let me know in the comments below. I'll see you there. Bye."
}
]
}
{
"usage": {
"elements_processed": 123,
"element_type": "image",
"credits_used": 123
},
"id": "<string>",
"created_at": "2023-11-07T05:31:56Z",
"completed_at": "2023-11-07T05:31:56Z",
"response": "<see JSON response example>",
"status": "enqueued"
}
Authorizations
Bearer authentication header of the form Bearer <token>, where <token> is your auth token.
Body
Request to the Audio API (i.e. structured prediction).
The domain identifier for the model (e.g. audio.transcription).
audio.transcription, audio.transcription-summary Optional metadata to pass to the model.
Hide child attributes
Hide child attributes
The environment where the request was made.
dev, staging, prod The session ID of the request
Whether to enable logs for this request.
Whether the file can be used for training
Whether to allow retention of the data
Extra metadata for the request (e.g. dataset_id, subset_id).
The VLM generation config to be used for //generate.
Hide child attributes
Hide child attributes
Additional user instructions appended to the application or skill prompt for this request.
The detail level to use for processing multimodal data.
auto, hi, lo The overridden JSON schema to use for the model. To be used instead of the response model.
List of agent skills to enable for this generation request.
Hide child attributes
Hide child attributes
The type of the skill. Use 'skill_reference' for DB-stored skills referenced by id/name. Use 'inline' to provide the skill as a base64-encoded zip bundle.
The unique identifier of the skill — a UUID or a name string (e.g., 'pillow', 'batch-processing').
Human-readable skill name for lookup (e.g., 'invoice-extraction'). Alternative to skill_id. Deprecated in favour of skill_id.
The version of the skill — an integer (e.g. 2) or 'latest'.
DEPRECATED: Use 'skill_version' instead. The version of the skill.
Human-readable name for the inline skill (used for discovery and logging).
Short description of what the inline skill does.
Source payload for inline skills. Contains the base64-encoded zip bundle with type, media_type, and data fields.
Hide child attributes
Hide child attributes
Base64-encoded zip bundle containing the skill files.
Encoding type for the inline skill data. Currently only 'base64' is supported.
"base64"MIME type of the skill bundle. Must be 'application/zip'.
DEPRECATED: Use 'source.data' instead. Base64-encoded zip bundle containing the skill files (inline skills only).
The GraphQL statement to use for the application. If provided, the response model will be generated from the GraphQL statement.
The maximum number of retries to use for the application.
The maximum number of tokens to use for the application.
The temperature to use for the application.
Include confidence scores in the response (included in the _metadata field).
Include grounding in the response (included in the _metadata field).
Include keyframes in the video transcription response.
Duration in seconds for each video segment when chunking a video for transcription. Defaults to 150.0s.
x >= 1Number of frames to sample per video segment for captioning. Defaults to 8.
x >= 1Model ID to use for video segment captioning (e.g. 'vlmrun-orion-1:fast'). When omitted, the server default is used.
How to pass video to the captioning model: 'frames' extracts N JPEG frames per segment, 'native_video' sends the mp4 clip directly via video_url for models with native video understanding. Defaults to 'native_video' for Qwen deployment models, 'frames' for others.
frames, native_video When True, transcribe the audio track to align segment boundaries. When False (default), skip ASR and use fixed-duration video segments only (visual-only captioning).
Plain-text chat transcript (prior turns + current request) used to ground video captioning / transcription on what the user wants extracted.
0-indexed page indices to process for document files. If None, all pages are processed.
Reuse cached representations of document/video content across calls. When True (default), the file is cached after the first call so repeated queries against the same file skip re-transmitting its contents. Set to False to always send the full content.
Delivery tier for the request. 'standard'/'default' uses baseline rates, 'flex' applies a 50% discount with higher latency, 'priority' applies a 1.8x premium. When omitted (or 'auto'), the server default ('standard') applies. The chosen tier drives both billing and the latency/availability SLO.
auto, default, standard, flex, priority The URL of the file (provide either file_id or url).
The ID of the uploaded file (provide either file_id or url).
Unique identifier of the request.
Date and time when the request was created (in UTC timezone)
The URL to call when the request is completed.
1The model to use for generating the response.
vlm-1, vlm-1:auto, vlm-1:fast, vlm-1:pro Whether to process the document in batch mode (async).
Response
Successful Response