Audio Transcriptions - VLM Run

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

with open("sample.mp3", "rb") as audio_file:
    response = client.audio.transcriptions.create(
        model="nvidia/parakeet-tdt-0.6b-v3",
        file=audio_file,
    )

print(response.text)

import fs from "fs";
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.vlm.run/v1/openai",
  apiKey: process.env.VLMRUN_API_KEY,
});

const response = await client.audio.transcriptions.create({
  model: "nvidia/parakeet-tdt-0.6b-v3",
  file: fs.createReadStream("sample.mp3"),
});

console.log(response.text);

vlmrun gw transcribe sample.mp3 -m nvidia/parakeet-tdt-0.6b-v3

curl https://gateway.vlm.run/v1/openai/audio/transcriptions \
  -X POST \
  -H "Authorization: Bearer $VLMRUN_API_KEY" \
  -F "model=nvidia/parakeet-tdt-0.6b-v3" \
  -F "file=@sample.mp3"

{
  "id": "transcription-b60d04e0e241422ebb7c2b",
  "object": "audio.transcription",
  "created": 1791250916,
  "model": "nvidia/parakeet-tdt-0.6b-v3",
  "served_model_id": "nvidia/parakeet-tdt-0.6b-v3",
  "backend": "nemo",
  "text": "After reading tons of productivity books, I came across so many rules. Like the two-year rule, the five-minute rule, the five-second rule. No, not that five-second rule. The problem is",
  "language": null,
  "duration": 10.032086167800454,
  "task": "transcribe",
  "segments": null,
  "words": null,
  "usage": {
    "type": "duration",
    "input_audio_seconds": 10.032086167800454,
    "input_tokens": 0,
    "output_tokens": 30,
    "total_tokens": 30,
    "cost": 0.001
  }
}

{
  "detail": [
    {
      "type": "missing",
      "loc": ["body", "model"],
      "msg": "Field required",
      "input": null
    }
  ]
}

Omit the Authorization header for anonymous access. This route shares per-tier quotas with chat completions. See Rate Limits. Transcription models are listed on Models.

Request parameters

Send an OpenAI-compatible multipart form.

Field	Type	Default	Description
`model`	`string`	required	Model id. Example: `nvidia/parakeet-tdt-0.6b-v3`.
`file`	file upload	optional	Audio bytes. Send `file` or `url`.
`url`	`string`	optional	Hosted audio URL. Alternative to `file`.
`language`	`string`	auto-detect	Language hint. Example: `en`. The schema does not constrain the format. The convention is an ISO-639-1 code.
`prompt`	`string`	`null`	Context that biases transcription. Example: spelled-out proper nouns.
`response_format`	`string`	`json`	`json`, `verbose_json`, `text`, `srt`, or `vtt`.
`temperature`	`number`	`0.0`	Sampling temperature.
`timestamp_granularities[]`	array of strings	`null`	Accepted for OpenAI SDK compatibility. Meaningful when `response_format` is `verbose_json`. Send `segment`, `word`, or both.

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

with open("sample.mp3", "rb") as audio_file:
    response = client.audio.transcriptions.create(
        model="nvidia/parakeet-tdt-0.6b-v3",
        file=audio_file,
    )

print(response.text)

import fs from "fs";
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.vlm.run/v1/openai",
  apiKey: process.env.VLMRUN_API_KEY,
});

const response = await client.audio.transcriptions.create({
  model: "nvidia/parakeet-tdt-0.6b-v3",
  file: fs.createReadStream("sample.mp3"),
});

console.log(response.text);

vlmrun gw transcribe sample.mp3 -m nvidia/parakeet-tdt-0.6b-v3

curl https://gateway.vlm.run/v1/openai/audio/transcriptions \
  -X POST \
  -H "Authorization: Bearer $VLMRUN_API_KEY" \
  -F "model=nvidia/parakeet-tdt-0.6b-v3" \
  -F "file=@sample.mp3"

{
  "id": "transcription-b60d04e0e241422ebb7c2b",
  "object": "audio.transcription",
  "created": 1791250916,
  "model": "nvidia/parakeet-tdt-0.6b-v3",
  "served_model_id": "nvidia/parakeet-tdt-0.6b-v3",
  "backend": "nemo",
  "text": "After reading tons of productivity books, I came across so many rules. Like the two-year rule, the five-minute rule, the five-second rule. No, not that five-second rule. The problem is",
  "language": null,
  "duration": 10.032086167800454,
  "task": "transcribe",
  "segments": null,
  "words": null,
  "usage": {
    "type": "duration",
    "input_audio_seconds": 10.032086167800454,
    "input_tokens": 0,
    "output_tokens": 30,
    "total_tokens": 30,
    "cost": 0.001
  }
}

{
  "detail": [
    {
      "type": "missing",
      "loc": ["body", "model"],
      "msg": "Field required",
      "input": null
    }
  ]
}

Hosted URL instead of a file upload

import httpx

response = httpx.post(
    "https://gateway.vlm.run/v1/openai/audio/transcriptions",
    headers={"Authorization": "Bearer <VLMRUN_API_KEY>"},
    data={
        "model": "nvidia/parakeet-tdt-0.6b-v3",
        "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/audio.transcription-summary/two_minute_rules.mp3",
    },
)

print(response.json()["text"])

const form = new FormData();
form.set("model", "nvidia/parakeet-tdt-0.6b-v3");
form.set(
  "url",
  "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/audio.transcription-summary/two_minute_rules.mp3",
);

const response = await fetch(
  "https://gateway.vlm.run/v1/openai/audio/transcriptions",
  {
    method: "POST",
    headers: { Authorization: `Bearer ${process.env.VLMRUN_API_KEY}` },
    body: form,
  },
);

const { text } = await response.json();
console.log(text);

vlmrun gw transcribe -m nvidia/parakeet-tdt-0.6b-v3 \
  --url https://storage.googleapis.com/vlm-data-public-prod/hub/examples/audio.transcription-summary/two_minute_rules.mp3

curl https://gateway.vlm.run/v1/openai/audio/transcriptions \
  -X POST \
  -H "Authorization: Bearer $VLMRUN_API_KEY" \
  -F "model=nvidia/parakeet-tdt-0.6b-v3" \
  -F "url=https://storage.googleapis.com/vlm-data-public-prod/hub/examples/audio.transcription-summary/two_minute_rules.mp3"

Models

Transcription models.

Error Codes

Status codes and response bodies.