Understanding TV News

VLM-1 can also extract structured data from TV broadcasts. Here’s an example of a TV news broadcast and the structured JSON output that VLM-1 can extract.

The structured JSON response from VLM-1 is as follows:

{
"id": "...",
"created_at": "...",
"completed_at": "...",
"status": "completed",
"response": {
  "description": "A TV broadcast screenshot showing a live market check with the latest figures for major stock indexes and a news reporter discussing them.",
  "chyron": "SPX CLIMBS AFTER LONGEST STREAK OF WEEKLY LOSSES",
  "tables": [
    {
      "description": "Market check displaying the last close values for major indexes including S&P 500, Nasdaq 100, Dow Jones, and Russell 2000 Index.",
      "title": "MARKET CHECK",
      "caption": null,
      "markdown": "| Index                | Value     | Change  | Percentage |\n|----------------------|-----------|---------|------------|\n| S&P 500              | 4,399.77  | ▲ 30.06  | 0.69%      |\n| Nasdaq 100           | 14,936.69 | ▲ 241.85 | 1.65%      |\n| Dow Jones            | 34,463.69 | ▼ 36.97  | 0.11%      |\n| Russell 2000 Index   | 1,856.01  | ▼ 3.41   | 0.18%      |"
    }
  ],
  "charts": null,
  "reporters": null,
  "speakers": [
    {
      "name": "Alexandra Semenova",
      "is_name_shown": true
    }
  ],
  "quotes": null,
  "others": null
}
}

​

Notebook Example

​

Key Features

Let’s take a look at some of the key features of the API for analyzing TV news broadcasts.

​

1. Extracting Chyrons and Scene Descriptions

VLM-1 can extract chyrons and scene descriptions from TV news broadcasts. This can be useful for summarizing the key points of the news segment, or for extracting structured data from the broadcast. As you can see in the sample output below, the API is able to extract the chyron text (“SPX CLIMBS AFTER LONGEST STREAK OF WEEKLY LOSSES”) and a general visual description of the scene with the latest closing values and changes for S&P 500, NASDAQ 100, Dow Jones, and Russell 2000 Index.

{
  "description": "A TV broadcast screenshot showing a live market check with the latest figures for major stock indexes and a news reporter discussing them.",
  "chyron": "SPX CLIMBS AFTER LONGEST STREAK OF WEEKLY LOSSES",
}

​

2. Extracting Detailed Table and Plot Information

VLM-1 can also extract detailed table and plot information from TV news broadcasts. This can be useful for analyzing the data presented in the broadcast, or for extracting structured data from the broadcast. As you can see in the sample output below, the API is able to extract a detailed table with the last close values and changes for S&P 500, NASDAQ 100, Dow Jones, and Russell 2000 Index.

{
  "tables": [
    {
      "description": "Market check displaying the last close values for major indexes including S&P 500, Nasdaq 100, Dow Jones, and Russell 2000 Index.",
      "title": "MARKET CHECK",
      "caption": null,
      "markdown": "| Index                | Value     | Change  | Percentage |\n|----------------------|-----------|---------|------------|\n| S&P 500              | 4,399.77  |  30.06  | 0.69%      |\n| NASDAQ 100           | 14,936.69 |  241.85 | 1.65%      |\n| DOW JONES            | 34,463.69 | -36.97  | -0.11%      |\n| RUSSELL 2000 INDEX   | 1,856.01  | -3.41   | -0.18%      |"
    }
  ],
  ...
}

​

3. Mining Speaker Information

VLM-1 can also extract speaker information from TV news broadcasts. This can be useful for identifying the speakers in the broadcast, or for analyzing the sentiment and tone of the speakers. As you can see in the sample output below, the API is able to extract the name of the news reporter (“Alexandra Semenova”) and whether the name is shown on the screen.

{
  "reporters": [
    {
      "name": "Alexandra Semenova",
      "is_name_shown": true
    }
  ]
}