TwelveLabs

TwelveLabsTools enable an Agent to understand and search video using TwelveLabs. The toolkit exposes two capabilities: analyze_video, which answers a natural-language question about a video using the Pegasus video understanding model, and embed_text, which generates a multimodal embedding with the Marengo model that lives in the same latent space as TwelveLabs video, audio and image embeddings (useful for searching a video corpus by text).

Prerequisites

You need to install the twelvelabs library and an API key which can be obtained from the TwelveLabs dashboard.

uv pip install twelvelabs openai

Set the TWELVELABS_API_KEY and OPENAI_API_KEY environment variables.

export TWELVELABS_API_KEY=****
export OPENAI_API_KEY=***

Example

The following agent will use TwelveLabs to answer a question about a video and to generate a text embedding.

cookbook/91_tools/twelvelabs_tools.py

from agno.agent import Agent
from agno.tools.twelvelabs import TwelveLabsTools

# Example 1: Enable all tools
agent = Agent(
    tools=[TwelveLabsTools(all=True)],
    markdown=True,
)

agent.print_response(
    "What is happening in this video? https://interactive-examples.mdn.mozilla.net/media/cc0-videos/flower.mp4",
)

# Example 2: Enable only text embedding (useful for embedding search queries
# against a TwelveLabs video index)
embedding_agent = Agent(
    tools=[
        TwelveLabsTools(
            enable_analyze_video=False,
            enable_embed_text=True,
        )
    ],
    markdown=True,
)

embedding_agent.print_response(
    "Embed the text 'a cat playing piano' and tell me how many dimensions it has."
)

Toolkit Params

Parameter	Type	Default	Description
`api_key`	`Optional[str]`	`None`	The TwelveLabs API key. Read from the `TWELVELABS_API_KEY` environment variable if not provided.
`analyze_model`	`str`	`pegasus1.5`	The Pegasus model used for `analyze_video`.
`embed_model`	`str`	`marengo3.0`	The Marengo model used for `embed_text`.
`max_tokens`	`int`	`2048`	Maximum number of tokens for `analyze_video` responses.
`enable_analyze_video`	`bool`	`True`	Enable the `analyze_video` functionality.
`enable_embed_text`	`bool`	`True`	Enable the `embed_text` functionality.
`all`	`bool`	`False`	Enable all functionality.

Toolkit Functions

Function	Description
`analyze_video`	Analyze a video and answer a natural-language question about it using the Pegasus model.
`embed_text`	Generate a multimodal (Marengo) embedding for a piece of text, which can be used to search a video corpus by text.

Developer Resources

Trello User Control Flow

⌘I

Get Started

Basics

Advanced

Production

Providers

Other

Additional Resources

Prerequisites

Example

Toolkit Params

Toolkit Functions

Developer Resources

​Prerequisites

​Example

​Toolkit Params

​Toolkit Functions

​Developer Resources

Prerequisites

Example

Toolkit Params

Toolkit Functions

Developer Resources