> ## Documentation Index
> Fetch the complete documentation index at: https://agno-v2-codex-docs-audit-20260719-0149.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Multimodal inputs

> Feed images, audio, video, and PDFs into any labeling or extraction agent.

Every labeler on the other pages takes text. To label other modalities, change the input argument and the model. The schema and the `output_schema` pattern stay the same.

```python theme={null}
from typing import Literal

from agno.agent import Agent
from agno.media import Image
from agno.models.google import Gemini
from pydantic import BaseModel, Field


class Classification(BaseModel):
    label: Literal["wildlife", "landscape", "sports", "architecture", "other"] = Field(
        ..., description="The primary scene type of the image"
    )


agent = Agent(
    model=Gemini(id="gemini-3.5-flash"),
    instructions="You classify images by scene type.",
    output_schema=Classification,
)

url = "https://storage.googleapis.com/generativeai-downloads/images/generated_elephants_giraffes_zebras_sunset.jpg"
result = agent.run("Classify this image.", images=[Image(url=url)]).content
# Classification(label='wildlife')
```

## Input argument per modality

| Modality | Import                         | Argument                                    |
| -------- | ------------------------------ | ------------------------------------------- |
| Image    | `from agno.media import Image` | `images=[Image(url=...)]`                   |
| Audio    | `from agno.media import Audio` | `audio=[Audio(content=...)]`                |
| Video    | `from agno.media import Video` | `videos=[Video(content=..., format="mp4")]` |
| PDF      | `from agno.media import File`  | `files=[File(url=...)]`                     |

All four classes accept `url`, `filepath`, or raw bytes via `content`. The cookbook examples fetch audio and video bytes first. Agno installs `httpx`.

```python theme={null}
import httpx

from agno.agent import Agent
from agno.media import Audio
from agno.models.google import Gemini
from pydantic import BaseModel, Field


class Transcript(BaseModel):
    text: str = Field(..., description="Verbatim transcript of all spoken audio")


transcription_agent = Agent(
    model=Gemini(id="gemini-3.5-flash"),
    instructions="Transcribe all spoken audio. Return only the transcript.",
    output_schema=Transcript,
)

url = "https://agno-public.s3.us-east-1.amazonaws.com/demo_data/QA-01.mp3"
response = httpx.get(url, timeout=30.0)
response.raise_for_status()

result = transcription_agent.run(
    "Transcribe this audio.",
    audio=[Audio(content=response.content)],
).content
# Transcript(text='...')
```

## Bounding boxes

For region detection, return normalized coordinates so the result is resolution-independent.

```python theme={null}
from pydantic import BaseModel, Field


class BoundingBox(BaseModel):
    label: str = Field(..., description="What the box contains")
    x: float = Field(..., ge=0.0, le=1.0, description="Top-left x in [0, 1]")
    y: float = Field(..., ge=0.0, le=1.0, description="Top-left y in [0, 1]")
    width: float = Field(..., ge=0.0, le=1.0, description="Width in [0, 1]")
    height: float = Field(..., ge=0.0, le=1.0, description="Height in [0, 1]")
```

<Warning>
  The per-field `description` on `x`, `y`, `width`, and `height` is load-bearing. Without it, and without the `[0, 1]` convention spelled out in the instructions, models return degenerate boxes (all-zero or whole-image). Spell out the coordinate system in both places.
</Warning>

The [bounding boxes cookbook](https://github.com/agno-agi/agno/tree/v2.7.4/cookbook/data_labeling/_08_image_bounding_boxes) has runnable single-object, multi-object, and per-box confidence variants.

## Transcription and diarization

Audio extraction covers transcription, speaker diarization, and timestamped segments. Each is a schema change over the same API.

| Output               | Schema shape                                               |
| -------------------- | ---------------------------------------------------------- |
| Flat transcript      | `{ text: str }`                                            |
| Speaker turns        | `{ turns: List[{ speaker, text }] }`                       |
| Timestamped segments | `{ segments: List[{ start_seconds, end_seconds, text }] }` |

## Model choice

The cookbook uses `gemini-3.5-flash` across its text, image, audio, video, and PDF recipes. A replacement model must support the input modality and structured output.

## Next steps

| Task                     | Guide                                                             |
| ------------------------ | ----------------------------------------------------------------- |
| Define the output schema | [Data extraction](/use-cases/data-labeling/structured-extraction) |
| Assign labels to media   | [Classification](/use-cases/data-labeling/classification)         |
| Review media labels      | [Quality pipeline](/use-cases/data-labeling/quality-pipeline)     |

## Developer Resources

* [Image extraction cookbook](https://github.com/agno-agi/agno/tree/v2.7.4/cookbook/data_labeling/_07_image_extraction)
* [Audio transcription cookbook](https://github.com/agno-agi/agno/tree/v2.7.4/cookbook/data_labeling/_11_audio_transcription)
* [Video extraction cookbook](https://github.com/agno-agi/agno/tree/v2.7.4/cookbook/data_labeling/_14_video_extraction)
* [Document extraction cookbook](https://github.com/agno-agi/agno/tree/v2.7.4/cookbook/data_labeling/_16_document_extraction)
* [Multimodal agents](/multimodal/overview)