Web Search Reader

WebSearchReader reads data from web search results.

Parameters

Parameter	Type	Default	Description
`chunk`	`bool`	`True`	Whether to chunk the documents
`chunk_size`	`int`	`5000`	Size of each chunk when chunking is enabled
`encoding`	`Optional[str]`	`None`	Text encoding used when reading content
`search_timeout`	`int`	`10`	Timeout for search operations in seconds
`request_timeout`	`int`	`30`	Timeout for HTTP requests in seconds
`delay_between_requests`	`float`	`2.0`	Delay between requests in seconds
`max_retries`	`int`	`3`	Maximum number of retries for failed requests
`user_agent`	`str`	`"Mozilla/5.0..."`	User agent string for HTTP requests
`search_engine`	`Literal["duckduckgo"]`	`"duckduckgo"`	Search engine to use. DuckDuckGo is the only supported engine
`search_delay`	`float`	`3.0`	Delay between search requests in seconds
`max_search_retries`	`int`	`2`	Maximum retries for search operations
`rate_limit_delay`	`float`	`5.0`	Delay when rate limited in seconds
`exponential_backoff`	`bool`	`True`	Whether to use exponential backoff for retries
`chunking_strategy`	`Optional[ChunkingStrategy]`	`None`	Strategy for chunking content. When `None`, uses `SemanticChunking` with the reader's `chunk_size`
`allowed_hosts`	`Optional[List[str]]`	`None`	Hostnames the reader is allowed to fetch from. See Restricting URL Fetches.

WebSearchReader also accepts the base Reader constructor parameters, including max_results for the number of search results to read.

Searches the web for the query (up to max_results search results), reads each result page, and returns a list of documents.

Parameter	Type	Default	Description
`query`	`str`	Required	Search query to run

Async variant of read(). Accepts the same parameters.