Serp Block

Scrape search engine results pages (SERP) for web data

The Serp block enables users to perform web searches and retrieve data from search engine results pages (SERP). This block is particularly useful for data collection and analysis tasks, allowing for the extraction of information based on specific search terms.

Configuration (Required)

Web Page Limit
integer

Defines the maximum number of web pages to retrieve from the search results. Adjust this limit based on the volume of data required. The default value is 1.

Chunk Limit
integer

Specifies the maximum number of data chunks to retrieve from each web page. This helps manage the amount of data processed. The default value is 1.

Search Term
stringRequired

The search term used to query the search engine. Ensure the term is well-defined to obtain relevant results. This input supports Jinja templating for dynamic query generation.

See Workflow Logic & State > State Management for details on using dynamic variables in this block.

Outputs

The Serp block outputs a list of search results based on the query and configured limits. The output includes:

  • Response Format: JSON
  • Additional Info: Includes cost and elapsed time details.

Usage Context

Use this block to perform web searches and retrieve data from search engine results. It is ideal for workflows requiring web data extraction based on specific search terms.

Best Practices

  • Ensure the query is well-defined to get relevant search results.
  • Adjust the chunk and page result limits based on the desired scope of data extraction.
  • Handle potential HTTP errors gracefully to ensure robust data retrieval.