Scrape SERP Block
Search the web using Scout’s SERP scraping capabilities
The Scrape SERP block allows users to perform web searches and retrieve data from search engine result pages (SERPs). This block is particularly useful for data collection and analysis tasks, enabling the extraction of information based on specific search terms.
Configuration (Required)
Defines the maximum number of web pages to retrieve from the search results. Adjust this limit based on the volume of data required. The default value is 1.
Specifies the maximum number of data chunks to retrieve from each web page. This helps manage the amount of data processed. The default value is 1.
The search term used to query the search engine. Ensure the term is well-defined to obtain relevant results. This input supports Jinja templating for dynamic query generation.
Outputs
The Scrape SERP block outputs a list of search results retrieved from the SERP, limited by the specified chunk and page limits. The output includes:
- Response Format: The format of the response, expected to be ‘json’.
- Cost: The cost associated with the operation.
- Elapsed Time (ms): The time taken to execute the block in milliseconds.
Usage Context
Use this block to perform web searches and retrieve search results for analysis or further processing. The block can handle dynamic queries and is designed to integrate with Scout workflows.
Best Practices
- Set appropriate limits for pages and chunks to optimize performance and avoid excessive data retrieval.
- Use Jinja templating in the query field to dynamically construct search queries based on workflow state.
- Ensure that the query is well-formed to get relevant search results.