Exa Webpage Contents Block
Execute Exa content search and scraping
The Exa Webpage Contents block enables users to perform content scraping on specified URLs using the Exa API. It supports options for live crawling, retrieving full page text, and crawling subpages, making it useful for gathering webpage contents for analysis or processing within workflows.
Configuration (Required)
The API key required to authenticate with the Exa service.
A comma-separated list of URLs to be crawled. This field supports Jinja templating for dynamic URL generation.
Whether to return the full page text from the crawled URLs. The default value is true.
Option to control live crawling behavior. Choices are ‘never’, ‘fallback’, ‘always’, or ‘auto’. The default value is never. This field supports Jinja templating for dynamic option selection.
The timeout duration for live crawling in milliseconds. The default value is 10000.
The number of subpages to crawl for each URL. The default value is 0.
A keyword to target specific subpages during crawling. This field supports Jinja templating for dynamic keyword targeting.
Outputs
The output is a JSON object containing the results of the content scraping, including any specified subpages and full page text if requested.
Usage Context
Use this block to scrape webpage contents for analysis or processing within workflows. It is particularly useful when needing to gather data from multiple URLs efficiently.
Best Practices
- Ensure your Exa API Key is valid and has the necessary permissions.
- Use Jinja templating to dynamically generate URLs and options based on workflow state.
- Set appropriate livecrawl options and timeouts to balance between speed and data freshness.
- Consider the number of subpages and target keywords to optimize the scraping process.