berrycrawl

A practical workflow

1. Create a workspace and API key, pick REST or SDK workflow, send first URL, and inspect result
2. Start crawl or extraction jobs asynchronously, then track status or receive webhook deliveries
3. For agent setup, create a Berrycrawl API key and add it to the server-side environment variable BERRYCRAWL_API_KEY

Based on the sources below. Editorial review does not imply hands-on product testing.

Candidates based on category and primary feature. Check feature and pricing differences before switching.

What changed

Changes to the facts recorded here, not a live scan of every vendor update. Save this tool to follow updates in your account.

1. 9/13/2026

   Updated: features, integrations, limitations, sources, summary, use cases, walkthrough

   See recorded changes

   features

   Before: ["Scrape pages and YouTube transcripts","Capture clean screenshots with cookie banner removal and lazy content loading","Map and crawl entire sites with depth controls, concurrency, deduplication, and webhooks","Search the web with advanced operators and return relevant page content in the same workflow","Extract structured data by describing the data or providing a schema, returning JSON"]

   After: ["Scrape pages and YouTube transcripts","Capture clean screenshots with cookie banner removal and tall page stitching","Map and crawl entire sites with depth controls, concurrency, deduplication, and webhooks","Search the web with advanced operators and return relevant page content","Extract structured data using JSON schema or AI extraction"]

   integrations

   Before: ["Integrates with coding agents by pasting a prompt and adding a server-side environment variable for the API key"]

   After: ["Supports REST API and SDKs with bearer token authentication"]

   limitations

   Before: ["API keys must not be exposed in frontend code or logs to maintain security","Failed scrape requests and cached ordinary pages are free and not charged, but successful YouTube transcript requests cost credits"]

   After: ["API keys must be kept server-side and never exposed in frontend code or logs","Async jobs require polling job status until COMPLETED, FAILED, or CANCELLED; do not create duplicate jobs when polling"]

   sources

   Before: [{"url":"https://berrycrawl.com/","label":"berrycrawl · Web data API for AI agents"},{"url":"https://berrycrawl.com/pricing","label":"Pricing · berrycrawl"},{"url":"https://docs.berrycrawl.com/docs/guides/agent-quickstart","label":"Agent quickstart · Berrycrawl Docs"}]

   After: [{"url":"https://berrycrawl.com/","label":"berrycrawl · Web data API for AI agents"},{"url":"https://docs.berrycrawl.com/docs/guides/agent-quickstart","label":"Agent quickstart · Berrycrawl Docs"}]

   summary

   Before: "Berrycrawl is an API platform that enables scraping pages, crawling sites, searching the web, capturing screenshots, parsing documents, and extracting structured data. It supports asynchronous jobs, webhooks, and brand profiling to turn live web content into usable data without managing browser infrastructure."

   After: "Berrycrawl provides an API platform to scrape web pages, crawl sites, search the web, capture screenshots, parse documents, and extract structured data. It supports asynchronous jobs, webhooks, and brand profiling to turn live web content into structured data for AI agents and workflows."

   useCases

   Before: ["Research agents that search, verify, and cite real sources by finding sources, scraping pages, and extracting facts","Brand enrichment by turning a company website into a usable brand profile with visual identity and social links"]

   After: ["Research agents that find sources, scrape pages, and extract facts for reasoning","Brand enrichment by turning a company website into a usable brand profile"]

   walkthrough

   Before: ["Create a workspace and API key, pick REST, MCP, or SDK workflow, send a URL, and inspect the result","For asynchronous jobs, start the job, persist the returned ID, poll the job status until completion, failure, or cancellation","Add the Berrycrawl API key to the server-side environment variable BERRYCRAWL_API_KEY and keep it out of frontend code"]

   After: ["Create a workspace and API key, pick REST or SDK workflow, send first URL, and inspect result","Start crawl or extraction jobs asynchronously, then track status or receive webhook deliveries","For agent setup, create a Berrycrawl API key and add it to the server-side environment variable BERRYCRAWL_API_KEY"]