Skip to content

Developer tools

Crawlee

Free softwareOpen sourceLocal

Crawl linked web pages and export selected text fields as structured records using JavaScript.

What you can use it for

Crawl linked web pages and export selected text fields as structured records using JavaScript.

  • Queue URLs and collect structured records with custom selectors.
  • Run HTTP crawlers locally and save results without a hosted subscription.
Workflow coverage and comparison notes

Fetch accessible HTML pages, follow links and export custom structured records using your own code and computer. Replaces basic crawling/extraction usage beyond Firecrawl and Apify free allowances, ScrapingBee subscriptions, ScraperAPI paid volume and Zyte API metering. No managed browsers, proxy network, anti-bot bypass, actor marketplace or cloud scheduling is included.

Before you switch

Requires JavaScript setup and your own compute, bandwidth and maintenance. We tested CheerioCrawler on accessible server-rendered pages. JavaScript rendering, Playwright/Puppeteer, proxies, anti-bot sites, deployment and scheduling were not tested; proxy services and cloud hosting may cost extra.

Alternatives to these paid apps

Each comparison covers the workflow described above, rather than every paid feature.