Skip to content

Developer tools

Scrapy

Free softwareOpen sourceLocal

Follow website links and export selected page fields as structured datasets with Python spiders.

What you can use it for

Follow website links and export selected page fields as structured datasets with Python spiders.

  • Write reusable spiders with selectors, pagination and export pipelines.
  • Save extracted records directly to JSON without a managed scraping plan.
Workflow coverage and comparison notes

Fetch accessible HTML pages, follow links and export custom structured records using your own code and computer. Replaces basic crawling/extraction usage beyond Firecrawl and Apify free allowances, ScrapingBee subscriptions, ScraperAPI paid volume and Zyte API metering. No managed browsers, proxy network, anti-bot bypass, actor marketplace or cloud scheduling is included.

Before you switch

Requires Python code, installation and your own compute, bandwidth and maintenance. Tested ordinary HTML pages only. No managed proxies, CAPTCHA handling, hosted actors or cloud scheduler is supplied. Dynamic sites can need separate browser tooling; infrastructure and proxy services may cost extra.

Alternatives to these paid apps

Each comparison covers the workflow described above, rather than every paid feature.