Comparison
Apify vs Scrapy, or just run your Scrapy spider on Apify?
Scrapy is a Python framework; Apify is a platform that can host it. What a spider costs on Apify, Scrapy Cloud or your own server, and which to pick.
Scrapy is a free Python framework for writing spiders. Apify is a cloud platform for running scrapers, and one of the things it can run is your Scrapy project, through its official SDK. Apify Scrapy guide, Scrapy on PyPI.
So the real decision is where your spider should run: on Apify, on Zyte’s Scrapy Cloud, or on your own server, or whether a ready-made Apify Actor makes the spider unnecessary. Below I price one spider on all three hosts, show what changes when Scrapy runs on Apify, and pick one for each kind of team.
Short on time?
Here's my take
- Existing Scrapy spiders, want managed runs, storage and proxies
- Scrapy on Apify, Starter $19
- The SDK wraps your project as an Actor. A nightly 1 GB, one-hour spider fits Starter's $19 of usage.
- A few long-running spiders, lowest hosting bill
- Zyte Scrapy Cloud, $9 a unit
- 1 GB, one job slot, unlimited runtime. An always-on 1 GB spider is $9 a month against $199 on Apify.
- A large fleet, full control or strict data residency
- Self-hosted Scrapy (Scrapyd or containers)
- About $6–$12 a month per small server on my estimate, plus your own time, monitoring and proxies.
- Your target already has a maintained Store Actor
- An Apify Store Actor
- The Store lists 118,000+ Actors. If one covers your site and fields, there's no spider to write or maintain.
- A JavaScript or TypeScript team
- Crawlee on Apify
- Apify's own Apache-2.0 library, with browser crawlers built in. It runs anywhere, Apify included.
Prices verified . Sponsors cannot buy a verdict or a position.
This page sits in the scraping APIs hub.
How do they compare at a glance? #
Scrapy is the free framework you write spiders in; Apify is a cloud platform that runs scrapers, Scrapy projects included. Verified 2026-11-12.
| Scrapy | Apify | |
|---|---|---|
| What it is | Python framework for writing spiders | Cloud platform that runs scrapers (“Actors”) |
| Licence and cost | Free, BSD-3-Clause | Free plan ($5 usage), then $19, $199, $999 a month |
| Language | Python 3.11+ | Python (SDK) or JS/TS; Crawlee comes in both |
| JavaScript pages | Add scrapy-playwright | Browser crawlers via Crawlee or Playwright Actors |
| Scheduling | None built in; Scrapyd, cron or a host | Built-in schedules |
| Storage | Feed exports to files | Datasets, key-value stores, request queues |
| Proxies | Bring your own | Apify Proxy: 30 datacenter IPs on Starter, residential $7–$8/GB |
| Ready-made scrapers | None | 118,000+ Store Actors |
| Where it runs | Your server, Scrapy Cloud, Apify, any container host | Apify cloud |
| Lock-in | None | Low for wrapped Scrapy projects; higher for code written to Apify storage |
| Free option | Free forever, you host it | $5 of usage a month, no card |
| Suits | Python developers who want control | Teams who want hosting, storage and an API without running servers |
So what’s the actual difference? #
Scrapy gives you the crawler; Apify gives you everything around a crawler. Scrapy 2.20.0, released November 6, 2026, handles requests, parsing and pipelines, but has no job scheduler, storage service or proxy pool of its own. Scrapy releases.
Apify runs code as Actors and adds schedules, datasets, request queues, a proxy network, an API and a Store of ready-made scrapers. You pay in compute units: “1024MB memory x 1 hour = 1 CU”. Usage and resources.
Apify isn’t built on Scrapy. Its own library is Crawlee, licensed Apache-2.0, and its README says “Crawlee is open-source and runs anywhere”, so you don’t need Apify to use it. Crawlee. If you’re choosing between the two frameworks rather than hosts, read Crawlee vs Scrapy.
Can you run Scrapy on Apify? #
Yes, and it’s documented by Apify. In the folder with your scrapy.cfg, you run apify init, test locally with apify run, and deploy with apify push. CLI guide. There’s also a “python-scrapy” template if you’re starting fresh. Template.
Under the hood, the SDK installs Twisted’s asyncio reactor, applies Apify settings and swaps several Scrapy components for Apify services. SDK guide, CLI guide.
| Scrapy default | On Apify |
|---|---|
| Scheduler (memory or disk queue) | ApifyScheduler, backed by an Apify request queue |
| Item pipeline or feed export | ActorDatasetPushPipeline, writing to a dataset |
| HTTP cache on disk | ApifyCacheStorage, in a key-value store |
HttpProxyMiddleware |
ApifyHttpProxyMiddleware, using Apify Proxy |
RetryMiddleware |
ApifyRetryMiddleware |
| Ctrl-C and signals | ApifyGracefulStopExtension on a graceful abort |
The caveats are worth reading before you push:
- “It is recommended to execute only one Scrapy spider per Apify Actor.” CLI guide
- “Don’t change the Twisted reactor unless you know what you’re doing.” SDK guide
- A restarted run “crawls the start URLs again” unless you override
start(), and unfinished requests are downloaded again after a migration or abort. - Removing
apply_apify_settingsbreaks the integration, per the template.
The integration has had rough edges, and they’ve been fixed in the open. Apify’s SDK issues show a fix to “stop the crawl gracefully on migration and abort to avoid duplicate items” (closed August 31, 2026) and a scheduler timeout bug closed in October 2025. SDK issues. One small trap: the CLI page spells the requirements file both requirements_apify.txt and requirements-apify.txt.
What does one spider cost to run? #
Here’s the same spider on three hosts, at list prices verified 2026-11-12. My arithmetic; proxies, storage operations and data transfer are left out.
The assumptions: on Apify, Actor memory equals your spider’s RAM and you pay $0.20 per CU on Free and Starter or $0.16 on Scale. On Scrapy Cloud, “Every unit assigned to a job gives that job 1 computing unit, 1 GB of memory, and 2.5 GB of disk”, at $9 a unit per month. Apify pricing, Scrapy Cloud units, Scrapy Cloud pricing.
| Job | Compute a month | Apify | Zyte Scrapy Cloud | Own server (estimate) |
|---|---|---|---|---|
| Nightly, 1 GB, 1 hour | 30 CU | $19 Starter ($6 of usage; Free’s $5 runs out) | $9, one unit | $6–$12 |
| Daily, 2 GB, 4 hours | 240 CU | $48: Starter $19 + $29 over the prepaid usage | $18, two units per job | $12–$24 |
| Always-on, 1 GB | 720 CU | $199 Scale ($115.20 of usage) | $9, one unit | $6–$12 |
The always-on row is the surprise. On Starter, 720 CU would cost $144, but “Each plan has a maximum usage limit: $100 on Starter, $1,000 on Scale, and $5,000 on Business.” So the job needs Scale at $199. Subscriptions.
The daily row fits Starter only if you raise the usage limit, because by default it “matches the amount of your prepaid usage” and services are suspended when that runs out. Overage is invoiced early once it reaches $20 on Starter. Subscriptions. Billed annually, Starter is $17 and Scale $179.
The own-server column is an estimate. It uses DigitalOcean’s Basic droplets: 1 GiB for $6, 2 GiB for $12, 4 GiB for $24 a month. Whether your spider and the operating system fit in 1 GiB is an assumption, so I show the next size up too, and none of it includes your time, monitoring or backups. DigitalOcean.
The same memory doesn’t buy the same CPU. Apify gives “one full CPU core” for every 4,096 MB, so a 1 GB Actor gets a quarter of a core. Zyte’s support article lists “1x CPU” per Scrapy Cloud unit, though it dates from 2022. Usage and resources, Zyte support.
My read: raw compute is cheapest on Scrapy Cloud or a small server. Apify’s price buys the queue, datasets, bundled proxies, Store, API and scheduling around the run, so it’s worth it when you’d otherwise build those yourself. For Apify’s full plan detail, see Apify pricing.
What about proxies? #
Apify bundles them; the other two hosts make you bring your own. Apify pricing, verified 2026-11-12.
| Apify plan | Datacenter IPs included | Extra IPs | Residential per GB | Google SERP per 1,000 |
|---|---|---|---|---|
| Free | 5 | – | $8 | $2.50 |
| Starter $19 | 30 | $1 each | $8 | $2.50 |
| Scale $199 | 200 | $0.80 each | $7.50 | $2.00 |
| Business $999 | 500 | $0.60 each | $7 | $1.70 |
On the Free plan, Apify Proxy can only be used from inside the platform. On Apify, ApifyHttpProxyMiddleware routes your Scrapy requests through it. SDK guide.
On Scrapy Cloud or your own server, you buy proxies separately, or pay for Zyte API, which is billed apart from Scrapy Cloud: $0.13–$1.27 per 1,000 HTTP responses and $1.01–$16.08 per 1,000 browser responses at pay-as-you-go. Zyte pricing. My Zyte API pricing breakdown covers those tiers, and scraping API vs proxies helps if you’re unsure which to buy.
What about JavaScript-heavy sites? #
Scrapy doesn’t run JavaScript on its own. The usual add-on is scrapy-playwright, “A Scrapy download handler that performs requests using Playwright for Python”, under BSD-3-Clause. scrapy-playwright.
Apify’s Scrapy template says plainly that it “Can’t run client-side JavaScript.” Template. I found no Apify guide for running scrapy-playwright inside an Actor, so for browser-heavy work on Apify, a Crawlee or Playwright Actor is the documented route.
Wherever you run it, a browser raises the memory your spider needs, and memory is what all three hosts bill. If a browser pushes a nightly spider from 1 GB to 2 GB, the Apify usage and the Scrapy Cloud bill both double.
Scheduling, storage and scaling: what’s built in? #
Scrapy alone gives you none of the three. Every host adds some.
| Scrapy + Scrapyd (own server) | Zyte Scrapy Cloud | Apify | |
|---|---|---|---|
| Starting runs | Scrapyd’s JSON API (schedule.json); timed runs need cron |
Schedules included | Schedules included |
| Storage | Your disk | Data kept up to 120 days (7 on the free unit) | Datasets, key-value stores, request queues |
| Parallel jobs | Your servers | One job per unit; 1–6 units per job | 32 concurrent runs on Starter, 128 on Scale |
| Max memory | Your servers | 6 GB per job (6 units) | 32 GB per run; 64 GB in total on Starter, 256 GB on Scale |
Sources: Scrapyd, Scrapy Cloud pricing and units, Apify pricing, Apify usage and resources.
Scrapyd 1.6.0 is “a service for deploying and running Scrapy spiders”, controlled over a JSON API, but its docs show no time-based scheduler, so you pair it with cron. Scrapyd, scrapy.org/deploy.
On Apify, keep an eye on retention. Its storage docs say named storages are “retained indefinitely”, while unnamed storages and older runs are cleaned up automatically, so name the dataset your spider writes to if you need to keep it. Storage usage.
When does a ready-made Actor beat writing a spider? #
When a maintained Actor already covers your site and the fields you need. The Apify Store lists “118,000+ ready-to-run tools called Actors”. Apify Store. Using one means no spider code and no selector upkeep on your side.
Write your own spider when you need custom fields, very high volume, or a site nobody has covered well. Two recent Trustpilot reviews of Apify show the trade-off: one notes “scrapers can occasionally require maintenance or waiting for actor updates”, and another that “runs that return no result are still billed”. Apify’s overall Trustpilot score is 4.8 from 850 reviews. Trustpilot.
How a Store Actor is billed varies from Actor to Actor, so read its pricing before you build on it. My Apify pricing page explains the models, and Apify alternatives covers hosted options beyond Apify.
How locked in would you be? #
Not much, if you wrap an existing Scrapy project. Apify’s CLI guide says “The project remains executable as a Scrapy project”, so scrapy crawl still works outside Apify. CLI guide.
The lock-in grows with every line you write against Apify services directly: dataset pushes, request queues, key-value stores, Store Actors you chain together. Moving that to Scrapy Cloud or your own server means replacing those calls.
The licences are all permissive: Scrapy and scrapy-playwright are BSD-3-Clause, Crawlee is Apache-2.0. Scrapy, Crawlee. For the wider build-or-buy question, see web scraping API vs building your own.
Can you run a spider for free? #
For a while, on either host. Apify’s Free plan gives $5 of usage a month with no card, about 25 CU at $0.20.
That’s 25 one-hour runs at 1 GB, after which “your access to Apify’s services will be blocked until the beginning of the next monthly cycle”. A nightly 1-hour spider runs out around day 25. Apify pricing.
Scrapy Cloud’s free unit has “half as many resources as a regular unit”, jobs of “up to 1 hour” and data kept “up to 7 days”. Students get one paid unit free. Scrapy Cloud pricing. Half a unit is presumably 0.5 GB, my inference since Zyte doesn’t say, so a 1 GB spider won’t fit.
Unused Apify usage doesn’t carry over: “Unused prepaid usage does not rollover, it expires at the end of the billing cycle.” Billing.
What other pages get wrong #
Claims I checked on November 12, 2026
- “Zyte (formerly Scrapy Cloud): from $25/month” (use-apify, an Apify affiliate site, dated March 19, 2026). Zyte still sells Scrapy Cloud by that name at “$9 per month, per unit”, with a free unit. Scrapy Cloud
- “70,000+ pre-built Actors” (same page). The Apify Store now lists “118,000+”. Apify Store
- Scrapy needs Scrapy Cloud or your own infrastructure. The same page never mentions that Apify runs Scrapy projects, which its SDK guide documents.
- Apify has “Limited customization compared to open-source alternatives” (ApiX-Drive, 2024). Actors are your own code, and Scrapy projects run on Apify with settings changes only.
- Scrapy “Lacks built-in scheduling and scaling” (same page). True of the framework alone, but Scrapyd, Scrapy Cloud and Apify all add it.
Which one should you pick? #
You have Scrapy spiders and want managed runs, storage and proxies: Scrapy on Apify, Starter at $19. A nightly 1 GB spider fits the prepaid usage, and you get datasets, schedules and 30 datacenter IPs without running a server.
A few long-running spiders and the lowest hosting bill: Zyte Scrapy Cloud at $9 a unit. An always-on 1 GB spider is $9 there and $199 on Apify. Budget proxies or Zyte API on top.
A large fleet, full control or strict data residency: self-hosted Scrapy. Scrapyd or containers on servers you choose, from about $6–$12 per small server on my estimate. You pay in time instead.
Your target is already covered by a maintained Store Actor: use the Actor. Check how recently it was updated and how it’s priced before you rely on it.
A JavaScript or TypeScript team: Crawlee on Apify. It’s Apify’s own library, with browser crawlers built in, and it runs elsewhere if you leave.
FAQ #
Is Scrapy free? #
Yes. Scrapy is open source under the BSD-3-Clause licence, and version 2.20.0 (November 6, 2026) needs Python 3.11 or later. You pay only for where it runs, such as your own server, Zyte’s Scrapy Cloud at $9 per unit a month, or Apify’s compute units. PyPI.
Can I deploy a Scrapy spider to Apify? #
Yes, officially. Run apify init in the folder with scrapy.cfg, test with apify run and deploy with apify push. Apify’s SDK swaps in its own scheduler, dataset pipeline, HTTP cache, retry and proxy middleware, and recommends one spider per Actor. The project still runs with scrapy crawl. CLI guide.
Is Apify built on Scrapy? #
No. Apify’s own open-source library is Crawlee, licensed Apache-2.0. Apify supports Scrapy as a guest through its Python SDK and CLI, which wrap a Scrapy project as an Actor. Crawlee, SDK guide.
Is Scrapy Cloud cheaper than Apify? #
For raw compute, yes, on list prices. An always-on 1 GB spider is one $9 Scrapy Cloud unit against $199 on Apify’s Scale plan, because 720 compute units exceed Starter’s $100 usage ceiling. Apify’s price also covers storage, queues, bundled datacenter proxies and the Store; on Scrapy Cloud, proxies or Zyte API are billed separately. Scrapy Cloud units, Apify subscriptions.
Do I need Apify to use Crawlee? #
No. Crawlee is open source under Apache-2.0 and its README says it runs anywhere. Deploying to Apify is optional. Crawlee.
Sources
25 sources, accessed .
Official pricing, docs, changelogs and status pages 17
- Apify SDK: Scrapy guide
- Apify CLI: integrating Scrapy
- Apify python-scrapy template
- Apify pricing
- Apify subscriptions and usage limits
- Apify billing
- Apify usage and resources
- Apify storage usage
- Apify Store
- Scrapy Cloud pricing docs
- Scrapy Cloud units
- Zyte support: what is a Scrapy Cloud unit
- Scrapy Cloud product page
- Zyte API pricing
- Scrapy deployment options
- Scrapyd documentation
- DigitalOcean droplet pricing
