Skip to content
CrawlfolioSourced research on web data for AI
Menu

You’ve shortlisted two to four scraping APIs, and each one hands out free credits. The decision is which one gets your pages right at a cost per good record you can live with, and the only fair way to find out is a proof of concept you design yourself.

This page is that design: which URLs to test, what each vendor bills, how far the free credits go, how many URLs you need before a gap is real, and a scoring sheet with a decision rule. It’s a plan you run on your own pages, so none of the numbers below are results of mine.

Short on time?

Short on time? Here's my take.

Biggest free allowance for JavaScript-heavy pages
ZenRows
5,000 credits every month with no card. It renews monthly, so it isn't a trial. That's 1,000 JavaScript pages or 200 JavaScript-plus-premium pages a month, at 5 concurrent requests.
A full trial on a deadline
ScraperAPI
7 days and 5,000 credits, no card: 500 rendered pages or 200 premium-plus-rendered ones. Clean dead URLs out first, because it bills 404s.
Failures must cost nothing
Zyte
Unsuccessful and rate-limited responses are free, and the price follows each site's difficulty tier. One catch: Zyte treats a target site's 404 as a successful response. $5 of credit to start.
Seeing what every request cost
Scrapfly (or Scrape.do)
Scrapfly returns the cost in an X-Scrapfly-Api-Cost header and lets you cap each request with cost_budget. Scrape.do's Scrape.do-Request-Cost header is the alternative, and it calls that header authoritative for billing.
Plain pages only
Scrapingdog
100 free credits and 1 concurrent request buy just 4 JavaScript-plus-premium pages. Fine for checking plain pages, too small for protected ones.

Prices verified . Sponsors cannot buy a verdict or a position.

To test a web scraping API, pick 30 to 100 of your own URLs by page type, write down the fields each should return, and run them on each vendor’s free credits, cheapest setting first. Among vendors whose lower-bound pass rate clears your floor, pick the lowest cost per good record.

This page sits in the scraping API hub. If you haven’t shortlisted yet, start with the best web scraping APIs; if you’re weighing free tiers against trials, free web scraping APIs covers that.

Which of your URLs should go in the test? #

Your own, sampled by page type, 30 to 100 in total. A vendor that clears a listing page can still fail on the product page behind it, so a single bucket of “URLs from the site” hides the failure you care about.

Sort your targets into the page types you’ll actually scrape in production:

  • Listing or category pages, where pagination and lazy loading show up.
  • Detail pages (product, job, property), where the fields you pay for live.
  • Search result pages on the target site, which are often the most protected.
  • JavaScript-heavy pages, where the data only appears after scripts run. Check whether a page needs rendering before you count it here.
  • Geo-locked pages, which need a country setting.
  • Known-dead URLs, kept as a separate bucket, because several vendors bill a 404.

Add a few practice pages as a control. toscrape.com calls itself “a safe place for beginners learning web scraping and for developers validating their scraping technologies”, and its sandboxes include JavaScript-generated content, delayed loading, infinite scroll, a login with a CSRF token and a ViewState filter form.

If a vendor fails there, the problem is your setup, not the target.

Can the free credits cover it? #

Not if you send every URL at the most expensive setting. 100 URLs run three times is 300 requests, and at a weight of 25 credits per request (JavaScript plus premium proxies at most vendors) that’s 7,500 credits, more than any credit-based free allowance on this page (my arithmetic).

So cut to 30 to 50 URLs per vendor, or escalate settings only for URLs that fail the cheaper one. The ladder is further down.

I’d run the sample twice, a day apart, at the concurrency you’d use in production. The MacMyths guide only says a pilot should be “long enough” to cover normal variation, without a number (MacMyths); two runs on different days is the cheapest version of that I can defend.

What counts as a successful scrape? #

A record with the right fields in it, not an HTTP 200. A block or challenge page can come back with a 200, and every published benchmark I read adds a content check for that reason.

ZenRows’ own benchmark counts a success as a 200 plus a valid page title (ZenRows). Proxyway validated by response code, page size and title (Proxyway). AIMultiple defines success as “correct text presence on extracted page” (AIMultiple).

For a buying decision, go one step further and check fields. Before the trial, open each URL in a normal browser, write down the 2 to 5 values you need (title, price, stock, date), and save that as your answer key.

Pass check for each URL

  1. The response came back before your timeout.
  2. The page is the real page, not a challenge, login wall or “access denied” page.
  3. Every field on your answer key is present.
  4. Each field matches the value you wrote down (allow for prices that change between runs).
  5. The page type’s layout parsed with the same selectors as the others in its bucket.

Pass = all five. Anything else is a fail, whatever the status code said.

MacMyths splits this into page success, field completeness and accuracy per domain (MacMyths). That split is useful later, because “the page loaded but the price was missing” points to a parsing problem, not a blocking one.

What will each vendor bill you for? #

Not only successes, and the differences change your cost per good record. Most vendors don’t bill server errors, but several bill a target’s 404, and one bills anything below 500 when you ask for the target’s status code.

Here’s what each vendor says it bills, from its own docs:

Vendor What bills What doesn’t How to see each request’s cost
ScraperAPI 200 and 404; requests you cancel before its 70-second window is up 500 (“Request failed. You will not be charged”) sa-credit-cost header on every response; account/urlcost estimates a URL before you scrape it
ScrapingBee 200, 404, 410, 413; with the transparent status option, “Anything the target returns below 500 is billed” 500 cost field with json_response; Spb-auto-cost in Auto-Mode
ZenRows “Only successful requests are billed”; 404 and 410 count as successful Other failures No cost header in the pages I read; 429 when over your concurrency
Scrapfly Successful scrapes, plus target statuses its failure protection doesn’t cover: 400 to 418 (except 403 and 408), 422, 424, 426, 428 and 456 Other failed scrapes (“Scrape Failed Protection”) X-Scrapfly-Api-Cost header, context.cost breakdown, cost_budget cap
Zyte Successful responses; a target’s 404 is a successful Zyte response “Rate-limiting and unsuccessful responses are free” Cost estimator at app.zyte.com
Scrape.do 2XX, 400, 404 and 410 count as successful Timeouts, blocks and errors Scrape.do-Request-Cost header, which it calls authoritative for billing
Scrapingdog Successful requests Failed, blocked and timed-out requests are “automatically refunded” No status list published, so 404 handling is unclear

Sources, all verified 2026-11-16: ScraperAPI credits and status codes; ScrapingBee docs; ZenRows pricing; Scrapfly billing; Zyte pricing docs and errors; Scrape.do request costs; Scrapingdog pricing.

Three details are easy to miss. Scrapfly’s failure protection can switch off if 30% or more of your scrapes in 24 hours (with at least 1,000 scrapes) are blocked without its Unblocker, so a sloppy test can lose it (Scrapfly billing). Scrape.do lets you claim back a 200 whose content was invalid through support (Scrape.do pricing).

And for Zyte, the docs use toscrape.com/not-found as the example: the target returns 404, and that counts as a successful Zyte response. The docs don’t say in so many words that it’s billed, so I’d expect it to bill and keep dead URLs out of the main sample (Zyte errors).

Of the two vendors outside the table, Bright Data’s Web Unlocker says “Pay only for success” (Bright Data), and Oxylabs’ pricing page doesn’t say whether failed results bill (Oxylabs). The full set of credit weights per feature is in how scraping API credits work.

A wall of flat pale square tiles in a regular grid with thin grout lines, the rows bowing slightly and the lower corners falling into shadow
From a distance every square looks the same, and so do requests, until a billing page tells you which responses it counts against your credits. Photo: Kin Shing Lai / Unsplash · Unsplash License

How far do the free credits go? #

Among the credit-weighted plans, furthest at ZenRows and ScraperAPI, and not far at all at Scrapingdog. Bright Data’s flat 5K requests and Zyte’s $5 work differently, so I’ve converted them too. The table turns each allowance into pages at the settings a hard target needs (my arithmetic: allowance divided by the credit weight, rounded down).

Vendor Free allowance Renews or expires Card Concurrency on the free tier JavaScript pages JavaScript + premium pages
ZenRows 5,000 credits Every month, no rollover Not needed 5 1,000 200
ScraperAPI 5,000 credits 7-day trial Not needed Free plan: 5 (trial not stated separately) 500 200
Zyte $5 credit First billing month (“30 days” on the pricing page) Not stated Not stated ~4,950 (tier 1) to ~310 (tier 5) Priced by site tier
Bright Data Web Unlocker 5K requests Every month Not stated Not stated 5,000 requests at one flat price Not split out
Oxylabs $5 toward any product Not stated Not stated Not stated 3,333 Not split out
Scrape.do 1,000 successful credits Listed as monthly; expiry not stated Not needed 5 200 40
ScrapingBee 1,000 credits Not stated Not needed Not stated 200 40
Scrapfly 1,000 credits, hard cap One-time, “no time limit” Not needed 5 166 (datacenter) 33 (residential)
Scrapingdog 100 credits Not stated Not needed 1 20 4

Sources, verified 2026-11-16: ZenRows, ScraperAPI, Zyte, Zyte docs, Bright Data, Oxylabs, Scrape.do, ScrapingBee, Scrapfly and its billing docs, Scrapingdog. Page counts are my arithmetic.

How I got the less obvious rows:

  • ZenRows: 5,000 ÷ 5 = 1,000 rendered pages; 5,000 ÷ 25 = 200 with premium proxies too. Its free plan is $0 every month, not a time-boxed trial, so you can repeat a test next month (ZenRows).
  • ScraperAPI: the page says “7-day trial and 5,000 API credits. No credit card required.” Rendering costs 10 and premium plus rendering 25, so 500 and 200 pages. Ultra premium (30, or 75 with rendering) is for paid plans only, so you can’t test it on the trial (ScraperAPI credits).
  • Zyte: $5 ÷ $1.01 per 1,000 browser requests on site tier 1 = about 4,950; ÷ $16.08 on tier 5 = about 310; ÷ $0.13 for plain HTTP on tier 1 = about 38,460 (Zyte pricing).
  • Scrapfly: a datacenter request is 1 credit plus 5 for rendering (6), a residential one 25 plus 5 (30), so 1,000 ÷ 6 = 166 and 1,000 ÷ 30 = 33 (Scrapfly billing).
  • Oxylabs: $5 ÷ $1.50 per 1,000 rendered results = 3,333, or 5,000 without rendering at $1.00 (Oxylabs).
  • Scrape.do: its free credits are “1,000 Successful API Credits”, so failed requests don’t eat them, but some domains carry their own cost: google.* is 10, linkedin.com 30, shopee.* 100 and sainsburys.co.uk 200 (Scrape.do costs).

Two gaps are worth saying plainly. ScrapingBee, Scrape.do and Scrapingdog don’t say when their free credits expire, so plan to finish your test in one sitting. Scrapfly is the exception: its pricing page describes the 1,000 signup credits as “no credit card, no time limit” (Scrapfly). ScraperAPI shows a 7-day, 5,000-credit trial and, in the same page’s FAQ, a free plan with 1,000 credits and 5 concurrent connections, without saying which you’re on after day 7 (ScraperAPI).

What does a credit cost once you pay? #

You need this to turn billed credits into dollars per good record. Here’s each vendor’s entry plan, divided out (my arithmetic: plan price ÷ credits × weight × 1,000, verified 2026-11-16).

Vendor, entry plan Per credit Per 1,000 plain Per 1,000 JavaScript Per 1,000 JavaScript + premium
ScraperAPI Hobby, $49 for 100,000 $0.00049 $0.49 $4.90 $12.25
ZenRows Build, $19 for 45,000 $0.000422 $0.42 $2.11 $10.56
Scrapingdog Lite, $40 for 200,000 $0.0002 $0.20 $1.00 $5.00
ScrapingBee Freelance, $49 for 250,000 $0.000196 $0.20 $0.98 $4.90
Scrapfly Discovery, $30 for 200,000 $0.00015 $0.15 $0.90 $4.50
Scrape.do Hobby, $29 for 250,000 $0.000116 $0.12 $0.58 $2.90
Zyte, pay as you go Priced by site tier $0.13 to $1.27 $1.01 to $16.08 Set by tier
Bright Data Web Unlocker, pay as you go Per request $1.50 flat $1.50 flat $1.50 flat

ScrapingBee’s Freelance plan is its second tier; the table uses it because its $19 Hobby plan has 75,000 credits, which works out to a higher rate per credit. Plan-by-plan detail lives in each vendor’s pricing post, linked from the hub.

Spend credits only where a page needs them #

Start every URL at the cheapest setting and move only the failures up a rung. On a ZenRows-style scale, a plain request costs 1 credit, JavaScript 5, premium proxies 10 and both together 25 (ZenRows), so sending everything at 25 burns credits on pages that never needed it.

The escalation ladder

  1. Plain request, no rendering, datacenter proxy. Score every URL.
  2. JavaScript rendering on, only for the URLs that failed step 1.
  3. Premium or residential proxies, only for what failed step 2. Add the country setting for geo-locked pages here.
  4. The vendor’s top setting (stealth or ultra premium), only for what’s left, and only if your plan allows it.

Record the rung where each URL first passed. That rung is the setting you’ll pay for in production.

Watch the defaults. ScrapingBee turns JavaScript rendering on by default at 5 credits, so a “plain” request there needs rendering switched off (ScrapingBee docs). Its Auto-Mode runs the ladder for you, charging 1, 5, 10, 25 or 75 for the setting that worked and nothing if all fail, and reports the charge in Spb-auto-cost.

At vendors that don’t bill failures, climbing the ladder costs time but no credits for the rungs that failed. The exception is a block page that comes back as a billable status: at ScraperAPI, a 200 is charged whatever the page says (ScraperAPI credits). That’s one more reason to score content, not status codes.

The ladder also tells you which page types need rendering at all, which is often the biggest line on the bill. If the answer is “only the search pages”, you’ll buy a much smaller plan.

What should you log besides pass or fail? #

Latency percentiles, concurrency errors and retries. These are what break in production when the pass rate looked fine in the trial.

Latency: log p50 and p95, not the average. Google’s SRE book lists the four golden signals as “latency, traffic, errors, and saturation” and notes how tempting it is to design a system around the mean.

Its sharper line: “the 99th percentile of one backend can easily become the median response of your frontend” (Google SRE book). Report p50 and p95 per page type, because rendered pages will be slower.

Concurrency: test at the rate you’ll run. Over the limit, ZenRows returns a 429 and sends Concurrency-Limit and Concurrency-Remaining headers; cancelling a request “can take up to 3 minutes” to free its slot (ZenRows concurrency). ScraperAPI and ScrapingBee also use 429 for too many simultaneous requests (ScraperAPI, ScrapingBee).

This is where free tiers mislead. The ones that state a limit cap you at 5 concurrent requests, or 1 at Scrapingdog, so a trial can’t show you how a vendor behaves at 20 or 50.

Proxyway’s 2025 report ran at 2 and 10 requests a second and noted that ZenRows and Firecrawl hit concurrency limits at 10 (Proxyway). If production concurrency matters, budget one month of the smallest paid plan for the finalist.

Retries: separate the vendor’s from yours. ScraperAPI retries internally for up to 70 seconds and ScrapingBee for 30, and that time shows up as latency, not as a failure (ScraperAPI, ScrapingBee). Zyte’s docs recommend retrying rate-limit and ban responses with exponential backoff (Zyte errors).

Your own retry of an unbilled failure costs nothing at the vendors that don’t bill failures. Retrying a billed response (a 404, a 413 at ScrapingBee, a ScraperAPI request you cancelled before its 70 seconds were up) is billed again, so log retried as its own column.

Credits: read the header, not the price list. Copy the cost header from every response into your sheet: sa-credit-cost, X-Scrapfly-Api-Cost, Scrape.do-Request-Cost, or ScrapingBee’s cost field. ScrapingBee’s usage endpoint is limited to 6 calls a minute, so poll it at the end of a run rather than per request (ScrapingBee).

How many URLs before a success rate means anything? #

More than most trials use. On 50 URLs, a vendor at 96% and one at 90% can’t be told apart; you need a few hundred attempts per vendor before a 6-point gap is real.

The tool for this is a confidence interval for a proportion. The NIST handbook warns that the simple normal approximation can give limits below zero, calling it “an inferior approach”, and gives the Wilson score interval instead, with exact binomial methods for very small samples or very few failures (NIST). Here’s the Wilson formula as NIST gives it, with z = 1.96 for 95% confidence:

          p + z²/(2n)  ±  z · √( p(1−p)/n + z²/(4n²) )
limits = ----------------------------------------------
                         1 + z²/n

Worked for 48 passes out of 50 (my calculation): p = 0.96, n = 50, z² = 3.8416.

  • Denominator: 1 + 3.8416/50 = 1.0768.
  • Centre: (0.96 + 3.8416/100) ÷ 1.0768 = 0.9984 ÷ 1.0768 = 0.9272.
  • Spread: 1.96 × √(0.96 × 0.04/50 + 3.8416/10,000) = 1.96 × √(0.000768 + 0.000384) = 1.96 × 0.03394 = 0.0665, and 0.0665 ÷ 1.0768 = 0.0618.
  • Interval: 0.9272 ± 0.0618, so 86.5% to 98.9%.

The same formula for other results (my calculation, 95% confidence):

Passes Pass rate Wilson interval What it tells you
21 of 30 70% 52.1% to 83.3% Below a 90% floor even at its best: 30 URLs is enough to drop a broken vendor
30 of 30 100% 88.6% to 100% A perfect small run still can’t promise 90%
45 of 50 90% 78.6% to 95.7% Overlaps the row below
48 of 50 96% 86.5% to 98.9% So these two vendors are indistinguishable on 50 URLs
90 of 100 90% 82.6% to 94.5% Still overlaps the row below
96 of 100 96% 90.2% to 98.4%
270 of 300 90% 86.1% to 92.9% No overlap with the row below
288 of 300 96% 93.1% to 97.7% At about 300 attempts, the 6-point gap shows

I’m using “the intervals don’t overlap” as a simple, cautious rule rather than a formal test. In practice: 30 URLs per page type catches a vendor that’s clearly broken on that type, and repeat runs on different days count toward the few hundred you need to separate two good ones.

Close-up of soft-edged off-white square tiles with uneven grout gaps, faint scuffs and a few hairline marks on the surface
Up close, small differences between tiles show that a quick glance misses, just as a few hundred attempts reveal a gap that 50 URLs hide. Photo: Ricardo Loaiza / Unsplash · Unsplash License

What does a good record cost? #

Total billed dollars divided by the records that passed your field checks. It’s cost per successful request, with success judged by content rather than status code. A vendor with a lower list price can lose here if it bills pages you couldn’t use.

cost per good record = (sum of credits billed × dollars per credit) ÷ records that passed

Here’s the arithmetic on a made-up run, to show the steps, not a result.

The example uses 50 URLs on ZenRows’ Build plan ($19 for 45,000 credits, so $0.000422 a credit): 30 pass on plain requests at 1 credit and 20 need JavaScript plus premium at 25. That’s 30 + 500 = 530 credits, or $0.224. If 46 of the 50 records pass your field checks, the cost is $0.224 ÷ 46 = $0.0049 per good record, or about $4.86 per 1,000.

Two rules keep this honest. Take the credits from the cost header, not from the price list, and handle dead URLs separately: a 404 adds to the top line at ScraperAPI, ScrapingBee, ZenRows and Scrape.do (and, I’d expect, Zyte) but never to the bottom one.

AIMultiple’s benchmark normalises cost per 1,000 successful pages the same way (AIMultiple). Run the formula per page type, because a vendor can be cheapest on detail pages and dearest on search pages.

The scoring sheet and the decision rule #

One row per request, one sheet per trial. Copy the header into a spreadsheet or CSV:

url,page_type,vendor,settings,run_no,run_date,vendor_status,target_status,fields_expected,fields_ok,pass,credits_billed,usd_cost,latency_ms,retried,notes

What goes in each column:

  • settings: the ladder rung, written as js / premium / geo flags.
  • vendor_status and target_status: both, where the vendor exposes them. Zyte returns the target’s statusCode, and ScrapingBee has a transparent status option (Zyte errors, ScrapingBee).
  • fields_expected and fields_ok: counts from your answer key.
  • pass: y only if all five checks above hold.
  • credits_billed: copied from the cost header; usd_cost is that times your per-credit price.
  • retried: y if this row is a retry, so billed retries show up.

Then decide in three steps:

  1. Gate. For each page type you need, compute the vendor’s pass rate and its Wilson lower bound. Drop any vendor whose lower bound sits under your floor on any required page type.
  2. Rank. Sort the survivors by cost per good record at the ladder rung each page type needed.
  3. Break ties on p95 latency, then on billing transparency: a cost header plus a per-request budget cap beats a monthly invoice you have to reconcile.

Set the floor before you look at the results, and set it with the sample size in mind. With 50 URLs per page type, a floor of 85% is about the most the data can support, because even 48 of 50 has a lower bound of 86.5%.

Can you trust published benchmarks instead? #

As a source of method, yes; as your decision, no. Every one I read is run by a vendor, funded by vendors or published by an affiliate site, and none used your target pages.

Here’s who runs each one and what I’d borrow from it:

Benchmark Who runs it Method Use it for
Proxyway Web Scraping API Report 2025 Proxyway (Adam Dubois, December 4, 2025), discloses affiliate links 11 providers, 15 protected sites, about 6,000 URLs per target at 2 and 10 requests a second, 600-second timeout How to test at two concurrency levels
AIMultiple (updated October 22, 2026) AIMultiple; Bright Data, Oxylabs, Decodo, Apify, SerpApi and Zyte subscribe to its benchmark service Success = “correct text presence on extracted page”; cost per 1,000 successful pages The cost-per-success idea
ZenRows benchmark ZenRows, which ranks itself first 100 requests per target at 2 a second, one run on June 19, 2026; success = 200 plus a valid title Its open script
MacMyths guide (September 29, 2026) A tips site, no vendor ranking Per-domain metrics: page success, field completeness, accuracy, freshness, schema stability, recovery, operational cost A checklist of what to measure

The most useful item here is ZenRows’ benchmark repository: Python under the MIT licence, writing a per-request CSV and a summary CSV. It points at four fixed targets, so you’d swap in your own URLs and add field checks, but the plumbing is done.

I’d discount any single number from these pages. ZenRows’ own table gives Zyte an average latency of 14.3 seconds while its text says 13.6, and one 100-request run on four targets says little about your fifth (ZenRows).

Where does the test have to stop? #

At pages you’re allowed to scrape. A trial is still scraping, and “I was only evaluating vendors” isn’t a permission.

Test on your own sites, on practice sandboxes such as toscrape.com, or on targets whose terms and robots.txt allow what you’re doing. RFC 9309 is clear that robots.txt rules “are not a form of access authorization”, so an allow line isn’t permission and a disallow isn’t a lock (RFC 9309). It also says a crawler that gets a server error for robots.txt must assume a complete disallow, and shouldn’t cache the file for more than 24 hours.

Vendors draw their own lines too. Zyte’s acceptable-use policy bars breaching “security or authentication measures”, collecting personal data in violation of data protection law, scraping LinkedIn, and use that “materially interferes with a website’s normal operations” (Zyte AUP). Read the policy of each vendor on your list before you point it at anything.

Run your trial at a gentle rate on third-party pages; the production-concurrency test belongs on targets that can take it. My ethical web scraping checklist has the house rules, and is web scraping legal covers the case law.

One open question: I couldn’t establish whether any vendor’s terms forbid publishing your test results. Zyte’s acceptable-use policy has no benchmarking clause (though clause 1.1.12 bars any practice that “reflects poorly on Zyte”) and my searches found none elsewhere, but I didn’t open each vendor’s terms of service, so check before you share numbers outside your team.

White rectangular wall panels seen at a steep angle, their thin dark joints converging, with the grid ending sharply against blank white space on the left
The grid stops at a clean edge, and a trial has edges too: robots.txt, a site's terms and each vendor's acceptable-use policy mark where testing ends. Photo: Ev / Unsplash · Unsplash License

Which APIs should you put on the shortlist? #

Three with different billing models, so the trial tests the trade-offs and not just the brands. Here’s how I’d pick them by situation.

You need lots of JavaScript-heavy pages and have no budget yet: ZenRows. 5,000 free credits every month, with no card, is 200 JavaScript-plus-premium pages a month, and you can rerun the test next month on the same plan (ZenRows).

You can finish in a week: ScraperAPI. Its 7-day trial gives 5,000 credits, the same 200 premium-plus-rendered pages, and its account/urlcost endpoint estimates a URL’s cost before you scrape it (ScraperAPI, credits). Leave dead URLs out, because 404s bill.

Your targets vary from easy to very hard, and failures must cost nothing: Zyte. Unsuccessful and rate-limited responses are free, and the price follows each site’s tier, from $0.13 per 1,000 plain requests on tier 1 to $16.08 per 1,000 browser requests on tier 5 (Zyte). Remember that a target’s 404 counts as a success there.

You need to reconcile spend per request: Scrapfly, for the X-Scrapfly-Api-Cost header and the cost_budget cap that fails a request rather than overspend (for deterministic configurations only) (Scrapfly billing). Scrape.do is the alternative, with the cheapest entry-plan credit on this page and a cost header it calls authoritative (Scrape.do).

You only scrape plain pages: any of them, including Scrapingdog. Its 100 free credits cover 100 plain requests but only 4 JavaScript-plus-premium ones, at 1 concurrent request (Scrapingdog).

If I were running this tomorrow, I’d shortlist ZenRows, Zyte and Scrapfly (or Scrape.do), so each billing model is in the test. I’d run 50 URLs across four or five page types, twice, a day apart, climbing the ladder only for failures. I’d keep a vendor only if its lower-bound pass rate cleared my floor on every page type I need, then pick the lowest cost per good record and buy one month to test production concurrency before committing.

FAQ

How many URLs do I need to test a scraping API?

About 30 per page type is enough to catch a vendor that's clearly broken on that type. Telling a 90% vendor from a 96% vendor takes a few hundred attempts: at 50 URLs each, the 95% Wilson intervals (86.5% to 98.9% and 78.6% to 95.7%) overlap, and they only separate at around 300 (my arithmetic with the NIST formula). Repeat runs on different days count toward the total.

Do scraping APIs charge for failed requests?

Mostly no, but the definition of failed differs. ScraperAPI charges 200 and 404 responses and requests you cancel before its 70-second window is up. ScrapingBee bills 200, 404, 410 and 413. ZenRows counts 404 and 410 as successful, and Scrape.do counts 2XX, 400, 404 and 410 as successful. Zyte says unsuccessful and rate-limited responses are free, but treats a target's 404 as a successful response. Checked November 16, 2026.

Is a 200 status a successful scrape?

No. A block or challenge page can come back as a 200, which is why published benchmarks add a title, page-size or text check. Count a URL as passed only when the fields you need are present and match a copy you checked by hand.

Which scraping API's free credits last longest?

ZenRows' free plan gives 5,000 credits every month with no card, so it isn't a time-boxed trial. Scrape.do lists its 1,000 free credits as monthly too. ScraperAPI's trial is 7 days, and Zyte's $5 credit covers the first billing month (its pricing page says 30 days). Scrapfly says its 1,000 signup credits have no time limit. ScrapingBee and Scrapingdog don't say when their free credits expire.

Can I test a scraping API on any website?

No. Only test on pages you're allowed to scrape: your own sites, practice sandboxes such as toscrape.com, or sites whose terms and robots.txt permit it. RFC 9309 says robots.txt rules are not a form of access authorization, so an allow line isn't permission either, and vendor policies such as Zyte's bar breaching security or authentication measures.

Can I trust published scraping API benchmarks?

Use them for method, not for your decision. The ones I read are run by a vendor (ZenRows), funded by vendors who subscribe to the benchmark service (AIMultiple), or published by an affiliate site (Proxyway), and none of them used your target pages.

How do I see what each scraping API request cost?

Read the vendor's cost field on every response: sa-credit-cost at ScraperAPI, X-Scrapfly-Api-Cost at Scrapfly, Scrape.do-Request-Cost at Scrape.do, and the cost field in ScrapingBee's JSON response (Spb-auto-cost in Auto-Mode). Scrapfly also offers cost_budget, which fails a request rather than let it cost more than you set.

Sources

27 sources, accessed .

Official pricing, docs, changelogs and status pages 20

  1. ScraperAPI pricing and free trial scraperapi.com · accessed
  2. ScraperAPI docs: credits and request costs docs.scraperapi.com · accessed
  3. ScraperAPI docs: API status codes docs.scraperapi.com · accessed
  4. ScrapingBee pricing scrapingbee.com · accessed
  5. ScrapingBee documentation: credits and status codes scrapingbee.com · accessed
  6. ZenRows pricing and free plan zenrows.com · accessed
  7. ZenRows docs: concurrency docs.zenrows.com · accessed
  8. Scrapfly pricing scrapfly.io · accessed
  9. Scrapfly docs: billing and cost reporting scrapfly.io · accessed
  10. Zyte API docs: pricing docs.zyte.com · accessed
  11. Zyte pricing page zyte.com · accessed
  12. Zyte API docs: errors and successful responses docs.zyte.com · accessed
  13. Zyte acceptable use policy zyte.com · accessed
  14. Scrape.do pricing scrape.do · accessed
  15. Scrape.do docs: request costs scrape.do · accessed
  16. Scrapingdog pricing scrapingdog.com · accessed
  17. Bright Data Web Unlocker pricing brightdata.com · accessed
  18. Oxylabs Web Scraper API pricing oxylabs.io · accessed
  19. RFC 9309: Robots Exclusion Protocol rfc-editor.org · accessed
  20. NIST e-Handbook 7.2.4.1: confidence intervals for a proportion itl.nist.gov · accessed

GitHub code, issues and releases 1

  1. ZenRows web-scraping-api-benchmark repository (MIT) github.com · accessed

Independent benchmarks and published studies 2

  1. Proxyway Web Scraping API Report 2025 (affiliate publisher) proxyway.com · accessed
  2. AIMultiple web scraping API benchmark (vendor-funded) aimultiple.com · accessed

Other blogs 2

  1. ZenRows blog: web scraping APIs benchmarked (vendor-run) zenrows.com · accessed
  2. MacMyths: how to evaluate a web scraping provider macmyths.com · accessed

Other sources 2

  1. toscrape.com practice sandboxes toscrape.com · accessed
  2. Google SRE book, chapter 6: monitoring distributed systems sre.google · accessed

Last updated: 16 November 2026. Researched and drafted with AI tools; human editorial review and publication approval are pending.

What readers say

Used one of the tools in this guide? Tell other readers how it went. I read every review before it goes up, and I never turn one down for being negative. How reviews work.

No reader reviews yet. Yours could be the first.

Write a review
Your rating

What did you use it for, what worked, and what didn't? 30 to 1500 characters. Please don't include links.

Your connection to the company

Your review appears with the name you enter, after I approve it. I don't ask for your email. Privacy.