How to choose
How to test a web scraping API on your own pages first
Before you pay, test a web scraping API on 30-100 of your own URLs: what each vendor bills, how far free trials go, and which one to keep.

You’ve shortlisted two to four scraping APIs, and each one hands out free credits. The decision is which one gets your pages right at a cost per good record you can live with, and the only fair way to find out is a proof of concept you design yourself.
This page is that design: which URLs to test, what each vendor bills, how far the free credits go, how many URLs you need before a gap is real, and a scoring sheet with a decision rule. It’s a plan you run on your own pages, so none of the numbers below are results of mine.
Short on time?
Short on time? Here's my take.
- Biggest free allowance for JavaScript-heavy pages
- ZenRows
- 5,000 credits every month with no card. It renews monthly, so it isn't a trial. That's 1,000 JavaScript pages or 200 JavaScript-plus-premium pages a month, at 5 concurrent requests.
- A full trial on a deadline
- ScraperAPI
- 7 days and 5,000 credits, no card: 500 rendered pages or 200 premium-plus-rendered ones. Clean dead URLs out first, because it bills 404s.
- Failures must cost nothing
- Zyte
- Unsuccessful and rate-limited responses are free, and the price follows each site's difficulty tier. One catch: Zyte treats a target site's 404 as a successful response. $5 of credit to start.
- Seeing what every request cost
- Scrapfly (or Scrape.do)
- Scrapfly returns the cost in an X-Scrapfly-Api-Cost header and lets you cap each request with cost_budget. Scrape.do's Scrape.do-Request-Cost header is the alternative, and it calls that header authoritative for billing.
- Plain pages only
- Scrapingdog
- 100 free credits and 1 concurrent request buy just 4 JavaScript-plus-premium pages. Fine for checking plain pages, too small for protected ones.
Prices verified . Sponsors cannot buy a verdict or a position.
To test a web scraping API, pick 30 to 100 of your own URLs by page type, write down the fields each should return, and run them on each vendor’s free credits, cheapest setting first. Among vendors whose lower-bound pass rate clears your floor, pick the lowest cost per good record.
This page sits in the scraping API hub. If you haven’t shortlisted yet, start with the best web scraping APIs; if you’re weighing free tiers against trials, free web scraping APIs covers that.
Which of your URLs should go in the test? #
Your own, sampled by page type, 30 to 100 in total. A vendor that clears a listing page can still fail on the product page behind it, so a single bucket of “URLs from the site” hides the failure you care about.
Sort your targets into the page types you’ll actually scrape in production:
- Listing or category pages, where pagination and lazy loading show up.
- Detail pages (product, job, property), where the fields you pay for live.
- Search result pages on the target site, which are often the most protected.
- JavaScript-heavy pages, where the data only appears after scripts run. Check whether a page needs rendering before you count it here.
- Geo-locked pages, which need a country setting.
- Known-dead URLs, kept as a separate bucket, because several vendors bill a 404.
Add a few practice pages as a control. toscrape.com calls itself “a safe place for beginners learning web scraping and for developers validating their scraping technologies”, and its sandboxes include JavaScript-generated content, delayed loading, infinite scroll, a login with a CSRF token and a ViewState filter form.
If a vendor fails there, the problem is your setup, not the target.
Can the free credits cover it? #
Not if you send every URL at the most expensive setting. 100 URLs run three times is 300 requests, and at a weight of 25 credits per request (JavaScript plus premium proxies at most vendors) that’s 7,500 credits, more than any credit-based free allowance on this page (my arithmetic).
So cut to 30 to 50 URLs per vendor, or escalate settings only for URLs that fail the cheaper one. The ladder is further down.
I’d run the sample twice, a day apart, at the concurrency you’d use in production. The MacMyths guide only says a pilot should be “long enough” to cover normal variation, without a number (MacMyths); two runs on different days is the cheapest version of that I can defend.
What counts as a successful scrape? #
A record with the right fields in it, not an HTTP 200. A block or challenge page can come back with a 200, and every published benchmark I read adds a content check for that reason.
ZenRows’ own benchmark counts a success as a 200 plus a valid page title (ZenRows). Proxyway validated by response code, page size and title (Proxyway). AIMultiple defines success as “correct text presence on extracted page” (AIMultiple).
For a buying decision, go one step further and check fields. Before the trial, open each URL in a normal browser, write down the 2 to 5 values you need (title, price, stock, date), and save that as your answer key.
Pass check for each URL
- The response came back before your timeout.
- The page is the real page, not a challenge, login wall or “access denied” page.
- Every field on your answer key is present.
- Each field matches the value you wrote down (allow for prices that change between runs).
- The page type’s layout parsed with the same selectors as the others in its bucket.
Pass = all five. Anything else is a fail, whatever the status code said.
MacMyths splits this into page success, field completeness and accuracy per domain (MacMyths). That split is useful later, because “the page loaded but the price was missing” points to a parsing problem, not a blocking one.
What will each vendor bill you for? #
Not only successes, and the differences change your cost per good record. Most vendors don’t bill server errors, but several bill a target’s 404, and one bills anything below 500 when you ask for the target’s status code.
Here’s what each vendor says it bills, from its own docs:
| Vendor | What bills | What doesn’t | How to see each request’s cost |
|---|---|---|---|
| ScraperAPI | 200 and 404; requests you cancel before its 70-second window is up | 500 (“Request failed. You will not be charged”) | sa-credit-cost header on every response; account/urlcost estimates a URL before you scrape it |
| ScrapingBee | 200, 404, 410, 413; with the transparent status option, “Anything the target returns below 500 is billed” | 500 | cost field with json_response; Spb-auto-cost in Auto-Mode |
| ZenRows | “Only successful requests are billed”; 404 and 410 count as successful | Other failures | No cost header in the pages I read; 429 when over your concurrency |
| Scrapfly | Successful scrapes, plus target statuses its failure protection doesn’t cover: 400 to 418 (except 403 and 408), 422, 424, 426, 428 and 456 | Other failed scrapes (“Scrape Failed Protection”) | X-Scrapfly-Api-Cost header, context.cost breakdown, cost_budget cap |
| Zyte | Successful responses; a target’s 404 is a successful Zyte response | “Rate-limiting and unsuccessful responses are free” | Cost estimator at app.zyte.com |
| Scrape.do | 2XX, 400, 404 and 410 count as successful | Timeouts, blocks and errors | Scrape.do-Request-Cost header, which it calls authoritative for billing |
| Scrapingdog | Successful requests | Failed, blocked and timed-out requests are “automatically refunded” | No status list published, so 404 handling is unclear |
Sources, all verified 2026-11-16: ScraperAPI credits and status codes; ScrapingBee docs; ZenRows pricing; Scrapfly billing; Zyte pricing docs and errors; Scrape.do request costs; Scrapingdog pricing.
Three details are easy to miss. Scrapfly’s failure protection can switch off if 30% or more of your scrapes in 24 hours (with at least 1,000 scrapes) are blocked without its Unblocker, so a sloppy test can lose it (Scrapfly billing). Scrape.do lets you claim back a 200 whose content was invalid through support (Scrape.do pricing).
And for Zyte, the docs use toscrape.com/not-found as the example: the target returns 404, and that counts as a successful Zyte response. The docs don’t say in so many words that it’s billed, so I’d expect it to bill and keep dead URLs out of the main sample (Zyte errors).
Of the two vendors outside the table, Bright Data’s Web Unlocker says “Pay only for success” (Bright Data), and Oxylabs’ pricing page doesn’t say whether failed results bill (Oxylabs). The full set of credit weights per feature is in how scraping API credits work.

How far do the free credits go? #
Among the credit-weighted plans, furthest at ZenRows and ScraperAPI, and not far at all at Scrapingdog. Bright Data’s flat 5K requests and Zyte’s $5 work differently, so I’ve converted them too. The table turns each allowance into pages at the settings a hard target needs (my arithmetic: allowance divided by the credit weight, rounded down).
| Vendor | Free allowance | Renews or expires | Card | Concurrency on the free tier | JavaScript pages | JavaScript + premium pages |
|---|---|---|---|---|---|---|
| ZenRows | 5,000 credits | Every month, no rollover | Not needed | 5 | 1,000 | 200 |
| ScraperAPI | 5,000 credits | 7-day trial | Not needed | Free plan: 5 (trial not stated separately) | 500 | 200 |
| Zyte | $5 credit | First billing month (“30 days” on the pricing page) | Not stated | Not stated | ~4,950 (tier 1) to ~310 (tier 5) | Priced by site tier |
| Bright Data Web Unlocker | 5K requests | Every month | Not stated | Not stated | 5,000 requests at one flat price | Not split out |
| Oxylabs | $5 toward any product | Not stated | Not stated | Not stated | 3,333 | Not split out |
| Scrape.do | 1,000 successful credits | Listed as monthly; expiry not stated | Not needed | 5 | 200 | 40 |
| ScrapingBee | 1,000 credits | Not stated | Not needed | Not stated | 200 | 40 |
| Scrapfly | 1,000 credits, hard cap | One-time, “no time limit” | Not needed | 5 | 166 (datacenter) | 33 (residential) |
| Scrapingdog | 100 credits | Not stated | Not needed | 1 | 20 | 4 |
Sources, verified 2026-11-16: ZenRows, ScraperAPI, Zyte, Zyte docs, Bright Data, Oxylabs, Scrape.do, ScrapingBee, Scrapfly and its billing docs, Scrapingdog. Page counts are my arithmetic.
How I got the less obvious rows:
- ZenRows: 5,000 ÷ 5 = 1,000 rendered pages; 5,000 ÷ 25 = 200 with premium proxies too. Its free plan is $0 every month, not a time-boxed trial, so you can repeat a test next month (ZenRows).
- ScraperAPI: the page says “7-day trial and 5,000 API credits. No credit card required.” Rendering costs 10 and premium plus rendering 25, so 500 and 200 pages. Ultra premium (30, or 75 with rendering) is for paid plans only, so you can’t test it on the trial (ScraperAPI credits).
- Zyte: $5 ÷ $1.01 per 1,000 browser requests on site tier 1 = about 4,950; ÷ $16.08 on tier 5 = about 310; ÷ $0.13 for plain HTTP on tier 1 = about 38,460 (Zyte pricing).
- Scrapfly: a datacenter request is 1 credit plus 5 for rendering (6), a residential one 25 plus 5 (30), so 1,000 ÷ 6 = 166 and 1,000 ÷ 30 = 33 (Scrapfly billing).
- Oxylabs: $5 ÷ $1.50 per 1,000 rendered results = 3,333, or 5,000 without rendering at $1.00 (Oxylabs).
- Scrape.do: its free credits are “1,000 Successful API Credits”, so failed requests don’t eat them, but some domains carry their own cost: google.* is 10, linkedin.com 30, shopee.* 100 and sainsburys.co.uk 200 (Scrape.do costs).
Two gaps are worth saying plainly. ScrapingBee, Scrape.do and Scrapingdog don’t say when their free credits expire, so plan to finish your test in one sitting. Scrapfly is the exception: its pricing page describes the 1,000 signup credits as “no credit card, no time limit” (Scrapfly). ScraperAPI shows a 7-day, 5,000-credit trial and, in the same page’s FAQ, a free plan with 1,000 credits and 5 concurrent connections, without saying which you’re on after day 7 (ScraperAPI).
What does a credit cost once you pay? #
You need this to turn billed credits into dollars per good record. Here’s each vendor’s entry plan, divided out (my arithmetic: plan price ÷ credits × weight × 1,000, verified 2026-11-16).
| Vendor, entry plan | Per credit | Per 1,000 plain | Per 1,000 JavaScript | Per 1,000 JavaScript + premium |
|---|---|---|---|---|
| ScraperAPI Hobby, $49 for 100,000 | $0.00049 | $0.49 | $4.90 | $12.25 |
| ZenRows Build, $19 for 45,000 | $0.000422 | $0.42 | $2.11 | $10.56 |
| Scrapingdog Lite, $40 for 200,000 | $0.0002 | $0.20 | $1.00 | $5.00 |
| ScrapingBee Freelance, $49 for 250,000 | $0.000196 | $0.20 | $0.98 | $4.90 |
| Scrapfly Discovery, $30 for 200,000 | $0.00015 | $0.15 | $0.90 | $4.50 |
| Scrape.do Hobby, $29 for 250,000 | $0.000116 | $0.12 | $0.58 | $2.90 |
| Zyte, pay as you go | Priced by site tier | $0.13 to $1.27 | $1.01 to $16.08 | Set by tier |
| Bright Data Web Unlocker, pay as you go | Per request | $1.50 flat | $1.50 flat | $1.50 flat |
ScrapingBee’s Freelance plan is its second tier; the table uses it because its $19 Hobby plan has 75,000 credits, which works out to a higher rate per credit. Plan-by-plan detail lives in each vendor’s pricing post, linked from the hub.
Spend credits only where a page needs them #
Start every URL at the cheapest setting and move only the failures up a rung. On a ZenRows-style scale, a plain request costs 1 credit, JavaScript 5, premium proxies 10 and both together 25 (ZenRows), so sending everything at 25 burns credits on pages that never needed it.
The escalation ladder
- Plain request, no rendering, datacenter proxy. Score every URL.
- JavaScript rendering on, only for the URLs that failed step 1.
- Premium or residential proxies, only for what failed step 2. Add the country setting for geo-locked pages here.
- The vendor’s top setting (stealth or ultra premium), only for what’s left, and only if your plan allows it.
Record the rung where each URL first passed. That rung is the setting you’ll pay for in production.
Watch the defaults. ScrapingBee turns JavaScript rendering on by default at 5 credits, so a “plain” request there needs rendering switched off (ScrapingBee docs). Its Auto-Mode runs the ladder for you, charging 1, 5, 10, 25 or 75 for the setting that worked and nothing if all fail, and reports the charge in Spb-auto-cost.
At vendors that don’t bill failures, climbing the ladder costs time but no credits for the rungs that failed. The exception is a block page that comes back as a billable status: at ScraperAPI, a 200 is charged whatever the page says (ScraperAPI credits). That’s one more reason to score content, not status codes.
The ladder also tells you which page types need rendering at all, which is often the biggest line on the bill. If the answer is “only the search pages”, you’ll buy a much smaller plan.
What should you log besides pass or fail? #
Latency percentiles, concurrency errors and retries. These are what break in production when the pass rate looked fine in the trial.
Latency: log p50 and p95, not the average. Google’s SRE book lists the four golden signals as “latency, traffic, errors, and saturation” and notes how tempting it is to design a system around the mean.
Its sharper line: “the 99th percentile of one backend can easily become the median response of your frontend” (Google SRE book). Report p50 and p95 per page type, because rendered pages will be slower.
Concurrency: test at the rate you’ll run. Over the limit, ZenRows returns a 429 and sends Concurrency-Limit and Concurrency-Remaining headers; cancelling a request “can take up to 3 minutes” to free its slot (ZenRows concurrency). ScraperAPI and ScrapingBee also use 429 for too many simultaneous requests (ScraperAPI, ScrapingBee).
This is where free tiers mislead. The ones that state a limit cap you at 5 concurrent requests, or 1 at Scrapingdog, so a trial can’t show you how a vendor behaves at 20 or 50.
Proxyway’s 2025 report ran at 2 and 10 requests a second and noted that ZenRows and Firecrawl hit concurrency limits at 10 (Proxyway). If production concurrency matters, budget one month of the smallest paid plan for the finalist.
Retries: separate the vendor’s from yours. ScraperAPI retries internally for up to 70 seconds and ScrapingBee for 30, and that time shows up as latency, not as a failure (ScraperAPI, ScrapingBee). Zyte’s docs recommend retrying rate-limit and ban responses with exponential backoff (Zyte errors).
Your own retry of an unbilled failure costs nothing at the vendors that don’t bill failures. Retrying a billed response (a 404, a 413 at ScrapingBee, a ScraperAPI request you cancelled before its 70 seconds were up) is billed again, so log retried as its own column.
Credits: read the header, not the price list. Copy the cost header from every response into your sheet: sa-credit-cost, X-Scrapfly-Api-Cost, Scrape.do-Request-Cost, or ScrapingBee’s cost field. ScrapingBee’s usage endpoint is limited to 6 calls a minute, so poll it at the end of a run rather than per request (ScrapingBee).
How many URLs before a success rate means anything? #
More than most trials use. On 50 URLs, a vendor at 96% and one at 90% can’t be told apart; you need a few hundred attempts per vendor before a 6-point gap is real.
The tool for this is a confidence interval for a proportion. The NIST handbook warns that the simple normal approximation can give limits below zero, calling it “an inferior approach”, and gives the Wilson score interval instead, with exact binomial methods for very small samples or very few failures (NIST). Here’s the Wilson formula as NIST gives it, with z = 1.96 for 95% confidence:
p + z²/(2n) ± z · √( p(1−p)/n + z²/(4n²) )
limits = ----------------------------------------------
1 + z²/n
Worked for 48 passes out of 50 (my calculation): p = 0.96, n = 50, z² = 3.8416.
- Denominator: 1 + 3.8416/50 = 1.0768.
- Centre: (0.96 + 3.8416/100) ÷ 1.0768 = 0.9984 ÷ 1.0768 = 0.9272.
- Spread: 1.96 × √(0.96 × 0.04/50 + 3.8416/10,000) = 1.96 × √(0.000768 + 0.000384) = 1.96 × 0.03394 = 0.0665, and 0.0665 ÷ 1.0768 = 0.0618.
- Interval: 0.9272 ± 0.0618, so 86.5% to 98.9%.
The same formula for other results (my calculation, 95% confidence):
| Passes | Pass rate | Wilson interval | What it tells you |
|---|---|---|---|
| 21 of 30 | 70% | 52.1% to 83.3% | Below a 90% floor even at its best: 30 URLs is enough to drop a broken vendor |
| 30 of 30 | 100% | 88.6% to 100% | A perfect small run still can’t promise 90% |
| 45 of 50 | 90% | 78.6% to 95.7% | Overlaps the row below |
| 48 of 50 | 96% | 86.5% to 98.9% | So these two vendors are indistinguishable on 50 URLs |
| 90 of 100 | 90% | 82.6% to 94.5% | Still overlaps the row below |
| 96 of 100 | 96% | 90.2% to 98.4% | |
| 270 of 300 | 90% | 86.1% to 92.9% | No overlap with the row below |
| 288 of 300 | 96% | 93.1% to 97.7% | At about 300 attempts, the 6-point gap shows |
I’m using “the intervals don’t overlap” as a simple, cautious rule rather than a formal test. In practice: 30 URLs per page type catches a vendor that’s clearly broken on that type, and repeat runs on different days count toward the few hundred you need to separate two good ones.

What does a good record cost? #
Total billed dollars divided by the records that passed your field checks. It’s cost per successful request, with success judged by content rather than status code. A vendor with a lower list price can lose here if it bills pages you couldn’t use.
cost per good record = (sum of credits billed × dollars per credit) ÷ records that passed
Here’s the arithmetic on a made-up run, to show the steps, not a result.
The example uses 50 URLs on ZenRows’ Build plan ($19 for 45,000 credits, so $0.000422 a credit): 30 pass on plain requests at 1 credit and 20 need JavaScript plus premium at 25. That’s 30 + 500 = 530 credits, or $0.224. If 46 of the 50 records pass your field checks, the cost is $0.224 ÷ 46 = $0.0049 per good record, or about $4.86 per 1,000.
Two rules keep this honest. Take the credits from the cost header, not from the price list, and handle dead URLs separately: a 404 adds to the top line at ScraperAPI, ScrapingBee, ZenRows and Scrape.do (and, I’d expect, Zyte) but never to the bottom one.
AIMultiple’s benchmark normalises cost per 1,000 successful pages the same way (AIMultiple). Run the formula per page type, because a vendor can be cheapest on detail pages and dearest on search pages.
The scoring sheet and the decision rule #
One row per request, one sheet per trial. Copy the header into a spreadsheet or CSV:
url,page_type,vendor,settings,run_no,run_date,vendor_status,target_status,fields_expected,fields_ok,pass,credits_billed,usd_cost,latency_ms,retried,notes
What goes in each column:
settings: the ladder rung, written as js / premium / geo flags.vendor_statusandtarget_status: both, where the vendor exposes them. Zyte returns the target’sstatusCode, and ScrapingBee has a transparent status option (Zyte errors, ScrapingBee).fields_expectedandfields_ok: counts from your answer key.pass: y only if all five checks above hold.credits_billed: copied from the cost header;usd_costis that times your per-credit price.retried: y if this row is a retry, so billed retries show up.
Then decide in three steps:
- Gate. For each page type you need, compute the vendor’s pass rate and its Wilson lower bound. Drop any vendor whose lower bound sits under your floor on any required page type.
- Rank. Sort the survivors by cost per good record at the ladder rung each page type needed.
- Break ties on p95 latency, then on billing transparency: a cost header plus a per-request budget cap beats a monthly invoice you have to reconcile.
Set the floor before you look at the results, and set it with the sample size in mind. With 50 URLs per page type, a floor of 85% is about the most the data can support, because even 48 of 50 has a lower bound of 86.5%.
Can you trust published benchmarks instead? #
As a source of method, yes; as your decision, no. Every one I read is run by a vendor, funded by vendors or published by an affiliate site, and none used your target pages.
Here’s who runs each one and what I’d borrow from it:
| Benchmark | Who runs it | Method | Use it for |
|---|---|---|---|
| Proxyway Web Scraping API Report 2025 | Proxyway (Adam Dubois, December 4, 2025), discloses affiliate links | 11 providers, 15 protected sites, about 6,000 URLs per target at 2 and 10 requests a second, 600-second timeout | How to test at two concurrency levels |
| AIMultiple (updated October 22, 2026) | AIMultiple; Bright Data, Oxylabs, Decodo, Apify, SerpApi and Zyte subscribe to its benchmark service | Success = “correct text presence on extracted page”; cost per 1,000 successful pages | The cost-per-success idea |
| ZenRows benchmark | ZenRows, which ranks itself first | 100 requests per target at 2 a second, one run on June 19, 2026; success = 200 plus a valid title | Its open script |
| MacMyths guide (September 29, 2026) | A tips site, no vendor ranking | Per-domain metrics: page success, field completeness, accuracy, freshness, schema stability, recovery, operational cost | A checklist of what to measure |
The most useful item here is ZenRows’ benchmark repository: Python under the MIT licence, writing a per-request CSV and a summary CSV. It points at four fixed targets, so you’d swap in your own URLs and add field checks, but the plumbing is done.
I’d discount any single number from these pages. ZenRows’ own table gives Zyte an average latency of 14.3 seconds while its text says 13.6, and one 100-request run on four targets says little about your fifth (ZenRows).
Where does the test have to stop? #
At pages you’re allowed to scrape. A trial is still scraping, and “I was only evaluating vendors” isn’t a permission.
Test on your own sites, on practice sandboxes such as toscrape.com, or on targets whose terms and robots.txt allow what you’re doing. RFC 9309 is clear that robots.txt rules “are not a form of access authorization”, so an allow line isn’t permission and a disallow isn’t a lock (RFC 9309). It also says a crawler that gets a server error for robots.txt must assume a complete disallow, and shouldn’t cache the file for more than 24 hours.
Vendors draw their own lines too. Zyte’s acceptable-use policy bars breaching “security or authentication measures”, collecting personal data in violation of data protection law, scraping LinkedIn, and use that “materially interferes with a website’s normal operations” (Zyte AUP). Read the policy of each vendor on your list before you point it at anything.
Run your trial at a gentle rate on third-party pages; the production-concurrency test belongs on targets that can take it. My ethical web scraping checklist has the house rules, and is web scraping legal covers the case law.
One open question: I couldn’t establish whether any vendor’s terms forbid publishing your test results. Zyte’s acceptable-use policy has no benchmarking clause (though clause 1.1.12 bars any practice that “reflects poorly on Zyte”) and my searches found none elsewhere, but I didn’t open each vendor’s terms of service, so check before you share numbers outside your team.

Which APIs should you put on the shortlist? #
Three with different billing models, so the trial tests the trade-offs and not just the brands. Here’s how I’d pick them by situation.
You need lots of JavaScript-heavy pages and have no budget yet: ZenRows. 5,000 free credits every month, with no card, is 200 JavaScript-plus-premium pages a month, and you can rerun the test next month on the same plan (ZenRows).
You can finish in a week: ScraperAPI. Its 7-day trial gives 5,000 credits, the same 200 premium-plus-rendered pages, and its account/urlcost endpoint estimates a URL’s cost before you scrape it (ScraperAPI, credits). Leave dead URLs out, because 404s bill.
Your targets vary from easy to very hard, and failures must cost nothing: Zyte. Unsuccessful and rate-limited responses are free, and the price follows each site’s tier, from $0.13 per 1,000 plain requests on tier 1 to $16.08 per 1,000 browser requests on tier 5 (Zyte). Remember that a target’s 404 counts as a success there.
You need to reconcile spend per request: Scrapfly, for the X-Scrapfly-Api-Cost header and the cost_budget cap that fails a request rather than overspend (for deterministic configurations only) (Scrapfly billing). Scrape.do is the alternative, with the cheapest entry-plan credit on this page and a cost header it calls authoritative (Scrape.do).
You only scrape plain pages: any of them, including Scrapingdog. Its 100 free credits cover 100 plain requests but only 4 JavaScript-plus-premium ones, at 1 concurrent request (Scrapingdog).
If I were running this tomorrow, I’d shortlist ZenRows, Zyte and Scrapfly (or Scrape.do), so each billing model is in the test. I’d run 50 URLs across four or five page types, twice, a day apart, climbing the ladder only for failures. I’d keep a vendor only if its lower-bound pass rate cleared my floor on every page type I need, then pick the lowest cost per good record and buy one month to test production concurrency before committing.
FAQ
How many URLs do I need to test a scraping API?
About 30 per page type is enough to catch a vendor that's clearly broken on that type. Telling a 90% vendor from a 96% vendor takes a few hundred attempts: at 50 URLs each, the 95% Wilson intervals (86.5% to 98.9% and 78.6% to 95.7%) overlap, and they only separate at around 300 (my arithmetic with the NIST formula). Repeat runs on different days count toward the total.
Do scraping APIs charge for failed requests?
Mostly no, but the definition of failed differs. ScraperAPI charges 200 and 404 responses and requests you cancel before its 70-second window is up. ScrapingBee bills 200, 404, 410 and 413. ZenRows counts 404 and 410 as successful, and Scrape.do counts 2XX, 400, 404 and 410 as successful. Zyte says unsuccessful and rate-limited responses are free, but treats a target's 404 as a successful response. Checked November 16, 2026.
Is a 200 status a successful scrape?
No. A block or challenge page can come back as a 200, which is why published benchmarks add a title, page-size or text check. Count a URL as passed only when the fields you need are present and match a copy you checked by hand.
Which scraping API's free credits last longest?
ZenRows' free plan gives 5,000 credits every month with no card, so it isn't a time-boxed trial. Scrape.do lists its 1,000 free credits as monthly too. ScraperAPI's trial is 7 days, and Zyte's $5 credit covers the first billing month (its pricing page says 30 days). Scrapfly says its 1,000 signup credits have no time limit. ScrapingBee and Scrapingdog don't say when their free credits expire.
Can I test a scraping API on any website?
No. Only test on pages you're allowed to scrape: your own sites, practice sandboxes such as toscrape.com, or sites whose terms and robots.txt permit it. RFC 9309 says robots.txt rules are not a form of access authorization, so an allow line isn't permission either, and vendor policies such as Zyte's bar breaching security or authentication measures.
Can I trust published scraping API benchmarks?
Use them for method, not for your decision. The ones I read are run by a vendor (ZenRows), funded by vendors who subscribe to the benchmark service (AIMultiple), or published by an affiliate site (Proxyway), and none of them used your target pages.
How do I see what each scraping API request cost?
Read the vendor's cost field on every response: sa-credit-cost at ScraperAPI, X-Scrapfly-Api-Cost at Scrapfly, Scrape.do-Request-Cost at Scrape.do, and the cost field in ScrapingBee's JSON response (Spb-auto-cost in Auto-Mode). Scrapfly also offers cost_budget, which fails a request rather than let it cost more than you set.
Sources
27 sources, accessed .
Official pricing, docs, changelogs and status pages 20
- ScraperAPI pricing and free trial
- ScraperAPI docs: credits and request costs
- ScraperAPI docs: API status codes
- ScrapingBee pricing
- ScrapingBee documentation: credits and status codes
- ZenRows pricing and free plan
- ZenRows docs: concurrency
- Scrapfly pricing
- Scrapfly docs: billing and cost reporting
- Zyte API docs: pricing
- Zyte pricing page
- Zyte API docs: errors and successful responses
- Zyte acceptable use policy
- Scrape.do pricing
- Scrape.do docs: request costs
- Scrapingdog pricing
- Bright Data Web Unlocker pricing
- Oxylabs Web Scraper API pricing
- RFC 9309: Robots Exclusion Protocol
- NIST e-Handbook 7.2.4.1: confidence intervals for a proportion
GitHub code, issues and releases 1
Independent benchmarks and published studies 2
Other blogs 2
Last updated: 16 November 2026. Researched and drafted with AI tools; human editorial review and publication approval are pending.


