How to choose
Buy a web dataset or scrape it yourself? How I'd decide
Buy web data or scrape it? Bright Data charges $250 for 100K dataset records, $150 via its scraper API. So freshness, fields and rights decide.

You need web data (product listings, company records, job posts, reviews or a pile of training text), and you can either buy a file someone already collected or collect it yourself. I priced both routes in the same unit, per 1,000 records, and on list price they’re closer than you might expect.
So price rarely decides this. What decides it is how fresh the data has to be, whether a dataset already has the fields you need, and what the licence actually lets you do with it.
This page sits in the web data hub. If you’ve already decided to collect and you’re weighing a scraping API against your own crawler, my cost model for a scraping API vs your own crawler covers that.
Short on time?
Short on time? Here's my take.
- A dataset already has your fields
- Buy: Bright Data Datasets
- $2.50 per 1,000 records with a $250 minimum for 100K, verified 2026-11-17. That's the lower entry price of the two catalogue price lists I could open. Pick it when a monthly or slower refresh is fine.
- Company, people or job records
- Buy: Coresignal datasets
- From $1,000, with records updated daily, weekly or monthly per its pricing page. The price depends on contract length and record count, so expect a quote.
- You need fresher or custom data
- Collect: Bright Data Web Scraper API or an Apify actor
- $1.50 per 1,000 delivered records pay-as-you-go, or $2.30 per 1,000 results for Apify's Instagram actor. Each run is as fresh as the day you run it.
- Training or research text
- Free: FineWeb, or Common Crawl itself
- $0 licence fee; you pay for storage and compute. FineWeb is ODC-By plus Common Crawl's terms, and neither grants rights in the underlying pages.
- One platform owns the data
- License it from the source
- Reddit's deal with Google, reported at about $60 million a year, shows the shape: first-party access under contract, not a file from a middleman.
- Most teams
- Blend: buy a baseline, collect the changes
- One snapshot gives you coverage, and targeted collection keeps the records you care about current without paying to re-collect everything.
Prices verified . Sponsors cannot buy a verdict or a position.
Pick by situation
| Your situation | Route | My pick | Why |
|---|---|---|---|
| A catalogue dataset has your fields, and monthly or slower is fresh enough | Buy | Bright Data Datasets | $250 for 100K records, the lower entry price of the two catalogue price lists I opened |
| You need B2B company, people or job records | Buy | Coresignal | Datasets from $1,000; records updated daily, weekly or monthly |
| You need fresher than monthly, custom fields or a niche site | Collect | Bright Data Web Scraper API, or an Apify Store actor | Billed per delivered record; each run is live |
| You need training or research text at scale | Free open data | FineWeb first, then raw Common Crawl | $0 licence; cleaned text with a stated licence |
| The data lives on one platform with a licensing programme | License | The platform itself | The contract comes from the party that holds the data |
| You’re like most teams | Blend | A dataset baseline plus collection of changed records | Coverage now, freshness only where it matters |
| Personal data of EU residents is involved | Any route | Whichever fits above, plus a written provenance statement | CNIL treats scraping case by case, and the compliance work stays with you |
You have five ways to get web data, not two #
There are five routes: a catalogue dataset, a marketplace listing, collecting it yourself, free open data, and a licence from the source. Most guides frame this as buy versus scrape, but each route has a different price model and a different owner of the rights.
| Route | What you get | Who sets the price | Examples |
|---|---|---|---|
| Buy from a catalogue | Ready-made records as files, one-time or refreshed | The seller’s price list | Bright Data Datasets, Coresignal |
| Buy through a marketplace | A provider’s listing, billed through your cloud account | Each provider | AWS Data Exchange, Snowflake listings |
| Collect it | Records from a run you trigger | Per record, per event or usage | Bright Data Web Scraper API, Apify Store actors, your own crawler |
| Use free open data | Raw crawled pages or cleaned text | $0 licence fee; you pay compute | Common Crawl, FineWeb |
| License from the source | First-party access under a contract | Negotiated | Reddit’s data deal with Google |
Catalogue sellers are the obvious “buy” option. Bright Data’s product page lists “1,000+” datasets and “21B+” records, delivered as JSON, NDJSON, JSONL, CSV or Parquet (plus XLSX) to Snowflake, S3, Google Cloud Storage, Azure, SFTP or a webhook (Bright Data). Coresignal focuses on company, employee and job data, with full datasets “Starting from $1,000” (Coresignal pricing).
Marketplaces sit between you and many providers. AWS Data Exchange lists “over 3,500 third-party data sets”, with prices “set by the data provider” and billing through AWS Marketplace (AWS pricing). Snowflake listings come as free, limited-trial or paid access, and the provider’s offer defines the purchase terms (Snowflake docs).
Collecting means a scraper API, a ready-made scraper (“actor”) from a store like Apify’s, or your own crawler. You choose the sites, the fields and the timing, and you carry the responsibility for how the data was gathered.
Free open data is the route most guides skip. Common Crawl publishes a new crawl of about two billion pages roughly every month, and FineWeb is a text corpus built from those crawls (Common Crawl blog, FineWeb card).
Licensing from the source applies when one platform holds the data. Engadget reported in February 2024 that Reddit’s licensing deal is worth “about $60 million” a year, citing Bloomberg, with Reuters naming Google as the buyer; Google gets access to “Reddit’s Data API” (Engadget). That figure is reported, not disclosed.
The same report says the deal “does not change Reddit’s Data API Terms” and “API access remains free for non-commercial usage”. So for some sources, the real choice is license versus scrape, and no dataset vendor is involved at all.
One more thing before the numbers: almost everything ranking for this question is written by someone selling one side. Coresignal’s guide says “we recommend buying data”, Bright Data’s names its own products best, and SourceForge’s provider list carries Bright Data affiliate tracking without a disclosure (Coresignal blog, Bright Data blog, SourceForge).
Buy, collect or take it free, side by side #
| Buy a dataset | Collect (scraper API or actor) | Free open data | |
|---|---|---|---|
| Effort to start | Pick a dataset, download or receive it | Supply targets, schedule runs, handle outputs | Download and process at scale yourself |
| List price for 100K records | $250 at Bright Data; from $1,000 at Coresignal | $150 at Bright Data’s API; $230 for Apify’s Instagram actor | $0 licence fee |
| Freshness | The seller’s refresh cycle | Whenever you run it | Monthly crawls (Common Crawl) or a fixed corpus (FineWeb) |
| Control over fields | Whatever the schema has | You choose | Raw pages or plain text |
| Quality evidence | Vendor claims and a sample; I found no independent accuracy data | Yours to check per run | A dataset card (FineWeb); none for raw crawls |
| Rights | The seller’s contract | Site terms and the law apply to you | Licence terms that grant no rights in the pages |
| Upkeep | Renewals and schema changes | Lower with an API; heavy with your own crawler | Your processing pipeline |
What do 100K records cost each way? #
On list price, 100K records cost $250 as a Bright Data dataset and $150 through Bright Data’s own Web Scraper API, both verified 2026-11-17. Over a year of monthly updates the gap grows, and the subscription side needs a quote.
| Route | Unit price | Minimum or plan | Refresh | Licence comes from |
|---|---|---|---|---|
| Bright Data dataset | $2.50 per 1,000 records | $250 for 100K, one-time | One-time; biannual, quarterly or monthly at 25%, 50% or 80% off | Bright Data's terms (not read) |
| Coresignal dataset | Not published | From $1,000 | Records updated daily, weekly or monthly | Your contract |
| AWS Data Exchange | Set by each provider | 1- to 36-month subscriptions | Set by the provider | Provider's Data Subscription Agreement |
| Snowflake listing | Set by each provider | Free, limited trial or paid | Live, read-only access (open-client listings) | Provider's offer |
| Bright Data Web Scraper API | $1.50 per 1,000 records pay-as-you-go; $1.30 over plan | Scale $499 a month includes 384,000 records | Each run is live | You (site terms apply) |
| Apify Store actor | e.g. $2.30 per 1,000 results (Instagram) | Free plan; paid plans from $19 a month | Each run is live | You (site terms apply) |
| Common Crawl | $0 licence | Compute and transfer are yours | Monthly crawls | No rights in the content |
| FineWeb | $0 licence | Storage and compute are yours | Fixed corpus, 2013 onward | ODC-By plus Common Crawl's terms |
Source: Bright Data dataset and Web Scraper API pricing, Coresignal pricing, AWS Data Exchange pricing and FAQ, Snowflake listings docs, Apify pricing, Common Crawl terms and FAQ, FineWeb card; all read 17 Nov 2026. Scroll sideways for more columns.
Sources: Bright Data datasets, Bright Data Web Scraper API, Coresignal, AWS FAQ, Apify, Common Crawl terms.
A few details from those pages change the picture:
- Bright Data datasets come in tiers of 100K, 500K, 1M, 5M and 20M records, plus a “Complete Dataset”. Prices shown in local currency are an estimate, and “USD pricing is binding” (Bright Data pricing).
- Bright Data’s Web Scraper API has a free tier of 5,000 records a month and says “Only pay for what’s successfully delivered” (Bright Data). Failed scrapes are its cost, not yours.
- Apify has Free ($5 of usage), Starter $19, Scale $199 and Business $999 plans a month. Store actors are priced per event, per usage or both: its Google Trends scraper lists $1 per 1,000 plus platform usage (Apify pricing).
- Coresignal also sells API access from $49 a month (Mini, 2,500 credits) up to $5,000 a month (Elite), with a 7-day free trial (Coresignal).
- AWS adds its own charges on top of the provider’s price: storage for data you load and S3 rates for cross-Region transfer (AWS pricing).
There’s one more price point on the record. In Meta’s lawsuit, a dataset of 615 million Instagram user records that Bright Data offered for sale at $860,000 came up as evidence (TechCrunch). That’s about $1.40 per 1,000 records (my arithmetic), from 2024, and an offer reported in a court story, not a price list.
A year of monthly data, priced out #
Here’s 100K records refreshed every month for a year, at list prices verified 2026-11-17. The totals are my arithmetic.
| Route | One pull of 100K | Monthly for a year | What decides the real number |
|---|---|---|---|
| Bright Data dataset | $250 | Quote needed | “Save 80%” on monthly, but the base and the minimum monthly commitment aren’t shown |
| Bright Data Web Scraper API, pay-as-you-go | $150 | $1,800 | The Scale plan ($499 a month) only pays off above about 333K records a month |
| Apify Instagram actor | $230 | $2,760 | Event pricing; usage-priced actors add platform costs |
| Coresignal dataset | From $1,000 | Contract | “agreement duration, record count, and other factors” |
| Common Crawl or FineWeb | $0 licence | $0 licence | Your storage, compute and transfer, not priced here |
Buying 12 separate one-time snapshots from Bright Data at list price would come to $3,000. Its subscription is advertised as a saving, but the pricing page doesn’t say what the 80% is taken off or what the minimum monthly commitment is, so the honest answer is that a quote decides it.
My read: if the subscription quote for your 100K records comes in under $1,800 a year, buying wins on price for a monthly feed. If you need weekly data, the same API arithmetic gives $7,800 a year (52 runs), and daily gives $54,750 (365 runs), which is when you should push hard on a dataset with daily updates.
Keep two caveats in mind. These are list prices for different products, and real quotes vary by target site and volume, so treat the table as a starting point, not a bid.
What the per-record price leaves out #
The per-record price leaves out upkeep, and that’s where the routes really differ. The vendor guides I read agree that the hidden cost of scraping is keeping it working as sites change their layouts and anti-bot defences (Coresignal blog, Zyte, Bright Data blog).
A scraper API absorbs most of that for you, which is what you pay per record for. Your own crawler doesn’t, and the full build-versus-API cost model is in scraping API vs building your own.
Datasets have hidden costs too: columns you never use, cleaning, storage, and a subscription commitment you have to manage. Free data moves the whole cost to your compute bill, and Common Crawl says its data is best accessed from AWS us-east-1, where it’s stored (Common Crawl).
Zyte frames the trade-off as a triangle of cost, time and quality (Zyte). For a dataset, time is the one you win; for collection, it’s control.

How stale is a bought dataset? #
As stale as its last refresh. Catalogue datasets run from one-time snapshots to monthly (sometimes daily) updates, while a scraper run is as fresh as the day you run it. If you need data fresher than the seller’s refresh, collect it.
| Source | How often it’s refreshed | Where I read it |
|---|---|---|
| Bright Data datasets | One-time, biannual, quarterly or monthly on the price list; the FAQ adds daily, weekly and yearly subscriptions | Pricing, product page |
| Coresignal datasets | “Records updated daily, weekly, or monthly” | Pricing |
| AWS Data Exchange | Set by each provider, on 1- to 36-month subscriptions | FAQ |
| Snowflake listings | “Live, read-only access” for open-client listings; how often the data changes is up to the provider | Docs |
| Common Crawl | A new crawl most months; 1.97 to 2.3 billion pages each in 2026 | Blog |
| FineWeb | A fixed corpus built from crawls since 2013, with newer dumps added | Card, page |
| A scraper API or actor | Every run | Your schedule |
Bright Data’s own pages don’t agree on this. Its price list offers one-time to monthly refreshes, its product FAQ lists “daily, weekly, monthly, quarterly or yearly” subscriptions, and its June 2026 blog table lists daily, monthly and quarterly (pricing, product page, blog).
The same goes for size: the product page says “1,000+” datasets and “21B+” records, while the June blog says “350+ domains, 17B+ records”. I’d trust the product page for today’s catalogue and ask in writing whether daily or weekly refreshes are offered for the dataset you want, and at what price.
Bright Data’s blog sums up the split plainly: its table calls datasets “Static or periodically refreshed” and its APIs “Real-time”. Olostep, which sells collection, puts it more bluntly: “Purchased datasets are snapshots, and snapshots decay” (Olostep).
How fast they decay depends on the data. I found no independent figures on how quickly product prices, job posts or company records go stale, so I can’t give you a decay rate, and none of the page-one guides I read gives one either.
Here’s the rule of thumb I’d use instead. Ask how old a record can be before it causes a wrong decision, then compare that with the seller’s refresh cycle. If the record can be older than the cycle, buy; if it can’t, collect.
Bulk crawls have a second kind of staleness: coverage. Common Crawl’s FAQ says it crawls a “randomly selected subset” of sites and that “JavaScript is not executed”, so pages that render their content in the browser can come through empty (Common Crawl FAQ).
I found little first-hand discussion of this choice. One recent Hacker News comment calls big crawl datasets just a snapshot of the internet, and says more specialised data such as code, commits and chat logs “needs to be curated by hand depending on your use case” (Hacker News).
What does the licence actually let you do? #
Only what the seller’s contract says, and the contract can’t give you more than the seller had. The licence doesn’t come with the file: it comes from the provider’s agreement, and for free data, from terms that grant no rights in the pages themselves.
| Source | Who sets the terms | What I found | What I’d ask |
|---|---|---|---|
| Bright Data | Bright Data | Its product page says “Data is ethically obtained and compliant with all privacy laws”; that’s the vendor’s claim, and I didn’t find its licence terms on the pages I read | Training and redistribution rights in writing |
| Coresignal | Your contract | Pricing depends on “agreement duration, record count, and other factors”; terms not read | The same, plus what happens to the data if you stop paying |
| AWS Data Exchange | The provider | Terms are the provider’s Data Subscription Agreement; refunds are set by the provider; the FAQ doesn’t address redistribution | Read each listing’s agreement |
| Snowflake | The provider | “Offers define the purchase terms for a listing” | Read each listing’s offer |
| Common Crawl | Common Crawl, plus each site’s owner | A limited licence to use the service; crawled content “may be subject to separate terms” from its owners; provided “AS IS” | Whether your use fits the owners’ terms |
| FineWeb | Hugging Face, under ODC-By | ODC-By v1.0, “also subject to CommonCrawl’s Terms of Use”; emails and IP addresses anonymised; a PII removal form | Attribution for public use |
| Your own collection | No seller; site terms and the law | Responsibility is yours | See is web scraping legal? |
Sources: Bright Data, Coresignal, AWS FAQ, Snowflake, Common Crawl terms, FineWeb card.
Common Crawl’s terms are the clearest example of how little “free” covers. They say crawled content is “the sole responsibility of the individual or entity from which such Crawled Content originated”, and they disclaim non-infringement (Common Crawl terms). You get access, not rights.
FineWeb’s ODC-By licence is generous about the database itself: you’re free “To share”, “To create” and “To adapt”, and “You must attribute any public use of the database” (ODC-By summary). That covers the compilation, not the web pages inside it, which is why the card adds Common Crawl’s terms on top.
Before you pay anyone, get answers to three questions in writing:
- Internal use: can the whole team use it, for any internal purpose, for as long as you keep it?
- Model training: is training a model on it allowed, and does that change if the model is sold?
- Redistribution: can you share it, resell it, or ship it inside a product, and with what attribution?
Sellers have a reason to answer. An old Hacker News comment from 2017 puts the seller’s side simply: “Finding an organization willing to buy a legally sourced, unique dataset is comparatively easy” (Hacker News). Legally sourced is the selling point, so ask them to show it.
DepositPhotos, which sells licensed media for AI training, offers a test I like for any dataset: is it “usable, defensible, and valuable six months from now?” (DepositPhotos). Defensible is the word that matters here.
What have courts and regulators said? #
The court record says how data was acquired matters, and that applies to what you buy as well as what you scrape. None of these cases is a general green light, and this section isn’t legal advice.
What the record shows (not legal advice)
- Meta v. Bright Data (N.D. Cal., January 2024). The court denied Meta’s motion for partial summary judgment on its breach of contract claim; TechCrunch reports it found no evidence that Bright Data scraped while logged in, treated it as a logged-off “visitor” under Meta’s terms, and found that a dataset of 615 million Instagram user records Bright Data offered for sale at $860,000 didn’t prove logged-in collection. A tortious interference claim remained (TechCrunch).
- Bartz v. Anthropic (June 2025). Judge Alsup ruled that training on books was fair use and that digitising purchased print books was fair use, but pirated copies were not. Later buying a copy of a book it had stolen “will not absolve it of liability for the theft” (NBC News).
- The Anthropic settlement (July 2026). Judge Araceli Martínez-Olguín approved a “$1.5 billion” settlement covering “more than 482,000 books”, “about $3,000 per book” (AP via WTOP).
- CNIL, France’s data regulator (January 2026). “Data scraping is not prohibited per se, but must be analysed on a case-by-case basis.” It asks scrapers to set collection criteria in advance, delete irrelevant data, exclude sites that “clearly oppose the scraping of their content” (for example through robots.txt or CAPTCHAs), and delete sensitive data once aware of it (CNIL focus sheet).
- CNIL on re-use (June 2024). Re-using data to build a training set “involves taking additional precautions” (CNIL). The detailed re-use page wouldn’t open for me, so I don’t list its steps.
What I take from this for buyers: Meta v. Bright Data is about Meta’s own terms and logged-off access, not a ruling that any scraping of public pages is fine. It does show that a court will look at how the data was collected.
Bartz is the case buyers should read. The court split the outcome by how the books were obtained, and the settlement put a price on the bad source. My reading, not a legal opinion: if a dataset seller’s source is bad, buying from it doesn’t make the problem go away, which is why a written provenance statement belongs on your checklist.
For personal data of EU residents, the CNIL sheets make the compliance work yours whichever route you take. For training specifically, I go further on whether scraping for AI training is legal.
When are Common Crawl and FineWeb enough? #
When you need lots of text for training or research and don’t need structured fields or today’s numbers. For product prices, company records or anything with a schema, they’re the wrong tool.
Common Crawl’s data is “free to access by anyone from anywhere”, from s3://commoncrawl/ in AWS us-east-1 or over HTTPS from data.commoncrawl.org, as WARC, WAT and WET files with URL indexes (Common Crawl). Its FAQ offers the data at no cost “for the purpose of research and analysis” and says its crawler obeys robots.txt and Crawl-delay (FAQ).
Each crawl is big. The October 2026 crawl held 2.16 billion pages and 371.19 TiB uncompressed, and Common Crawl’s blog also announced an Opt-Out Registry in September 2025 that publishes legal opt-out requests (Common Crawl blog).
FineWeb gives you text instead of raw crawl files. The card describes “more than 18.5T tokens (originally 15T tokens)” from “96 CommonCrawl dumps” between summer 2013 and April 2024, and the dataset page now lists subsets through CC-MAIN-2025-26 and 25.9 billion rows (FineWeb card, FineWeb page).
It also says “We anonymize email addresses and public IP addresses” and offers a PII removal form.
So my order for training text is FineWeb first, because it’s already text, with a stated licence and a way to request PII removal, and raw Common Crawl when you need a crawl FineWeb doesn’t include or your own filtering. For how to actually query and download Common Crawl, see what Common Crawl is and how to use it; I won’t repeat it here.
Neither one gets you a clean licence for commercial model training. That’s a legal question about the pages inside, not about the free download.

Buy the base, collect the gaps #
For most teams, the best answer is both: buy one snapshot for coverage, then collect only the records that change or that the dataset lacks. Bright Data’s own guide recommends a dataset baseline plus API refreshes (Bright Data blog).
Some sellers build this in. Bright Data’s dataset subscriptions can deliver “New Records” or “Updated Records” rather than the full file each time (Bright Data). Zyte describes a hybrid of building and buying too, and Olostep pitches its own product for the gaps (Zyte, Olostep).
Bear in mind that a vendor selling both halves benefits either way. The case for blending still holds on the numbers.
Here’s an example. Suppose 10% of your 100K records change in a month; that’s an assumption, since I found no published decay rates. A $250 baseline plus 10K re-collected records a month on Bright Data’s pay-as-you-go API ($15 a run) comes to $430 for the year, against $1,800 for re-collecting everything monthly (my arithmetic, list prices verified 2026-11-17).
Blending beats either pure route when three things are true: most of the dataset’s fields are what you need, a minority of records drive your decisions, and you can tell which records those are. If you can’t target the changes, a dataset subscription with “Updated Records” delivery is the simpler fix.
What should you check before you buy? #
Nine things, from the fields and record timestamps to the licence scope, provenance and exit terms. SourceForge’s guide says to use free samples, and Coresignal’s guide puts data quality first, which is a start (SourceForge, Coresignal blog). Here’s the full list I’d work through:
- Fields. Does the dataset have every column you need, and how many records actually fill them?
- Record timestamps. Does each record say when it was collected, not just when the file was built?
- Refresh cadence and its price. Which cadences are on offer for this dataset, and what does each cost in writing?
- A sample. Get one from the exact dataset and check it against the live site.
- Licence scope. Internal use, model training and redistribution, each answered explicitly.
- Provenance. A written statement of how the data was collected, logged-in or not, and from where.
- Personal data. What’s anonymised, and how removal requests are handled (FineWeb has a form; ask paid sellers for theirs).
- Delivery. Format (JSON, CSV, Parquet) and destination (S3, Snowflake, SFTP), plus any storage or transfer charges on your side.
- Exit terms. What you can keep, and what you must delete, if you stop paying.
If you’re leaning toward collecting instead, the same list applies to you, minus the seller. Items 5 to 7 become your job, which is where the CNIL steps above come in.
So which should you pick? #
Pick the route that matches how fresh and how custom your data has to be, then let price settle ties. Here’s how I’d decide on the evidence above.
A catalogue dataset already has your fields, and monthly or slower is fine: buy. Bright Data Datasets has the lower published entry price of the two catalogue price lists I could open, $250 for 100K records ($2.50 per 1,000, verified 2026-11-17). Get the subscription quote before you compare it with collecting.
You need B2B company, people or job data: Coresignal, with datasets from $1,000 and records updated daily, weekly or monthly (verified 2026-11-17). Its API from $49 a month is a cheaper way to check the data first.
You need fresher than monthly, custom fields or a niche site: collect with a per-record scraper. Bright Data’s Web Scraper API at $1.50 per 1,000 delivered records (verified 2026-11-17) bills only on success; Apify’s Store actors list per-result prices, such as $2.30 per 1,000 for Instagram. I compare how Bright Data and Apify bill separately. Build your own crawler only when the build-vs-API cost model favours it.
You need training or research text at scale: start with FineWeb, then raw Common Crawl. The licence fee is $0, but neither grants rights in the underlying pages.
The data lives on one platform that licenses it: license it from the source, as Google reportedly did with Reddit.
You’re like most teams: blend. Buy one baseline, then collect only the records that change.
Personal data of EU residents is involved: whichever route you pick, assume the compliance work stays with you, and get the seller’s provenance in writing.
If I needed 100K product or company records for a monthly report tomorrow, I’d buy one snapshot, check the sample against the live sites, and price a monthly subscription against $1,800 a year of API runs. Whichever came in lower, I’d keep a small collection job for the records that drive decisions, and I wouldn’t sign anything without the three licence answers in writing.
FAQ
Is it cheaper to buy a dataset than to scrape the data?
Not on list price, at least at Bright Data, which sells both. 100K records cost $250 as a dataset and $150 through its Web Scraper API at $1.50 per 1,000 delivered records (verified 2026-11-17). Over a year of monthly updates the API comes to $1,800, and the dataset subscription needs a quote because the base of its "Save 80%" isn't shown. The gap is small enough that freshness, fields and rights decide, not price.
Is it legal to buy data someone else scraped?
Buying doesn't clean the data. The licence you get is the seller's contract, Common Crawl warns that crawled pages may carry their owners' own terms, and in Bartz v. Anthropic the way books were acquired decided which use was fair. CNIL says re-using data needs additional precautions. Ask the seller for its provenance in writing and get legal advice for your use. This isn't legal advice.
How fresh is a purchased web dataset?
As fresh as its last refresh. Bright Data's price list runs from one-time snapshots to monthly refreshes, and its FAQ adds daily and weekly subscriptions. Coresignal says its records are updated daily, weekly or monthly. Common Crawl publishes a new crawl about once a month, and FineWeb is a fixed corpus built from crawls since 2013. A scraper run is as fresh as the day you run it.
Can I use Common Crawl or FineWeb commercially?
Read the terms before you plan on it. Common Crawl offers its data at no cost "for the purpose of research and analysis" and says crawled pages may be subject to their owners' terms, with no rights in that content passed to you. FineWeb is licensed under ODC-By, which asks you to attribute any public use, and it is also subject to Common Crawl's terms. For model training in particular, get legal advice for your use.
Can I share or resell a dataset I bought?
Only if your contract says so. On AWS Data Exchange the provider's Data Subscription Agreement governs, and the FAQ I read doesn't address redistribution. On Snowflake the provider's offer sets the terms. I didn't find public licence terms for Bright Data or Coresignal. FineWeb's ODC-By licence does allow sharing with attribution. Get redistribution rights in writing before you build a product on bought data.
Where can I buy web datasets?
From catalogue sellers such as Bright Data (1,000+ datasets on its product page) and Coresignal for company, people and job data; from marketplaces such as AWS Data Exchange (over 3,500 data products, priced by each provider) and Snowflake listings (free, trial or paid). For training text, FineWeb and Common Crawl are free to download. I couldn't read Kaggle's dataset docs, so I can't vouch for its licences.
What should I check before buying a dataset?
Nine things: that the fields you need are filled in, that records carry timestamps, the refresh cadence and its price, a sample, the licence scope for internal use, model training and redistribution, a written provenance statement, how personal data is handled and removed, the delivery format and destination, and the exit terms if you stop paying.
Sources
29 sources, accessed .
Official pricing, docs, changelogs and status pages 17
- Bright Data: Datasets product page
- Bright Data: dataset pricing
- Bright Data: Web Scraper API pricing
- Coresignal pricing
- AWS Data Exchange pricing
- AWS Data Exchange FAQ
- Snowflake docs: about listings
- Common Crawl terms of use (last updated March 7, 2024)
- Common Crawl FAQ
- Common Crawl: get started
- Common Crawl blog (2026 crawl announcements)
- FineWeb dataset card
- FineWeb on Hugging Face
- Open Data Commons Attribution License (ODC-By) summary
- Apify pricing
- CNIL: focus sheet on web scraping (January 5, 2026)
- CNIL: ensuring the lawfulness of data processing (June 7, 2024)
Community discussions 2
News and press 4
Other blogs 6
- Bright Data: datasets or web scraping APIs (vendor, Jun 18, 2026)
- Coresignal: buying vs scraping data (vendor, updated Dec 18, 2025)
- DepositPhotos: licensed vs scraped data for AI (vendor, May 5, 2026)
- Zyte: build or buy (vendor, Dec 26, 2024)
- SourceForge: data provider guide (Apr 30, 2026; carries Bright Data affiliate tracking)
- Olostep: data marketplaces (vendor, Jul 14, 2026)
Last updated: 17 November 2026. Researched and drafted with AI tools; human editorial review and publication approval are pending.


