Explainer
Is Web Scraping Legal? What US, EU and Indian Law Says
Is web scraping legal? It depends on access, terms, copyright and privacy. What hiQ, Van Buren, EU mining rules and India's IT Act say. Not legal advice.
Web scraping does not have one universal legal status. I would decide whether a particular collection can proceed only after checking access permission, applicable terms, copying and reuse rights, personal data and jurisdiction. US computer-access law, EU text-and-data-mining exceptions and India’s permission rules ask different questions. CFAA, EU DSM Directive, India IT Act.
This is general research, not legal advice. If your product depends on an unresolved answer, get qualified legal review of the exact service, data and intended use before collection.
Short on time?
Here's my take
- Before collecting
- Resolve permission and intended use
- Keep access, terms, content rights and personal data as separate review questions.
- Across borders
- Review the relevant jurisdiction
- Do not export a narrow US access holding into EU or Indian rules.
- Unclear facts
- Pause for qualified review
- Record the exact service, data, licence and reuse before deciding to proceed.
Prices verified . Sponsors cannot buy a verdict or a position.
What should you check before you scrape? #
Before scraping, I would separately check access permission, terms, content rights, personal data and jurisdiction. A favorable computer-access argument does not supply a copyright licence, and a copyright exception does not remove personal-data obligations. Computer-access statute, fair-use statute, EDPB AI-model opinion.
| Question | What I would record | What remains separate |
|---|---|---|
| Access | Who authorized access, to which service and by which method | Copying and onward use |
| Terms and notice | Applicable agreement, acceptance and restrictions | Whether a computer-access claim succeeds |
| Content rights | Licence, applicable exception, works and output use | Privacy and database rights |
| Personal data | Data categories, people affected, applicable legal basis | Whether a page can be opened publicly |
| Geography | Relevant people, operator, collection and use locations | A ruling from a different jurisdiction |
This is my review worksheet, not a legal safe harbor. Its separation follows the limitations in hiQ, the CV-Online database judgment and the EDPB’s scraping announcement.
Is web scraping legal in the US? #
The US cases I read support narrow conclusions about particular access and use, rather than a nationwide permission slip. The CFAA distinguishes access without authorization from exceeding authorized access, alongside its other statutory elements. 18 USC §1030.
In Van Buren, the Supreme Court held that an improper purpose did not alone make access to information the person was entitled to obtain exceed authorized access. Footnote 8 left open whether the relevant authorization limits depend only on technical restrictions or also on contracts or policies. Supreme Court majority opinion, pp. 13 n.8 and 20.
In the hiQ/LinkedIn case, the Ninth Circuit affirmed a preliminary injunction on the public-profile facts before it. The opinion expressly left room for contract, copyright, privacy and other claims; it did not declare every public-site scraper immune. 2022 opinion, pages 39–44.
The later record matters: a December 8, 2022 consent judgment entered $500,000 and a permanent injunction against hiQ. An agreed judgment is a different procedural event from the earlier appellate access analysis, but leaving it out would give the wrong impression of the dispute. Consent judgment, pp. 2–4.
For a proposed collection involving credentials, a paywall, a cease-and-desist or an effective access restriction, I would stop for specific review. I would not extrapolate a public-profile preliminary-injunction decision to those facts. Copyright and anti-circumvention also need separate analysis under §107 and §1201.
Is web scraping legal in Europe? #
EU text-and-data-mining provisions are conditional exceptions, not general ownership of anything online. Article 3 of the DSM Directive concerns qualified research organizations and cultural heritage institutions with lawful access; Article 4 provides a broader exception conditioned on lawful access and the absence of an appropriate express rights reservation. Directive 2019/790, Articles 3–4.
Article 7’s protection against contrary contractual terms lists specified exceptions and does not list Article 4. I would have counsel check the national implementation, the actual rights reservation and the applicable agreement instead of assuming that any commercial mining project qualifies. Articles 4 and 7.
Database rights can remain relevant even when a database is publicly accessible. In CV-Online Latvia, the CJEU required examination of substantial investment and the risk to its recovery in the particular circumstances. Judgment, paragraphs 35–47.
Personal data adds another layer. The EDPB’s AI-model opinion calls for case-specific anonymity and legitimate-interest assessments; public availability does not settle them. Opinion 28/2024 announcement. If the data is meant for model training, my guide to whether scraping for AI training is legal goes through those steps.
The EDPB’s July 2026 scraping materials discuss lawful basis, transparency and sensitive-data conditions. Guidelines 03/2026 are still a version 1 consultation draft: feedback closed on October 30, 2026, and I found no adopted final version on November 11. They are not a new statute, and their discussion of robots signals must be read in that status; my explainer on whether robots.txt is legally binding covers that question separately. July announcement, consultation status, version 1, paragraphs 25–30 and 43–47.
Is web scraping legal in India? #
I would review permission and use under Indian law directly. Section 43 of the IT Act addresses specified acts without permission, including access, downloading, copying and extraction; section 66 adds dishonest or fraudulent conduct. These provisions should not be reduced to the US CFAA test. IT Act, sections 43 and 66.
The enacted DPDP Act’s public-data exclusion is narrower than “everything found on the internet”: section 3(c)(ii) identifies publication by the data principal or by someone legally obliged to publish it. Commencement is phased, and the November 13, 2025 notification places the main sections 3–17 generally eighteen months after publication. I am not treating either that exclusion or all main duties as operative before that point. DPDP Act section 3, commencement Gazette.
On copyright, the December 2025 DPIIT working paper describes purpose-limited fair dealing and proposes a mandatory licensing framework. The government’s announcement calls it a proposal seeking feedback; neither source establishes that a commercial-training licence has been enacted. I could not certify the current complete Copyright Act or subsequent reform status. DPIIT working paper, government announcement.
| Region | Useful starting point | Assumption I would reject |
|---|---|---|
| United States | Access facts, contract, copyright, anti-circumvention | “hiQ legalized every public-site collection” |
| European Union | Lawful access, TDM conditions, rights reservations, privacy, database rights | “Publicly visible means freely reusable” |
| India | Permission under the IT Act, applicable copyright rules, staged data-law commencement | “An enacted exclusion is already fully operative” |
The comparison summarizes the cited US opinion, EU Directive and Indian statute with the commencement notification. It does not cover every local law.
What did the court decide in Reuters v. ROSS? #
In September 2026, a Third Circuit panel affirmed partial summary judgment against ROSS, including rejection of its fair-use defense for the copyrighted headnotes at issue; the system was non-generative, and direct competition mattered. Reuters v. ROSS panel opinion.
That holding is not an answer about every generative model.
ROSS filed a petition for rehearing on November 3, 2026 (Third Circuit docket 25-2153, entry 225), and the docket I read on November 11 shows no order on it yet, so I would not treat the appeal as finished. My AI-training legal guide separates acquisition, retention, training and output use. Law360 report.
Is scraping Amazon legal? #
I cannot give an Amazon-specific yes or no from this research. I could not read the current retail Conditions of Use, and I would not substitute unrelated employment or marketplace-worker terms for the service you intend to use.
My recommendation is explicit permission or appropriately licensed access, plus review of the exact service, region, data categories and reuse. A US public-profile decision does not decide those facts, and access-control questions remain separate. hiQ limitations, anti-circumvention statute.
What have the SerpApi lawsuits decided so far? #
Each order decides only the step in front of it. On November 9, 2026, the Google court granted SerpApi’s motion to dismiss the amended complaint’s two DMCA anti-circumvention claims, with leave to amend within 21 days only to cure the deficiencies it identified. For each type of licensed content Google relied on, the court found missing allegations: that the content was copyrighted, that it appeared in the results SerpApi allegedly scraped, or that the copyright owner authorized SearchGuard. That is a pleading ruling, not a finding that scraping Google Search is lawful. ECF 62, pp. 1, 6–7 and 19.
The court had vacated the October 13 hearing and decided the motion on the papers; Google’s October 12 statement of a recent decision was a party filing. ECF 60, ECF 61.
In the separate Reddit case, a November 9 order gave SerpApi until November 23 to amend its counterclaims or stand on its opposition to Reddit’s motion to dismiss them, and a November 10 order dealt with Perplexity’s counsel; neither is a merits ruling. Reddit ECF 183, ECF 186. I read these court PDFs on November 11, 2026, not the complete dockets, so later filings remain a release-day check. See the case-specific Google guide, while keeping its dated scope in view.
What should you write down before you start? #
I would create a short permission record: target and service; access method; applicable terms; robots snapshot; data categories; copyright or database basis; intended retention and reuse; relevant countries; unresolved questions; and the person approving the work. This is my recommended review process, not a guarantee of compliance.
I would pause collection when the record depends on bypassing controls, treating an unclear licence as permission, or assuming that a vendor accepts all downstream liability. A robots rule itself is not access authorization. RFC 9309, section 1. If your plan routes traffic through household IPs, whether residential proxies are legal adds the questions that come with them.
The data hub, robots legal explainer, AI-training guide and scraping API pillar cover the related decisions. A tool choice should come after the permission decision; when you get there, scraping API vs proxies compares buying a managed API with running your own proxies.
FAQ #
Is scraping public data always legal? #
No. Public visibility does not settle access, contract, copyright, privacy or database-right questions. Review the specific collection and use. hiQ opinion, CV-Online judgment.
Did hiQ make web scraping legal in the US? #
No blanket rule followed. The 2022 appeal concerned a preliminary injunction on public-profile facts, and a later consent judgment imposed payment and an injunction against hiQ. Appellate opinion, consent judgment.
Is scraping Amazon legal? #
This research does not establish an Amazon-specific answer. Current retail terms were inaccessible; obtain permission or licensed access and review the actual service, data and reuse. A general public-profile case does not decide those facts. hiQ limitations.
Is scraping for research automatically fair use? #
No. US section 107 calls for a case-specific four-factor assessment; calling a project research does not resolve it. 17 USC §107.
Does a robots.txt Allow rule mean I have permission? #
No. The protocol is not access authorization and does not itself supply a licence for later reuse. RFC 9309.
Is all of India’s DPDP Act in force yet? #
No. The commencement notification phases the Act; main sections 3–17 generally commence eighteen months after its November 2025 publication. Do not assume the public-data exclusion is operative before that point. Gazette notification.
Sources
25 sources, accessed to .
Official pricing, docs, changelogs and status pages 21
- Van Buren v United States, Supreme Court2021
- hiQ v LinkedIn, NinthCircuit2022
- hiQ consent judgment ECF406
- Reuters v ROSS, ThirdCircuit panel opinion ECF214
- Google v SerpApi ECF60
- Google v SerpApi ECF61
- Google v SerpApi ECF62, order granting dismissal with leave to amend, 9 Nov 2026
- Reddit v SerpApi ECF183, counterclaim amend-or-oppose order, 9 Nov 2026
- Reddit v SerpApi ECF186, counsel order, 10 Nov 2026
- EU DSM Directive2019/790
- CJEU CV-Online Latvia C762/19 judgment
- EDPB Opinion28/2024 AI-models announcement
- EDPB July2026 scraping announcement
- EDPB03/2026 consultation status
- EDPB03/2026 version1 PDF
- India ITAct2000 official text
- India DPDPAct2023 enacted text
- DPDP commencement Gazette GSR843E
- DPIIT AI-copyright Working Paper Part1
- PIB DPIIT publication announcement
- RFC9309 Robots Exclusion Protocol
