Explainer
Need JavaScript rendering to scrape a page? How to check
Five checks with curl and Chrome DevTools that show if a page needs JavaScript rendering before you scrape it, and what rendering costs in API credits.

If the value you need is already in the raw HTML, or in a block of JSON inside a script tag, you don’t need to pay for JavaScript rendering. A rendered request costs 5 or 10 credits where a plain one costs 1 on the three scraping APIs whose docs list both costs, so the choice adds up fast.
This guide gives you five checks in order of cost, explains what Google does with JavaScript, and prices the wrong choice at 100,000 pages. It’s about diagnosis only: if a page shows a challenge or a block, that’s not a rendering problem, and the answer is to stop.
Short on time?
Short on time? Here's my take.
- The value is in the raw HTML
- Plain fetch
- 1 credit on ScrapingBee, ScraperAPI and ZenRows, and free with curl.
- The value is in __NEXT_DATA__, JSON-LD or another script blob
- Plain fetch plus a JSON parse
- The data shipped with the HTML. Parse it; no browser needed.
- The value arrives from a JSON endpoint in the Network tab
- Check that endpoint's terms before calling it; otherwise render
- A documented or permitted endpoint is cheaper than a browser. An undocumented one may be off-limits.
- The value only appears after JavaScript runs or after scrolling
- Render, selectively
- Turn rendering on only for the page types that need it. On ScrapingBee, set render_js=false as your default.
Prices verified . Sponsors cannot buy a verdict or a position.
Why do some pages come back empty? #
Because the server sent a shell and JavaScript fills it in later. A plain fetch gets the shell; a browser gets the finished page.
Google’s web.dev guide gives the three terms you need. Server-side rendering (SSR) is “Rendering an app on the server to send HTML, rather than JavaScript, to the client”. Client-side rendering (CSR) is “Rendering an app in a browser, using JavaScript to modify the DOM”. web.dev.
Hydration sits between them: “Running client-side scripts to add application state and interactivity to server-rendered HTML”. A hydrated page usually has its content in the HTML already. web.dev.
The difference shows up as two views of the same page. Chrome’s docs put it this way: “HTML represents initial page content and the DOM represents active, current page content.” And “When JavaScript adds, removes, or edits nodes, the DOM becomes different than the HTML.” Chrome DevTools.
View source shows the HTML. Inspect element shows the DOM. A scraper without a browser only ever sees the first.
How do you check a page in five minutes? #
Run five checks in order and stop at the first one that finds your value. Each step is cheaper than the next.
The five-minute check
- Fetch the raw HTML and search it. Save the page with curl and search the file for one value you need, such as a price or a title. If it’s there, no rendering is needed.
- Look in the script tags. Search the same file for
__NEXT_DATA__,application/ld+jsonor other JSON inside<script>tags. If the value is there, parse the JSON.- Search the Network tab. In Chrome DevTools, filter Network to Fetch/XHR, reload, and search the responses for the value. If a JSON response holds it, that’s the data source, so check you may use it.
- Disable JavaScript and reload. If the value disappears, the page renders client-side.
- Only then render, and wait for a selector or a response, not a fixed delay.
Steps written from the curl manual, Chrome DevTools docs, web.dev and Playwright. Here’s each one in more detail.
Step 1: fetch the raw HTML #
This curl command saves the page as the server sends it, with no JavaScript run. I wrote it from the curl manual, not from a run.
curl -sL --compressed -A "YourBot/1.0 (+mailto:you@example.com)" "https://example.com/product/123" -o page.html
grep -c "49.99" page.html
The flags: -s is silent mode, -L follows a moved page, --compressed asks for a compressed response and decompresses it, and -o writes to a file instead of the screen. -A sets the user agent; the manual describes it as “Specify the User-Agent string to send to the HTTP server”. curl man page.
Put a real name and contact in that user agent. It tells the site who you are, which is the polite default for any collection.
Search for something specific to the page, not a word that appears in every template. A count above zero means the value shipped in the HTML.
Step 2: search the script tags #
Many server-rendered pages ship their data twice: once as HTML, and once as JSON for the browser to hydrate. web.dev says “most server-side rendering solutions serialize the response from a UI’s data dependencies as script tags in the document”. web.dev.
So a page that looks empty in its visible HTML can still hold everything in a script tag. The next section covers the three common shapes.
Step 3: search the Network tab #
Open Chrome DevTools, go to Network, filter to Fetch/XHR and reload. Then press Cmd+F or Ctrl+F; the search runs “across request headers, payloads, and responses”. Chrome DevTools Network reference.
If a JSON response holds your value, the Response tab shows the full body, and right-click, Copy, “Copy as cURL” gives you the exact request. Chrome DevTools Network reference.
Before you call that endpoint directly, read the site’s terms. An undocumented internal API isn’t automatically yours to use; my guide to whether web scraping is legal covers where terms and access rules come in.
Step 4: disable JavaScript #
In DevTools, open the Command Menu with Cmd+Shift+P on a Mac or Ctrl+Shift+P elsewhere, type javascript, and run “Disable JavaScript”. It stays off in that tab only while DevTools is open. Chrome DevTools.
Reload. If your value vanishes, the page builds it client-side, and step 5 applies.
Step 5: render, and wait for the right thing #
Scrapfly’s own docs give the same advice as this list: “try scraping without javascript rendering, check whether desired content exists”. Scrapfly. Only render when the first four checks fail.
When you do render, wait for the element or response you need. Web Scraper’s guide says it well: “Fixed delays are easy to configure but weak as readiness conditions.” Web Scraper.
What if you get a challenge page instead? #
Then the page doesn’t need rendering; the site is blocking automated access. A CAPTCHA or a “verify you’re human” screen is a no, and this guide doesn’t cover getting past it.
Stop and check whether you have permission to collect that page. Is web scraping legal? is the place to start.

Is the data hiding in a script tag? #
Often, yes. Three shapes cover most cases: Next.js’s __NEXT_DATA__, JSON-LD structured data, and a framework’s serialized state.
Next.js __NEXT_DATA__ #
Sites on the Next.js Pages Router embed each page’s data in a script tag. The Next.js docs say the router “serializes page data as __NEXT_DATA__ JSON, which the client must parse before hydrating”, and that this data “is inlined in every HTML response”. Next.js.
In the browser console, the docs’ own one-liner reads it:
JSON.parse(document.getElementById("__NEXT_DATA__").textContent)
From a script, find the <script id="__NEXT_DATA__"> element in the raw HTML and parse its text as JSON. No browser needed. Next.js warns developers when this payload passes 128 kB, so expect it to be large on content-heavy pages. Next.js.
One catch: the newer App Router “streams data via Server Components and is not affected”, so App Router pages generally don’t carry __NEXT_DATA__. Next.js. Search the raw HTML anyway; the value may still be there as plain text.
JSON-LD #
Google recommends JSON-LD for structured data “if your site’s setup allows it”, and it lives in <script type="application/ld+json"> blocks. If the site marks up its products or articles this way, the fields you want may already be there as clean JSON. Google structured data.
The catch here is that “Google can read JSON-LD data when it is dynamically injected”. Some sites add it with JavaScript, so it won’t always be in the raw HTML. Google structured data. Step 1 tells you which kind you have.
Other serialized state #
Other frameworks serialize state under their own names. Search the raw HTML for a long <script> block that starts with { or [, or for a distinctive value you saw on the page. If it’s there, parse it.
What does Googlebot do with JavaScript? #
It renders it, but in a separate phase. Google says it processes JavaScript apps “in three main phases”: crawling, rendering and indexing. Google JavaScript SEO basics.
Between crawling and rendering, pages wait in a render queue. A page “may stay on this queue for a few seconds, but it can take longer than that”. Then “a headless Chromium renders the page and executes the JavaScript”, using “an evergreen version of Chromium”. Google JavaScript SEO basics.
You’ll see this called “two waves” in older articles. Google’s current page doesn’t use that phrase; it describes three phases and a queue.
Google’s advice to site owners is the mirror of this guide: “Keep in mind that server-side or pre-rendering is still a great idea.” And “If the content isn’t visible in the rendered HTML, Google won’t be able to index it.” Google JavaScript SEO basics.
Google has also moved away from serving bots a separate rendered version. “Dynamic rendering was a workaround and not a long-term solution”, and Google now writes “we recommend that you use server-side rendering, static rendering, or hydration as a solution.” Google dynamic rendering.
If the page is your own site, you don’t need any of the five checks. Search Console’s URL Inspection or the Rich Results Test shows the rendered HTML Google sees. Google JavaScript SEO basics.
What does rendering cost on a scraping API? #
Several times a plain request. On the docs I read today, rendering costs 5 credits on ScrapingBee, ZenRows and Scrapfly and 10 on ScraperAPI, and a plain request is 1 credit on the first three.
| Request type | ScrapingBee | ScraperAPI | ZenRows | Scrapfly |
|---|---|---|---|---|
| Plain HTML, no rendering | 1 (with render_js=false) |
1 | 1 | Not on the page I opened |
| JavaScript rendering | 5 (on by default) | 10 | 5 | 5 |
| Premium proxy + rendering | 25 | 25 | 25 | Not opened |
Credits per request, verified 2026-11-16: ScrapingBee docs, ScrapingBee credits, ScraperAPI credits, ScraperAPI JS FAQ, ZenRows, Scrapfly.
The default is the trap. ScrapingBee’s render_js parameter defaults to true and “costs 5 credits per request”, so an unchanged call pays for rendering whether the page needs it or not. ScrapingBee docs.
ScrapingBee’s credit article, updated 07/08/2026, also lists premium without rendering at 10 credits and stealth at 75, where JavaScript can’t be turned off. ScrapingBee credits.
ScraperAPI states that “All JS Rendered requests cost 10 API Credits”, and its own FAQ says “We highly recommend that you only use JS rendering if you absolutely need it”. Rendered requests are limited to 10 per second by default. ScraperAPI JS FAQ.
Its credits page lists the same 10 under an “Extra Cost” heading, so I go by the FAQ’s wording that 10 is the total. ScraperAPI credits.
ZenRows prices rendering as a flat 5 credits, not a multiplier: “A JavaScript-rendered request always costs 5 credits, a protected request always costs 25, on every plan.” ZenRows.
Scrapfly says rendering “will cost 5 Scrape API Credits” and lets you set rendering_wait in milliseconds, up to 25 seconds. Scrapfly.
Billing rules differ too. ScraperAPI bills responses with a 200 or 404 status, while ZenRows charges “per successful request”. ScraperAPI credits, ZenRows. How failed requests and retries are billed is covered in how scraping API credits work.
What’s the bill at 100,000 pages? #
Rendering only the pages that need it cuts the credit bill by 64% to 72% in my estimate below. The share of pages that need rendering is an assumption: I found no reliable public figure, so I picked 20% for illustration.
My estimate. Assumptions: 100,000 pages a month; 20% need rendering and 80% don’t; every request succeeds and is billed once; standard proxies, no premium.
| Provider | Render every page | Render only the 20% | Saving |
|---|---|---|---|
| ScraperAPI | 100,000 × 10 = 1,000,000 credits | (20,000 × 10) + (80,000 × 1) = 280,000 credits | 72% fewer |
| ScrapingBee (default on) | 100,000 × 5 = 500,000 credits | (20,000 × 5) + (80,000 × 1) = 180,000 credits | 64% fewer |
| ZenRows | 100,000 × 5 = 500,000 credits | (20,000 × 5) + (80,000 × 1) = 180,000 credits | 64% fewer |
| Scrapfly | 100,000 × 5 = 500,000 credits | Can’t compute: plain-request cost not opened | n/a |
My arithmetic on the credit costs above, verified 2026-11-16. No job was run.
The saving percentage is 1 minus the selective bill over the all-rendered bill: 280,000 ÷ 1,000,000 = 28%, so 72% fewer on ScraperAPI, and 180,000 ÷ 500,000 = 36%, so 64% fewer on the other two.
If your real share of JavaScript pages is higher, the saving shrinks; if most of your pages carry __NEXT_DATA__ or JSON-LD, it grows. To turn credits into dollars, use the per-plan rates in my ScraperAPI pricing, ScrapingBee pricing and ZenRows pricing breakdowns, which carry their own verified dates.

When do you really need a browser? #
When the value only exists after something happens: a scroll, a click, or a delayed load. Those pages fail every check above, and a rendering API or your own browser is the honest answer.
The toscrape.com sandbox, built for scraping practice, has pages for each case. quotes.toscrape.com/js is “JavaScript generated content”, /js-delayed is “Same as JavaScript but with a delay (?delay=10000)”, and /scroll loads quotes by infinite scrolling. toscrape.com.
On quotes.toscrape.com/js the quotes are generated by script after load, so a plain, non-rendering fetch gets an empty shell: no quotes, only the “Quotes by: GoodReads.com” line and a Next link.
If you drive the browser yourself, Playwright’s network docs give the two tools you want. Start page.waitForResponse() before the click that triggers the data, so you wait for the response itself, and use page.route() with route.abort() to skip resource types you don’t need, such as images. Playwright.
// A sketch written from Playwright's network docs, not run.
await page.route('**/*', route =>
route.request().resourceType() === 'image' ? route.abort() : route.continue());
const responsePromise = page.waitForResponse('**/api/items*');
// ...trigger the click or scroll that loads the data here...
const response = await responsePromise;
On a rendering API, the equivalent is a wait setting. Scrapfly offers both a fixed rendering_wait and a wait_for_selector option; prefer the selector, for the reason Web Scraper gives about fixed delays. Scrapfly.
How slow is rendering compared with a plain fetch? One vendor guide says “ten to twenty times slower” but shows no measurement, and I found no independent figure. Zyte on DEV. If your choice is broader than fetch versus render, my guide to search, fetch or browser covers it for AI agents.
What I would do #
Before choosing a plan, I’d run the five checks on 10 sample URLs for each page template on the target site: product pages, listing pages, articles. Then I’d turn rendering on per template, not per site.
On ScrapingBee I’d set render_js=false as my default and switch it on only for the templates that fail step 4. On the others, I’d leave rendering off and add it per URL pattern.
If the data is in __NEXT_DATA__ or JSON-LD, I’d parse that and skip rendering entirely. And if a page answers with a challenge, I’d stop rather than escalate.
For choosing the API itself, the scraping APIs hub picks up from here.
FAQ
Is "view source" the same as "inspect element"?
No. View source shows the HTML the server sent. Inspect element shows the live DOM, which changes when JavaScript adds, removes or edits nodes. Chrome's docs call the HTML the initial page content and the DOM the active, current content.
Does Google render JavaScript?
Yes. Google describes three phases, crawling, rendering and indexing, with pages waiting in a render queue before a headless, evergreen Chromium runs the JavaScript. Google still says server-side or pre-rendering is a great idea.
Why does my scraper get an empty div?
Usually because the page renders client-side: the server sends a shell and JavaScript fills it in. Before paying for rendering, check the raw HTML for a script tag holding the data as JSON, and check the Network tab for a JSON response.
How many credits does JavaScript rendering cost?
On the docs I read on November 16, 2026: 5 credits per request on ScrapingBee, ZenRows and Scrapfly, and 10 on ScraperAPI. A plain request is 1 credit on ScrapingBee, ScraperAPI and ZenRows.
Does ScrapingBee render JavaScript by default?
Yes. ScrapingBee's render_js parameter defaults to true, at 5 credits a request. Set render_js=false and a classic request costs 1 credit.
Is a CAPTCHA or 'verify you're human' page a rendering problem?
No. That's the site blocking automated access, not a page that needs JavaScript. Stop there and check whether you have permission to collect the page.
Sources
20 sources, accessed .
Official pricing, docs, changelogs and status pages 18
- Google Search Central: JavaScript SEO basics
- Google Search Central: dynamic rendering as a workaround
- web.dev: Rendering on the web (updated Jan 5, 2026)
- Next.js: large page data and __NEXT_DATA__
- Google Search Central: introduction to structured data (JSON-LD)
- Chrome DevTools: get started with viewing and changing the DOM
- Chrome DevTools: disable JavaScript
- Chrome DevTools: Network features reference
- curl man page
- Playwright: network
- ScrapingBee: API documentation (render_js)
- ScrapingBee: credit system explained (updated 07/08/2026)
- ScraperAPI: credits and request costs
- ScraperAPI: JS rendering FAQ
- ZenRows: pricing and credit costs
- Scrapfly: JavaScript rendering
- toscrape.com: scraping sandbox pages
- Quotes to Scrape: JavaScript-generated version


