Skip to content
2.5Advanced9 min

JavaScript Rendering and AI Crawlers: The Visibility Divide

Lucas Blochberger
Definition

AI crawlers like GPTBot, ClaudeBot and PerplexityBot cannot execute JavaScript. Vercel's analysis of over 500 million GPTBot fetches found zero evidence of JavaScript execution. Client-rendered content is invisible to approximately 70 percent of AI crawlers, so server-side rendering is a visibility prerequisite, not a performance choice.

Key Takeaways

  • AI crawlers cannot execute JavaScript: zero evidence in 500M+ GPTBot fetches
  • GPTBot fetched JS files (11.5% of requests) but does not execute them
  • ClaudeBot downloaded JS (23.84%) but does not render
  • Only Googlebot, Google Gemini, and AppleBot use browser-based rendering
  • Client-rendered content is invisible to approximately 70% of AI crawlers
  • SSR is a visibility requirement, not a performance optimization
  • Google has officially marked Dynamic Rendering as deprecated (August 2022)
  • The "9x rendering tax" on crawl budget is well documented

The technical SEO landscape has split into two parallel worlds: Google renders JavaScript, AI crawlers don't.

The Evidence

Vercel's and MERJ's joint analysis of over 500 million GPTBot fetches found zero evidence of JavaScript execution. GPTBot fetches JavaScript files (11.5 percent of requests) and ClaudeBot downloads them (23.84 percent), but neither can execute them. PerplexityBot, Meta-ExternalAgent, CCBot, and Bytespider also lack rendering capability.

OnlyGooglebot(Headless Chromium),Google Gemini(via Googlebot infrastructure), andAppleBotuse browser-based rendering.

The Consequence

Client-side rendered content is invisible to approximately 70 percent of AI crawlers. For Next.js 15, the App Router with Server Components offers an enormous structural advantage: components render on the server with zero client-side JavaScript and deliver complete HTML to every crawler type.

Dynamic Rendering is Dead

Google officially marked Dynamic Rendering as a "workaround, not a long-term solution" in August 2022. By 2025, Google removed the implementation sections entirely and switched to past tense: Dynamic Rendering "was" a workaround.

The Rendering Tax

The oft-cited "9x rendering tax on crawl budget" comes from Onely and Prerender.io. Onely documented delays of up to 313 hours for deep JavaScript pages versus 36 hours for static HTML. The exact multiplier varies, but the direction is clear.

Data & Statistics

No major AI crawler renders JavaScript (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Bytespider, Meta-ExternalAgent, PerplexityBot); ChatGPT crawlers fetch JS in 11,50 percent, Claude in 23,84 percent of requests, but do not execute it

Vercel (mit MERJ), The rise of the AI crawler (2024)

GPTBot generated 569 million requests and Claude 370 million requests per month in the Vercel network, together around 20 percent of Googlebot volume of 4.5 billion requests

Vercel Blog, The rise of the AI crawler (2024)

404 rate of crawlers: ChatGPT 34,82 percent, Claude 34,16 percent, Googlebot only 8,22 percent of fetches

Vercel Blog, The rise of the AI crawler (2024)

Googlebot required around nine times more time to crawl JavaScript pages than pure HTML pages (313 hours versus 36 hours); it followed a JS-injected link in 52 instead of 25 hours

Onely, Ziemek Bucko, Rendering Queue: Google Needs 9X More Time To Crawl JS Than HTML (2022)

Median difference between rendered and raw word count on homepages: 13,6 percent (mobile), 17,5 percent (desktop), up to 32 percent at the 10th percentile on desktop

HTTP Archive Web Almanac 2024, SEO-Kapitel (2024)

Median JavaScript payload: 558 kilobytes (mobile), 613 kilobytes (desktop); around 44 percent (206 kilobytes) of delivered bytes remain unused during loading

HTTP Archive Web Almanac 2024, JavaScript-Kapitel (2024)

48 percent of mobile and 56 percent of desktop pages pass all three Core Web Vitals; good LCP is achieved by only 62 percent (mobile) versus 74 percent (desktop)

HTTP Archive Web Almanac 2025, Performance-Kapitel (2025)

Google holds a search engine market share of 81,87 percent in Austria in May 2026, Bing 9,01 percent, DuckDuckGo 2,75 percent [Austria]

StatCounter Global Stats, Search Engine Market Share Austria (2026)

In Austria, there were around 8.69 million internet users in January 2025 with a penetration of 95.3 percent

DataReportal, Digital 2025: Austria (2025)

ChatGPT, Perplexity and Claude could not read the content of a client-side rendered page because it relied on JavaScript rendering

GSQI, Glenn Gabe, AI Search and JavaScript Rendering (2025)
Summarize with AIChatGPTClaudePerplexity

Opens the chat with a prepared prompt.

FAQ

Can AI crawlers like GPTBot and ClaudeBot execute JavaScript?
No. Joint analysis by Vercel and MERJ confirms that no major AI crawler renders JavaScript, including GPTBot, ClaudeBot, PerplexityBot, Bytespider and Meta. They sometimes download JavaScript files (ChatGPT in 11.50 percent, Claude in 23.84 percent of requests), but do not execute them and only read the raw HTML.
Why is server-side rendering important for AI visibility?
Because AI crawlers only process the server-delivered HTML. If the main content only appears in the client-side rendered DOM, it is invisible to ChatGPT, Perplexity and Claude. SSR, SSG or ISR deliver the content ready in the HTML, making it readable for all crawlers. In a documented test, ChatGPT, Perplexity and Claude were unable to read a pure CSR page.
Which content must be included in the server-delivered HTML?
The core text content, title and meta description, structured data according to Schema.org as JSON-LD, internal linking as real HTML links, hreflang attributes and the canonical tag. These elements control indexing, language assignment and evaluation by AI systems, and are lost if they are only generated via JavaScript.
Does Googlebot render JavaScript, and is that sufficient for visibility?
Googlebot renders JavaScript, as do Google Gemini and AppleBot. However, this is not sufficient, as AI crawlers do not. Moreover, JavaScript costs time even with Google: crawling takes around nine times longer than with pure HTML (313 versus 36 hours in the experiment), which extends time-to-index.
How do I check if my page is visible to AI crawlers?
Compare the raw HTML with the rendered DOM, for example via the page source or a direct HTML fetch. If the main content is missing in the source, there is a CSR problem. Additionally, Google Rich Results Test, URL Inspection in Search Console, a JavaScript crawl with Screaming Frog and server log analysis of AI bots show what is actually readable.
Should I block or allow AI crawlers via robots.txt?
This is a deliberate consideration. Via robots.txt, GPTBot, ClaudeBot, PerplexityBot and Google-Extended can be controlled specifically. Blocking prevents the use of content in AI systems, but also excludes your own brand from visibility in their responses. Those who want to be cited in ChatGPT or Perplexity should allow the relevant crawlers.
Can cookie banners hide content from crawlers?
Content should not be hidden behind consent. If the main content is only loaded after consent via JavaScript, AI crawlers only see the banner. GDPR-compliant is to deliver non-personal content server-side and limit consent management to tracking and non-essential cookies. This keeps content visible without violating data protection.

How does your website perform?

Get a free, AI-powered SEO report of your website by email: technical SEO, on-page, keywords & competitors. No obligation.

Get a free SEO audit →

Or go straight to SEO Agency in Vienna

Previous← CLS Optimization: Ensuring Visual StabilityNextNext.js 15 SEO Architecture: SSR, PPR and Server Components →