Googlebot JavaScript Crawling

The cluster discusses Googlebot's ability to crawl, render, and index JavaScript-heavy websites and SPAs for SEO purposes, including its limitations, execution reliability, and solutions like prerendering or server-side rendering.

📉 Falling 0.4x Web Development
1,497
Comments
20
Years Active
5
Top Authors
#2127
Topic ID

Activity Over Time

2007
3
2008
11
2009
28
2010
77
2011
104
2012
101
2013
189
2014
141
2015
163
2016
79
2017
84
2018
46
2019
108
2020
63
2021
47
2022
56
2023
74
2024
70
2025
50
2026
3

Keywords

SSR JS S3 webdatarender.com XML SFW blogspot.com CSR google.com of.html javascript google js rendered page html pages render crawl crawler

Sample Comments

alxndr Dec 24, 2013 View on HN

Doesn't Google's crawler run JS these days?

nmjohn Dec 5, 2014 View on HN

Google crawlers do index javascript now as many people have already pointed out.The other common practice is to use a "prerender" server which uses a tool like phantomjs to render the HTML of the page. When googlebot or any other robot hits your http server, you check the useragent, and if it matches known robots you forward the request to prerender instead of your app server.

wahnfrieden Jan 16, 2012 View on HN

You should be able to help googlebot index the right content by using different HTTP status codes in the response. Also, although googlebot executes JS, in my experience it doesn't use it for indexing content. My guess is that it's primarily used for the instant preview and for verifying that you're not cheating by hiding SEO keywords on load or whatnot. This could change any day, of course.

b112 Aug 1, 2024 View on HN

I kinda wish Google would not index JS rendered stuff. The world would be so much better.

acchow Apr 20, 2022 View on HN

Googlebot can render with javascript now, so why do we still need SSR for SEO?

BHSPitMonkey Jan 19, 2016 View on HN

Google renders JavaScript when it crawls.

madeofpalk Jan 22, 2016 View on HN

Google crawls Javascript sites, but it doesn't execute AJAX calls so most SPA's still won't be crawlable by Google.

a2tech Oct 14, 2015 View on HN

This is good-one of my current projects for a customer is entirely AJAX/JS rendered and we were worried that Googlebot would have a fit with it.

gpnt Dec 9, 2020 View on HN

Not really, Google and Bing will read the page correctly with Javascript. Check the page insight: https://developers.google.com/speed/pagespeed/insights/?url=...But it would be even better if they could get the information directly on JSON. So much carbon saved ;)

cj Nov 17, 2013 View on HN

One solution is to detect Googlebot and render the page server-side in a headless browser, and send the generated HTML instead of the full JS app. You could even cache the generated HTML to S3 to speed up pageload, which Google likes.If you use Node...http://zombie.labnotes.org/http://prerender.io/http: