AI & bot directory · Search engine
Googlebot: what it is, how to verify it and how to block it
Googlebot is run by Google. Crawls the web for Google Search.
Key facts
| Operator | Google |
|---|---|
| Category | Search engine |
| robots.txt name | Googlebot |
| Follows robots.txt | OK: Yes |
| Can it be verified? | OK: Yes, by IP ranges and reverse DNS |
| Official documentation | |
| Registry checked | Nov 11, 2026 |
User agent
Its name in robots.txt is Googlebot.
Google documents this user-agent string:
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/W.X.Y.Z Safari/537.36W.X.Y.Z stands for a browser version number, which changes over time.
Can you verify Googlebot?
Yes Google publishes the IP addresses Googlebot uses. We download that list every 6 hours: our copy is from Nov 20, 2026 and has 317 entries; Google dated it Nov 20, 2026.
Its addresses also have reverse-DNS names under googlebot.com, geo.googlebot.com; a name only counts if it points back to the same address.
Does Googlebot follow robots.txt?
Yes Google says Googlebot follows robots.txt.
“always obey robots.txt rules when crawling automatically”
How to block or allow Googlebot
To ask it to stay away from the whole site, add this to robots.txt:
User-agent: Googlebot
Disallow: /
To keep it out of one folder only:
User-agent: Googlebot
Disallow: /private/
To allow it while asking every other bot to stay away:
User-agent: Googlebot
Allow: /
User-agent: *
Disallow: /
robots.txt is a request, not a lock. To enforce a block, add a firewall or CDN rule. Because Google publishes Googlebot's addresses, a rule can match them exactly, so a fake Googlebot can't slip through as the real one.
To build one robots.txt for several bots at once, use the robots.txt generator.
How often does Googlebot visit?
Google doesn't publish how often Googlebot visits a site, and iplover doesn't see your traffic. Your own access log is the only reliable count: paste it into the log checker, which counts each crawler's visits and flags fakes, in your browser.
Official documentation
developers.google.com/search/docs/crawling-indexing/google-common-crawlers
Registry checked Nov 11, 2026. The address lists are downloaded every 6 hours.