Is this really Googlebot? How to verify a crawler's IP
A line in your log says Googlebot/2.1, but it's hammering your login page or downloading your whole product catalogue every hour. Is Google really doing that? Usually not, and you can prove it in a minute.
A user agent is a line of text the client writes about itself. Scrapers and attack tools copy Googlebot's because site owners are afraid to block it, and some firewalls wave it through. The address is different. For a completed web request it's the real network the connection came from, and Google controls the DNS names of its own crawler addresses. That's what you check.
The two-lookup test
Google's documented method has two steps, and you need both:
- Reverse lookup: ask DNS which name belongs to the address. A genuine crawler address has a name under
googlebot.com,google.comorgoogleusercontent.com. - Forward lookup: ask DNS for the address of that name. It must be the address you started with.
Step two is what makes this proof. Whoever controls an address can set its reverse name to anything, including crawl-1-2-3-4.googlebot.com. Only Google can make a googlebot.com name point back to an address.
On Linux or macOS:
host 66.249.66.1
host crawl-66-249-66-1.googlebot.comOn Windows, use nslookup 66.249.66.1 and then nslookup crawl-66-249-66-1.googlebot.com, or in PowerShell Resolve-DnsName 66.249.66.1.
The answer for this address: the reverse name is crawl-66-249-66-1.googlebot.com, and that name points back to 66.249.66.1. Both steps pass, so it's Google.
The quicker check: Google's published lists
Google also publishes its crawler addresses as JSON files, linked from the same documentation page. The one for Googlebot and the other common crawlers is common-crawlers.json. If an address falls inside one of its ranges, it's Google's crawler. This is easier to automate than DNS, and it's what iplover's tools use, refreshed daily.
A worked example: a real Googlebot and a fake one
Here are two lines from the same log. Both claim to be Googlebot.
66.249.66.1 - - [05/Nov/2026:09:15:40 +0000] "GET /2026/09/router-settings/ HTTP/1.1" 200 21004 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
3.5.140.1 - - [05/Nov/2026:09:20:55 +0000] "GET /.env HTTP/1.1" 404 548 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"Put both addresses into iplover's IP lookup, and the results are not close:
| Check | 66.249.66.1 | 3.5.140.1 |
|---|---|---|
| Verdict | Verified crawler This is Googlebot. | Cloud or hosting provider This address belongs to Amazon Web Services. |
| Matched list | Google common crawlers (Googlebot) | AWS IP address ranges |
| Reverse DNS | crawl-66-249-66-1.googlebot.com, confirmed forward | No name |
| Network owner | Google LLC (AS15169), ARIN | Amazon Data Services Northern Virginia (AS16509), ARIN |
The first is Google, by every test. The second is a server rented from Amazon that typed “Googlebot” into its user agent, and it was looking for an .env file full of passwords. Google's crawlers don't run on Amazon's cloud. That line is a scanner in disguise.
The lookup's advice for the real one is blunt: don't block it, because blocking it can remove your pages from Google's results.
If you see this, do this
| What you find | What it means | What to do |
|---|---|---|
| Both DNS steps pass, or the address is in Google's list | Real Googlebot | Leave it alone. To change how much it crawls, use robots.txt and Search Console. |
Reverse name is under googleusercontent.com | A Google service fetching for a user or a Google Cloud customer | It's Google's network, but not necessarily the search crawler. Judge it by what it requests. |
| Reverse name looks right, but the forward lookup gives a different address | A forged reverse name | Treat it as fake. |
| No reverse name, or a name unrelated to Google | Not Google | Treat it like any other unknown bot. Block it if it misbehaves. |
Fake “Googlebot” requests for .env, backups or wp-login.php | A scanner hiding behind a trusted name | Make sure those paths return 404, then block or rate-limit the address. |
The other big crawlers
The same rule applies to every crawler: check the address, never the user agent. Most large operators publish their crawler ranges in the same JSON format as Google. These are the lists iplover uses:
- Bing:
https://www.bing.com/toolbox/bingbot.json - Apple (Applebot):
https://search.developer.apple.com/applebot.json - DuckDuckGo:
https://duckduckgo.com/duckduckbot.json - OpenAI (GPTBot):
https://openai.com/gptbot.json
Our data sources page lists every crawler list we load and when it was last refreshed.
Blocking fakes without hurting the real thing
Never block by user agent alone in the direction of “allow anything that says Googlebot”. That's exactly the hole the fakes rely on. If your firewall or CDN has a “verified bots” category (Cloudflare does), use it: it checks the address for you. If you write your own rule, allow the published ranges and treat everything else that claims to be a crawler as an ordinary visitor.
And remember what a fake crawler is: a bot that doesn't want to be judged as a bot. Treat it like one. Whether it's worth blocking depends on what it does, not on the name it uses.
Sources
Official documents we used to check the facts in this guide.