Check my connection

AI & bot directory · AI training

ClaudeBot: what it is, how to verify it and how to block it

ClaudeBot is run by Anthropic. Collects web content that may contribute to training Anthropic's Claude models. It also supports Crawl-delay.

Key facts

Key facts
Operator
Anthropic
Category
AI training
robots.txt name
ClaudeBot
Follows robots.txt
OK: Yes
Can it be verified?
OK: Yes, by IP ranges
Official documentation
Registry checked
Nov 11, 2026

User agent

Its name in robots.txt is ClaudeBot.

Anthropic doesn't publish a full user-agent string; look for ClaudeBot in the user agent.

Can you verify ClaudeBot?

Yes Anthropic publishes the IP addresses ClaudeBot uses. We download that list every 6 hours: our copy is from Nov 15, 2026 and has 14 entries; Anthropic dated it Oct 20, 2026.

Anthropic publishes one list for all its bots, so the address proves the request came from Anthropic, but not which of its bots made it.

Check an address from your logs

Does ClaudeBot follow robots.txt?

Yes Anthropic says ClaudeBot follows robots.txt.

“Anthropic's Bots respect "do not crawl" signals by honoring industry standard directives in robots.txt.”

Anthropic says ClaudeBot also honours Crawl-delay, which slows it down without blocking it.

How to block or allow ClaudeBot

To ask it to stay away from the whole site, add this to robots.txt:

User-agent: ClaudeBot
Disallow: /

To keep it out of one folder only:

User-agent: ClaudeBot
Disallow: /private/

To allow it while asking every other bot to stay away:

User-agent: ClaudeBot
Allow: /

User-agent: *
Disallow: /

robots.txt is a request, not a lock. To enforce a block, add a firewall or CDN rule. Because Anthropic publishes ClaudeBot's addresses, a rule can match them exactly, so a fake ClaudeBot can't slip through as the real one.

To build one robots.txt for several bots at once, use the robots.txt generator.

How often does ClaudeBot visit?

Anthropic doesn't publish how often ClaudeBot visits a site, and iplover doesn't see your traffic. Your own access log is the only reliable count: paste it into the log checker, which counts each crawler's visits and flags fakes, in your browser.

Official documentation

support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler

Registry checked Nov 11, 2026. The address lists are downloaded every 6 hours.