AI & bot directory · SEO crawler
SemrushBot: what it is, how to verify it and how to block it
SemrushBot is run by Semrush. Crawls the web for Semrush's SEO and marketing tools. Semrush says it does not use fixed IP blocks.
Key facts
| Operator | Semrush |
|---|---|
| Category | SEO crawler |
| robots.txt name | SemrushBot |
| Follows robots.txt | OK: Yes |
| Can it be verified? | Couldn't check: No, user agent only |
| Official documentation | |
| Registry checked | Nov 11, 2026 |
User agent
Its name in robots.txt is SemrushBot.
Semrush doesn't publish a full user-agent string; look for SemrushBot in the user agent.
Can you verify SemrushBot?
No Semrush publishes no IP addresses or reverse-DNS names for SemrushBot, so any visitor can claim to be it. Treat its user agent as a claim.
Does SemrushBot follow robots.txt?
Yes Semrush says SemrushBot follows robots.txt.
How to block or allow SemrushBot
To ask it to stay away from the whole site, add this to robots.txt:
User-agent: SemrushBot
Disallow: /
To keep it out of one folder only:
User-agent: SemrushBot
Disallow: /private/
To allow it while asking every other bot to stay away:
User-agent: SemrushBot
Allow: /
User-agent: *
Disallow: /
robots.txt is a request, not a lock. To enforce a block, add a firewall or CDN rule that matches SemrushBot in the user agent. That stops the real SemrushBot; a bot that lies about its user agent can still get through.
To build one robots.txt for several bots at once, use the robots.txt generator.
How often does SemrushBot visit?
Semrush doesn't publish how often SemrushBot visits a site, and iplover doesn't see your traffic. Your own access log is the only reliable count: paste it into the log checker, which counts each crawler's visits and flags fakes, in your browser.
Official documentation
Registry checked Nov 11, 2026. The address lists are downloaded every 6 hours.