Check my connection

AI & bot directory · SEO crawler

SemrushBot: what it is, how to verify it and how to block it

SemrushBot is run by Semrush. Crawls the web for Semrush's SEO and marketing tools. Semrush says it does not use fixed IP blocks.

Key facts

Key facts
Operator
Semrush
Category
SEO crawler
robots.txt name
SemrushBot
Follows robots.txt
OK: Yes
Can it be verified?
Couldn't check: No, user agent only
Official documentation
Registry checked
Nov 11, 2026

User agent

Its name in robots.txt is SemrushBot.

Semrush doesn't publish a full user-agent string; look for SemrushBot in the user agent.

Can you verify SemrushBot?

No Semrush publishes no IP addresses or reverse-DNS names for SemrushBot, so any visitor can claim to be it. Treat its user agent as a claim.

Check an address from your logs

Does SemrushBot follow robots.txt?

Yes Semrush says SemrushBot follows robots.txt.

How to block or allow SemrushBot

To ask it to stay away from the whole site, add this to robots.txt:

User-agent: SemrushBot
Disallow: /

To keep it out of one folder only:

User-agent: SemrushBot
Disallow: /private/

To allow it while asking every other bot to stay away:

User-agent: SemrushBot
Allow: /

User-agent: *
Disallow: /

robots.txt is a request, not a lock. To enforce a block, add a firewall or CDN rule that matches SemrushBot in the user agent. That stops the real SemrushBot; a bot that lies about its user agent can still get through.

To build one robots.txt for several bots at once, use the robots.txt generator.

How often does SemrushBot visit?

Semrush doesn't publish how often SemrushBot visits a site, and iplover doesn't see your traffic. Your own access log is the only reliable count: paste it into the log checker, which counts each crawler's visits and flags fakes, in your browser.

Official documentation

www.semrush.com/bot/

Registry checked Nov 11, 2026. The address lists are downloaded every 6 hours.