Check my connection

AI & bot directory · Search engine

meta-webindexer: what it is, how to verify it and how to block it

meta-webindexer is run by Meta. Indexes web pages for search in Meta AI.

Key facts

Key facts
Operator
Meta
Category
Search engine
robots.txt name
meta-webindexer
Follows robots.txt
OK: Yes
Can it be verified?
Couldn't check: No, user agent only
Official documentation
Registry checked
Nov 11, 2026

User agent

Its name in robots.txt is meta-webindexer.

Meta documents this user-agent string:

meta-webindexer/1.1

Can you verify meta-webindexer?

No Meta publishes no IP addresses or reverse-DNS names for meta-webindexer, so any visitor can claim to be it. Treat its user agent as a claim.

Check an address from your logs

Does meta-webindexer follow robots.txt?

Yes Meta says meta-webindexer follows robots.txt.

“In order to block these crawlers, add a disallow for the relevant crawler to robots.txt.”

How to block or allow meta-webindexer

To ask it to stay away from the whole site, add this to robots.txt:

User-agent: meta-webindexer
Disallow: /

To keep it out of one folder only:

User-agent: meta-webindexer
Disallow: /private/

To allow it while asking every other bot to stay away:

User-agent: meta-webindexer
Allow: /

User-agent: *
Disallow: /

robots.txt is a request, not a lock. To enforce a block, add a firewall or CDN rule that matches meta-webindexer in the user agent. That stops the real meta-webindexer; a bot that lies about its user agent can still get through.

To build one robots.txt for several bots at once, use the robots.txt generator.

How often does meta-webindexer visit?

Meta doesn't publish how often meta-webindexer visits a site, and iplover doesn't see your traffic. Your own access log is the only reliable count: paste it into the log checker, which counts each crawler's visits and flags fakes, in your browser.

Official documentation

developers.facebook.com/docs/sharing/webmasters/web-crawlers/

Registry checked Nov 11, 2026. The address lists are downloaded every 6 hours.