Check my connection

AI & bot directory · AI assistant fetch

meta-externalfetcher: what it is, how to verify it and how to block it

meta-externalfetcher is run by Meta. Fetches individual links at a user's request, for example for Meta's AI assistant features.

Key facts

Key facts
Operator
Meta
Category
AI assistant fetch
robots.txt name
meta-externalfetcher
Follows robots.txt
Warning: In part
Can it be verified?
Couldn't check: No, user agent only
Official documentation
Registry checked
Nov 11, 2026

User agent

Its name in robots.txt is meta-externalfetcher.

Meta documents this user-agent string:

meta-externalfetcher/1.1

Can you verify meta-externalfetcher?

No Meta publishes no IP addresses or reverse-DNS names for meta-externalfetcher, so any visitor can claim to be it. Treat its user agent as a claim.

Check an address from your logs

Does meta-externalfetcher follow robots.txt?

In part Only in part: Meta says meta-externalfetcher may not follow robots.txt in some cases, so robots.txt alone won't always keep it out.

“this crawler may bypass robots.txt rules”

How to block or allow meta-externalfetcher

To ask it to stay away from the whole site, add this to robots.txt:

User-agent: meta-externalfetcher
Disallow: /

To keep it out of one folder only:

User-agent: meta-externalfetcher
Disallow: /private/

To allow it while asking every other bot to stay away:

User-agent: meta-externalfetcher
Allow: /

User-agent: *
Disallow: /

robots.txt is a request, not a lock. To enforce a block, add a firewall or CDN rule that matches meta-externalfetcher in the user agent. That stops the real meta-externalfetcher; a bot that lies about its user agent can still get through.

To build one robots.txt for several bots at once, use the robots.txt generator.

How often does meta-externalfetcher visit?

Meta doesn't publish how often meta-externalfetcher visits a site, and iplover doesn't see your traffic. Your own access log is the only reliable count: paste it into the log checker, which counts each crawler's visits and flags fakes, in your browser.

Official documentation

developers.facebook.com/docs/sharing/webmasters/web-crawlers/

Registry checked Nov 11, 2026. The address lists are downloaded every 6 hours.