Check my connection

AI & bot directory · AI training

Google-Extended: what it is, how to verify it and how to block it

Google-Extended is run by Google. A robots.txt control, not a crawler: it decides whether pages Google already crawls may be used to train and ground Gemini models. It does not affect Google Search.

Key facts

Key facts
Operator
Google
Category
AI training
robots.txt name
Google-Extended
Follows robots.txt
Note: A robots.txt control
Can it be verified?
Note: Not a crawler
Official documentation
Registry checked
Nov 11, 2026

User agent

Its name in robots.txt is Google-Extended.

Google-Extended has no user agent of its own. Googlebot does the crawling; this name only appears in robots.txt, where it decides what the crawled pages may be used for.

Can you verify Google-Extended?

Not needed Google-Extended never visits your site itself, so there is nothing to verify.

Does Google-Extended follow robots.txt?

A robots.txt control It exists only as a robots.txt rule, so robots.txt is exactly where you control it.

“Google-Extended doesn't have a separate HTTP request user agent string.”

How to block or allow Google-Extended

To ask it to stay away from the whole site, add this to robots.txt:

User-agent: Google-Extended
Disallow: /

To keep it out of one folder only:

User-agent: Google-Extended
Disallow: /private/

To allow it while asking every other bot to stay away:

User-agent: Google-Extended
Allow: /

User-agent: *
Disallow: /

Blocking Google-Extended only changes how Google may use your pages; Googlebot still crawls them. To stop the crawling itself, block Googlebot, which also removes you from its search results.

To build one robots.txt for several bots at once, use the robots.txt generator.

How often does Google-Extended visit?

Google doesn't publish how often Google-Extended visits a site, and iplover doesn't see your traffic. Your own access log is the only reliable count: paste it into the log checker, which counts each crawler's visits and flags fakes, in your browser.

Official documentation

developers.google.com/search/docs/crawling-indexing/google-common-crawlers

Registry checked Nov 11, 2026. The address lists are downloaded every 6 hours.