In one sentence
Google-Extended is a robots.txt token that lets publishers control whether Google may use their content to train and ground Gemini models. It is not a separate crawler.
What it means
Google fetches pages with its usual crawlers. Google-Extended is a control token that you address in robots.txt. Google states it does not affect inclusion in Google Search or how a site ranks.
Google documents that AI Overviews and AI Mode are part of Search, so their use of your content is governed by Search controls such as Googlebot access and snippet settings, not by Google-Extended. Read Google's current documentation before relying on this, since it has changed before.
What to do about it
Decide whether you are comfortable with Gemini using your content. If you disallow Google-Extended, keep Googlebot allowed so that you remain eligible for AI Overviews and AI Mode.
Keep a record of the decision and the date. If your visibility in Gemini changes later, you will want to know whether a robots.txt edit came first.
How Pineprompt measures it
Pineprompt tracks results separately in the Google AI Overview tracking, AI Mode tracking and Gemini visibility tracker.
Frequently asked
- What is Google-Extended?
- Google-Extended is a robots.txt token that lets publishers control whether Google may use their content to train and ground Gemini models. It is not a separate crawler.
- How does Pineprompt measure Google-Extended?
- Pineprompt tracks results separately in the Google AI Overview tracking, AI Mode tracking and Gemini visibility tracker.