AI crawler · Google · Ads checks and site-owner services

Google-CloudVertexBot: what it is, its user agent, and how to block it

Google-CloudVertexBot is Google's service crawler. Crawls a site at its owner's request to build a Vertex AI Agents data store. In our 13 October 2026 study, 160 of 2,332 live top sites with readable robots.txt rules (6.9%) disallow Google-CloudVertexBot. Part of the AI crawler list.

What Google-CloudVertexBot does

Operator: Google. Job: Ads checks and site-owner services. Obeys robots.txt.

“Crawling preferences addressed to the Google-CloudVertexBot user agent affect crawls requested by the site owners' for building Vertex AI Agents. It has no effect on Google Search or other products.”

Google's Google-CloudVertexBot documentation Published IP ranges (JSON)

Google-CloudVertexBot user agent

robots.txt token: Google-CloudVertexBot (matched case-insensitively).

Google doesn't publish a full user-agent string for Google-CloudVertexBot. Look for Google-CloudVertexBot in the User-Agent header.

robots.txt: allow or block Google-CloudVertexBot

Allow

User-agent: Google-CloudVertexBot
Allow: /

Block

User-agent: Google-CloudVertexBot
Disallow: /

Training only, not citations. Blocking it has no effect on AI answers or search citations.

Once a group names Google-CloudVertexBot, it ignores your User-agent: * rules, so copy any paths you still want blocked into it.

How many top sites block Google-CloudVertexBot

Disallow Google-CloudVertexBot in robots.txt

6.9%

160 of 2,332 sites

In our 13 October 2026 study, 160 of 2,332 live top sites with readable robots.txt rules (6.9%) disallow Google-CloudVertexBot.

Sample: Tranco top 5,000, tested once on 13 October 2026 from one Indian fixed-line connection. The robots.txt share counts live sites (home page 2xx) whose rules we could read, as on the AI visibility hub. Method and CSV in the study.

Check your own site

The AI crawler checker reads your robots.txt the way Google-CloudVertexBot does and tests what your firewall returns. To write the rules, use the AI robots.txt generator.

Check your page: free, about 20 seconds, no sign-in

What it checks

robots.txt for Google-CloudVertexBot: Finds the group Google-CloudVertexBot obeys under RFC 9309 and the rule that decides your page.
What your CDN lets through: Requests the page as OAI-SearchBot, PerplexityBot and Claude-SearchBot and compares the answers with a browser request.
Content without JavaScript: AI crawlers don't run JavaScript. The audit checks how much of the page is in the raw HTML.

Free to run. Pay only for the fixes you want.

Free

Free account

Your score without signing in; with a free account, every check's result and the full fix for the top 1 issues.

★ Starter

$9 / ₹499 once

Every issue with its exact fix for one site, PDF/Markdown/JSON exports, re-audit for 30 days. Nothing renews.

Pro

$29 / ₹1,499 / month

Full reports on every site, weekly alerts when AI access breaks, competitor compare.

Agencies: white-label widget, branded reports and lead capture on Agency.

Questions

What is Google-CloudVertexBot?+

Google-CloudVertexBot is Google's service crawler. Crawls a site at its owner's request to build a Vertex AI Agents data store.

What is Google-CloudVertexBot's user agent?+

Google publishes only the token Google-CloudVertexBot. Match on it in the User-Agent header.

How do I block Google-CloudVertexBot?+

Add "User-agent: Google-CloudVertexBot" followed by "Disallow: /" to your robots.txt. Obeys robots.txt, per Google.

Does blocking Google-CloudVertexBot affect AI search citations?+

Blocking it has no effect on AI answers or search citations.

How many websites block Google-CloudVertexBot?+