Cloudflare Blocking Googlebot: Act Before Sept 15

Two seemingly unrelated problems landed on marketers' desks in the same week: Cloudflare will start automatically blocking Googlebot as an AI crawler from September 15, 2026, and Google has simultaneously launched the first dedicated AI impressions report in Search Console. Both change how you need to think about technical site accessibility and AI visibility measurement.

Key takeaways

• From September 15, 2026, Cloudflare classifies Googlebot and Bingbot as mixed-usage bots, meaning an AI training block can return 403 errors to the Google crawler.
• The new GSC report tracks AI impressions across AI Overviews, AI Mode, and Discover, but currently has no click, CTR, or query data.
• AI impressions and organic clicks are incompatible metrics. Never combine them into a single KPI.
• AI search visibility depends heavily on off-site signals: mentions, reviews, awards, YouTube, Reddit.
• Consistent brand messaging across authoritative sources is the new NAP for AI recommendations.

Cloudflare and Googlebot: a concrete risk with a concrete deadline

Cloudflare introduced the category of mixed-usage bots, crawlers that both index the web for search engines and collect data for AI model training. Googlebot and Bingbot now fall into this category. If you have AI training blocking or Bot Fight Mode enabled in your dashboard, those settings may start returning HTTP 403 responses to the legitimate Google crawler from September 15, including access to your sitemap. A Reddit user confirmed that Googlebot and Bingbot appear as automatically blocked in the AI Crawlers section. Google's John Mueller asked for details to investigate (Search Engine Journal).

What to do now, in this order:

1. Log into your Cloudflare dashboard and open the AI Crawlers section. Check whether Googlebot or Bingbot are flagged as blocked.
2. Run a fetch-as-Googlebot test in Google Search Console, especially for your sitemap.
3. If you need to preserve full indexation, opt out of the new defaults before September 15.
4. If you want to block specific AI scrapers (OpenAI, Anthropic, Common Crawl), use targeted robots.txt rules for those specific user-agents rather than a broad block that puts your SEO at risk.

Monitor your server logs for a spike in HTTP 403 responses to the Googlebot user-agent after September 15. If they start appearing, you have a clear diagnostic signal.

How to read the new GSC AI impressions report

Google launched a dedicated Search Console report for AI Overviews, AI Mode, and Discover on June 3, 2026. Data starts from May 18, 2026, with no historical backfill (Neil Patel). The report currently shows impressions only. No clicks, no CTR, no queries.

Here is the critical detail most people will miss: impression counting logic differs by context. In AI Overviews, a link only counts as an impression after the user scrolls or expands the panel. In AI Mode, each follow-up question generates new impressions. And the property-level total does not equal the sum of page-level data (Search Engine Journal). This is why mixing AI impressions with organic clicks in a single report will give you meaningless numbers.

A practical analysis workflow:

• Export AI-visible URLs over a longer time window.
• Pull organic data from the standard Performance report for the same URLs.
• Segment pages: high AI + high organic (priority content), high AI + low organic (audit the conversion path, add CTAs), low AI + high organic (run a vulnerability audit before AI Overviews expand further).
• For pages with low AI visibility, check whether key information is in readable HTML or buried in JavaScript, tabs, or images.

The opt-out toggle exists, but use it only with a specific reason. Without a strategic justification (paywalled content, licensing model), you are giving up a growing visibility channel for nothing.

Why AI recommends your competitor and what to do about it

Cloudflare and the GSC report are technical problems. But there is a third layer most teams ignore: what other websites say about you.

According to Kris Jones of LSEO, 70-80% of AI search optimization is still classic SEO practice. The remaining 20-30% comes from third-party signals: forum mentions, reviews, awards, citations on authoritative sites (Search Engine Journal). AI systems do not produce a list of results, they produce a recommendation. And recommendations are drawn from the entire web, not just your well-optimized pages.

One concrete example: schools lost conversions because a Google AI Overview pulled a 10-year-old price from Reddit that was 30% lower than the actual current price. A single old thread can damage conversions more than poor technical SEO.

What this means in practice:

• Run an AI brand audit: ask ChatGPT, Google AI, and other assistants for recommendations in your category, then verify what they say about your brand.
• Ensure product information, pricing, and positioning are consistent across authoritative sources.
• Invest in presence on Reddit and YouTube. AI answers cite Reddit roughly 40 times more often than classic organic results, and YouTube is the second most-cited source in AI Overviews (WordStream).
• Pursue relevant industry awards and rankings. LLMs prefer these sources when generating recommendations.

Connecting all three into one reporting framework

The three things that changed this week are directly related. If Cloudflare blocks Googlebot, you lose indexation, and without indexation you have no AI impressions. If you have AI impressions but misread their logic, you make bad content decisions. And if your content is solid but you ignore third-party signals, AI simply will not recommend you.

Charlie helps marketing teams and agencies track AI visibility as a separate channel alongside organic and paid. If you are just now building a reporting infrastructure for AI impressions, the time is now: baseline data in GSC is available from May 18, 2026, and waiting means losing your historical comparison point.

Practical priority for this week: check your Cloudflare settings, set up a separate dashboard for AI impressions, and run an AI brand audit. In that order.