← Logbook

14 Jun 2026

Is Cloudflare hiding you from ChatGPT and Claude without you knowing?

You work hard on your content so ChatGPT or Perplexity will cite you… but there might be a bouncer at your door kicking out the AI crawlers before they read a single line. It's called Cloudflare, half the internet uses it, and we had one of its switches flipped on without knowing.

The idea is simple: if an AI can't get in and read your site, it won't cite you. Ever. No matter how good your content is.

Cloudflare's three bouncers

  • robots.txt: the sign on the door. A Disallow for GPTBot, ClaudeBot or PerplexityBot means «closed».
  • "Block AI bots": a one-click toggle in the panel (Security → Bots). Flip it and you block every AI at once. Handy to protect content, a disaster if you turned it on by mistake.
  • Bot Fight Mode: the bouncer with no judgement. It throws CAPTCHA-style challenges an AI bot can't solve → it leaves. And you never find out.

How to check in 2 minutes

  1. Open yourdomain.com/robots.txt in your browser. See bot names with Disallow? Bad sign.
  2. Cloudflare panel → Security → Bots. Check if "Block AI bots" is on.
  3. The free, definitive check: Google Search Console tells you whether Google can get in (a good proxy for the rest).

Why it matters

Models don't improvise: they cite what they've been able to read. If your site is closed to GPTBot or PerplexityBot, it's not that the AI ignores you… it's that it has never read you.

And it's not all-or-nothing: you can let in the ones that cite you (ChatGPT, Claude, Perplexity) and still block the ones that just scrape your data. It's about fine-tuning, not throwing the doors wide open.

Want us to check whether your site is open to the right AIs? Tell us about it, no fluff.