AI Search Visibility: What Data Your Website Needs for AI to Read It

Your business ranks on page one of Google. But when a potential customer asks ChatGPT “where should I stay for three days with my family” or “which restaurant should I choose for a business dinner,” your name is not in the answer.

This does not mean your content is bad. In most cases it means AI tools cannot read your data.

And the gap is significant. According to Ahrefs, only 37.9% of pages cited in Google’s AI answers also appeared in the top ten organic results. Ranking well on Google and being recommended by an AI are two separate achievements.

Here is how it works: the answers ChatGPT, Gemini and Perplexity give are built by automated programs called crawlers, which visit and read websites. If the crawlers cannot reach your site — or reach it and cannot understand what they see — your business simply does not exist to them.

Four things, in order of priority.

1. Can AI crawlers reach your website at all?

This is the first thing to check and the one almost nobody checks.

AI crawlers are separate programs from Googlebot: GPTBot and OAI-SearchBot from OpenAI, ClaudeBot from Anthropic, PerplexityBot and others. Each can be allowed or blocked independently, and they are blocked far more often by accident than by decision.

Two places to look. The first is your robots.txt file, where many sites still carry a Disallow for these user agents, a leftover from 2024, when the prevailing logic was “don’t let them take our content.” The setting stayed; the market moved on.

The second is more insidious: your firewall or CDN. Cloudflare’s Bot Fight Mode and the legacy “Block AI bots” toggle operate before robots.txt, at the network level. The crawler never reaches your server to read your permissions, it gets a 403 and leaves. The owner believes the site is open to AI search, while every attempt has been failing for months.

There is also a deadline. From 15 September 2026 Cloudflare is retiring that single toggle and replacing it with three independent categories: Search, Agent and Training. Multi-purpose crawlers — such as Googlebot, which crawls both for search and for model training — are judged by the strictest rule you have set. Which means: if someone once enabled training-bot blocking, you can end up blocking Googlebot too, and with it your ordinary rankings.

Open Security → Settings → AI bot policies and make a deliberate choice before the date. Then verify in your server logs: look for visits from those user agents and check their status codes.

2. Is your content in the raw HTML?

AI crawlers do not execute JavaScript.
Vercel’s analysis of hundreds of millions of fetches found no evidence of JavaScript execution by GPTBot, ClaudeBot or PerplexityBot. They make one HTTP request, read whatever HTML the server returns in that first response, and leave. There is no second pass. Googlebot, by contrast, does render — so “if Google can see it, everyone can” no longer holds.

In practice this hits exactly the information that matters:

• prices and availability delivered through an iframe or third-party widget
• services and facilities hidden in tabs and accordions that fire an AJAX request on click
• text written into the DOM by a plugin, inside sliders and animations

A 30-second check: open the page, right-click, View Page Source — not the DevTools Elements panel, which shows the DOM after JavaScript has run. Search for a sentence of your real content. If it is there, crawlers can read it. If you find only empty divs and script tags, they cannot.

The fix is not rebuilding your website. It is server-side rendering for your fixed data — descriptions, amenities, policies, location — with the widget handling live availability and pricing on top.

3. Structured data: what it does and does not do

Structured data (schema markup, usually in JSON-LD format) is a standardised way of declaring the basics of your business to machines: what type of business it is, where it is, what hours it keeps, what it offers, how it is rated.

Let us be honest: it is not a ranking lever. An Ahrefs study added JSON-LD to 1,885 pages with no meaningful uplift in AI citations, and Google states plainly that structured data is not required for its AI features.

What it does is remove ambiguity. Without it, a system reads words and infers. With it, the system has typed facts. And that certainty is often the difference between being named and being skipped.

Four types carry the weight: your core business type — Hotel, Restaurant, LocalBusiness — with address, coordinates, telephone, opening hours, priceRange and amenities; the specific type for your products or services on their own pages; AggregateRating for your score; and FAQPage on pages with real, visible Q&A.

One rule gets broken constantly: the data you declare must match what is visible on the page. Google stops using it when it detects mismatches, until they are fixed and trust holds for several consecutive days. A priceRange from last season is worse than no priceRange at all. Validate with Google’s Rich Results Test after every price or content change.

4. The specific details customers are searching for

AI queries are compound and constraint-heavy: quiet, suitable for children, ten minutes from the centre, under a certain price, with parking, cancellable.

A phrase like “a space where time slows down” matches none of those constraints. You need the details written clearly, in plain text:

• distances and times as numbers, not “a stone’s throw away”
• who you are for
• what is included and what costs extra
• policies: hours, cancellations, pets, children
• opening dates or hours

And one final point of equal weight: consistency beyond your own site. AI assistants also draw on Google’s Knowledge Graph — fed by Google Business Profile and schema markup — along with booking and review platforms. If your website says one thing and your profile elsewhere says another, the system has no way of knowing which is true. So it recommends the competitor whose details agree with each other.

Would you like us to check this for your business? Get in touch — it is part of our AI Marketing & LLM Visibility service.