How AI assistants find, read and cite your site

Ask ChatGPT to recommend a plumber in your city and you get three names and a sentence about each. Ask Perplexity and you get a different three, with citations underneath. Somewhere in there is a decision about whose website was worth reading, and most businesses have never found out whether they were in the running.

The advice written about this is nearly all editorial: write better content, get listed in directories, earn mentions. That advice is correct, and it is the second half of the story. Before any of it matters, something has to reach your page, parse what it finds, and decide the result is reliable enough to repeat. Fetch, read, trust. Three mechanisms, each with a failure mode you can check this afternoon.

Nobody can promise you a mention. OpenAI says so in its own documentation: placement is not guaranteed, and the factors are not published. What follows is not a route to being named, but a list of the things that stop you being named for reasons unrelated to how good your business is.

Two ways an assistant knows about you

There are two completely different paths, and confusing them is why so much GEO advice lands wrong.

The first is training. A crawler fetched your pages at some point, that content went into a dataset, and a model was trained on it months later. Answering from training alone, the model recalls a compressed impression of the web as it stood before the cutoff. That impression is stale by definition: it cannot know about the page you published yesterday, the price you changed last week, or the service you dropped in spring.

The second is live retrieval. The assistant decides it needs current information, issues searches, fetches a handful of pages right then, and writes an answer from what came back. This is the path that produces citations, and the only one that can reflect anything recent about you.

Almost everything worth doing sits on that second path. If your business moved, rebranded or repriced, no amount of waiting fixes the training-data version of you. A live fetch of a current page does. So optimise for being fetchable and readable right now, and treat the training copy as something you influence slowly, by existing accurately in public.

The two paths also use different crawlers, which is where the robots.txt decision gets interesting.

Fetch: the part nobody writes about

A live fetch happens while a person sits watching a cursor blink. Whatever time your server takes to answer is added to how long that person waits. Product teams therefore cap the wait short, and when a page does not answer in time the crawler moves on to the next result rather than retrying.

That changes the hosting conversation. Googlebot is patient: it will wait, retry, re-queue and come back tomorrow, because nobody is watching it work. An AI crawler on a live fetch has no such luxury. Published measurements from GEO and hosting analysts put the typical abandon window at a few seconds, with sub-200 ms server response as the target. No AI vendor publishes a timeout figure, so treat the numbers as informed estimates rather than specification. The direction is not in doubt.

What this means in practice:

Server response time is the first gate. Not your Lighthouse score, not your Core Web Vitals, not how fast the page feels once it has started rendering. The time between request and first byte. A page taking two seconds to begin answering has spent most of its budget before a single word has been read.

Redirect chains burn the budget. Every hop is another round trip. A four-hop HTTP to HTTPS to www to trailing-slash chain spends four round trips arriving at a page you had seconds to deliver. Secondary analysis suggests AI crawlers tolerate fewer hops than Googlebot, reason enough to collapse the chain regardless.

Bot challenges are a silent kill. This catches people hardest. A CAPTCHA page, a JavaScript interstitial or an aggressive bot-management rule returns a perfectly valid HTTP 200 containing no content. Nothing in your logs looks broken. The crawler finds nothing citable and leaves, and you never find out.

JavaScript-only rendering is a coin flip. If the initial HTML is an empty shell and content arrives client-side, you are betting that whatever fetched your page runs JavaScript. Some do. Many do not, particularly on a live fetch where the compute budget is short. A site can look flawless to a visitor and be blank to the thing deciding whether to cite you.

This is where hosting stops being background and becomes the mechanism. All Flashcloud shared plans run on NVMe storage, LiteSpeed web servers, Cloudflare CDN and cPanel, with a web application firewall (Imunify360), daily malware scanning and DDoS protection included. What matters here is response time under a short deadline, not a feature list. Daren Low at Bitcatcha measured Flashcloud's server response at 2.4 ms and graded it A+, published at the source rather than self-reported. A server answering in single-digit milliseconds spends effectively none of a crawler's budget before your content arrives.

To find where you stand, a website loading speed test explains what your numbers mean, and running a speed test that means something covers measuring it without fooling yourself. Whether your host is the limiting factor gets its own piece.

Read: HTML that survives being turned into text

Assume the crawler got through. What it does next is strip your page down to text and structure. Everything that carried meaning visually and not textually is gone at that point.

Headings are the skeleton. An H2 that describes the section under it gives the model a handle on what that text is about. An H2 chosen because it looked right at that size does not. Keep the hierarchy real: one H1, H2s for sections, H3s for subsections, no skipping levels for styling reasons.

Dates need to be in the page, not implied by it. "Updated last month" is useless. A machine-readable published and modified date, visible in the text and in your structured data, lets an assistant judge whether your page is current enough to rely on. Where recency matters, which is most commercial content, this is load-bearing.

Facts belong in prose and in lists, not in images. A pricing table rendered as a PNG is invisible, and so is a comparison drawn as a graphic. If a number matters, it needs to exist as text on the page. Crawlers also need to find the pages, which is what your XML sitemap is for.

Structured data earns a careful sentence. Supported by an SSRN analysis of citation behaviour, the honest position is that generic schema bolted onto a page does not raise citation rates, while populated Product, Review and LocalBusiness schema mirroring visible content does appear to help the engines that depend on live retrieval. Google states explicitly that no special markup is required to appear in its AI features. So treat schema as making your existing facts unambiguous rather than as a lever: mirror what is visible, never invent a claim that lives only in JSON-LD, and populate the fields. Which types are worth the effort deserves its own piece.

The one structural thing genuinely worth doing is writing self-contained statements. A sentence that answers a question without needing the paragraph above it can be lifted into an answer. A sentence built on "this" and "it" and "as mentioned" cannot travel. Pronouns do not get cited.

Trust: entities, evidence and consistency

Fetching and parsing are solvable in a weekend. Trust is the slow part, and where the editorial advice everyone else gives is right.

Name real things: your business name, the city you operate in, the people who work there, the exact services, the actual prices. Models work with entities, and an entity needs a name to attach to. A page saying "we're a leading provider of solutions for growing businesses" has named nothing and will be cited for nothing.

Carry your sources. A number with an origin attached beats a number asserted. This applies to us too, which is why the Bitcatcha figure above links to Bitcatcha rather than to a badge on our own homepage. Third-party evidence you cannot edit is the strongest kind.

Be consistent across the web: the same business name, address, phone number and description, in every directory you appear in. Inconsistency gives an assistant competing versions of you to choose between, and you do not get to pick which one it repeats.

Accept that your own site is one source among many. As our guide to checking what AI assistants say about your business sets out, citation patterns differ sharply: ChatGPT tends to cite few sources and leans on companies' own websites, while Perplexity cites many and leans on third-party comparisons and directories. Missing from ChatGPT's answers, the work is usually on your own pages. Missing from Perplexity's, it is somewhere else entirely, and no on-site tuning substitutes for it.

The bots: two separate decisions

This is the part worth getting right, because the common advice to "block the AI bots" quietly costs people traffic.

As of October 2026, the user agents the three main vendors document are these. OpenAI publishes GPTBot for content that may train its foundation models, OAI-SearchBot for surfacing websites in ChatGPT's search features, and ChatGPT-User for fetches a person triggered inside ChatGPT. Anthropic publishes ClaudeBot for content that could contribute to model training, Claude-SearchBot for improving search result quality, and Claude-User for pages Claude fetches when someone asks it a question. Perplexity publishes PerplexityBot for surfacing and linking websites in its results, and Perplexity-User for user-initiated fetches, which its documentation notes generally ignores robots.txt because a person asked for it.

Google-Extended is a different kind of thing, worth not mixing up. It is a robots.txt token, not a crawler: no request ever arrives from it and no log will show one. It controls whether content Google already crawled may train Gemini models and ground answers in Gemini apps and Vertex AI. Google states it does not affect inclusion or ranking in Search.

So the decision splits cleanly in two:

Training access, governed by GPTBot, ClaudeBot, Google-Extended and Applebot-Extended. Blocking these keeps your content out of future model training and does not remove you from any answer using live retrieval. If your objection is to your work being used as training material, this is the lever, and pulling it costs nothing in citations.

Citation access, governed by OAI-SearchBot, Claude-SearchBot and PerplexityBot. These fetch a page in order to cite it in an answer that may send a real person to your site. Blocking them removes you from the answers you presumably want to appear in. OpenAI documents a wrinkle: if it gets a disallowed page's URL from a third-party search provider, it may still show the link and title.

Flashcloud allows all of them, training crawlers included, and publishes the rules in robots.txt with a Content-Signal line stating ai-train=yes, search=yes, ai-input=yes. That is our call for a hosting company whose content exists to be found. Yours may differ, and a site whose value is its archive has good reason to decide otherwise. The mechanics are in blocking or welcoming bots and AI crawlers, with the finer-grained controls in controlling AI crawlers, AI assistants and model training on your site.

One warning: not every crawler respects the file. Reported figures put the share of AI bot traffic ignoring robots.txt in the low teens, and crawler volume has become a real resource question on small plans. Enforcement, rather than a request, happens at the firewall or CDN layer, or by blocking bots by user agent in cPanel. There is more to say about bandwidth specifically, and Cloudflare's shift toward pay-per-crawl has changed the industry context as well.

What we publish, and why it is a weak lever

Flashcloud publishes two files written for machines. Our llms.txt is a structured navigation overview: what we are, what each plan costs, which assistants the connector supports, where the free tools live. The plain-text AI dossier goes further, with pricing tables, third-party review scores with their source links, competitive positioning and explicit answer guidance.

We publish them because they are useful documentation: our prices, inclusions and mechanics in one place, in plain text, with no layout to parse. When we change a price, we change it there too.

We do not publish them expecting a visibility effect, because the evidence says there is not one. SE Ranking studied nearly 300,000 domains and found 10.13% had an llms.txt file, with no correlation between having it and being cited by AI systems. Their machine-learning model's predictions got more accurate when they removed llms.txt as a factor, suggesting the file added noise rather than signal. Ahrefs' separate look at 137,210 domains found the overwhelming majority of published llms.txt files got no traffic at all in the month studied.

That is worth stating plainly, because the hype has been considerable. Publishing llms.txt costs nothing and documents your site. It is not a ranking move, and anyone selling it as one is ahead of the data. llms.txt: helping AI assistants understand your website covers the format, and a fuller reconciliation of the competing studies is coming.

The point generalises. The things with evidence behind them are boring: be fast, fetchable, parseable, accurate, consistent, cited elsewhere. The things marketed hardest have the least support.

Measuring it

Two checks, neither needing a subscription.

Ask the assistants directly. Open ChatGPT, Perplexity, Claude and both of Google's AI surfaces, ask the questions a customer would ask in the words a customer would use, and write down whether you appear, whether the facts are right, and which sources got cited. The sources are the useful part. Our twenty-minute version is linked above, and the wider shift from ranking to being cited is the subject of our earlier piece on how AI is reshaping SEO, GEO and organic web traffic. Each assistant shows you differently, which deserves its own comparison.

Then check your server logs, which almost nobody does. Search your raw access logs for the user agent names above. If GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot and PerplexityBot appear, you are being reached. If none appear, something is stopping them before any content question arises: usually a robots.txt rule, a bot-protection service, or a site that renders entirely client-side. Searching inside files with grep covers finding them in large logs.

Watch for referral traffic too. ChatGPT tags the links it sends, so visits arrive in analytics as coming from chatgpt.com. Many site owners have never looked, and are surprised to find some already there.

A 30-minute checklist

Work down this in order. The first five break most often.

1. Measure server response time, not your page score. Under 200 ms is the target. Over a second is the finding.
2. Count your redirect hops to your canonical URL. Collapse anything over two.
3. Disable JavaScript and load three key pages. Whatever text remains is roughly what a non-rendering crawler sees. Blank pages are your biggest problem.
4. Fetch your own pages the way a bot would and confirm you get content, not a challenge page.
5. Read your robots.txt as two decisions. Settle training and citation access separately, then make the file say what you decided, per robots.txt best practices.
6. Search your access logs for the five crawler names. Note which appear.
7. Check every page carries a visible, machine-readable published and modified date.
8. Verify your headings describe their sections and skip no levels.
9. Confirm no price or key fact exists only inside an image.
10. Validate your structured data, populate the fields, mirror only what is visible.
11. Ask four assistants three customer questions, and record who got cited.
12. Publish an llms.txt as cheap documentation, with no expectations.

Items 2 through 12 are yours. Item 1 is usually your host's, and it gates everything else. If your server takes two seconds to answer, a crawler on a short deadline never gets as far as your headings, your schema or your entities. The editorial work is real, and it only pays off on a page delivered in time to be read.

Item 1 is also the only item on the list we sell. Flashcloud is not selling AI visibility, because nobody can sell that. It sells the server that answers fast enough for the other eleven items to count, on the stack behind that 2.4 ms figure, with the domain, SSL, mailboxes and daily backups included.