LLM SEO: how to get cited, not just ranked

LLM SEO is the work of getting your pages retrieved and quoted by systems that answer in prose instead of listing links — ChatGPT search, Perplexity, Google's AI Overviews, Copilot, Claude with search turned on. The name is new; most of the mechanism is not.

The thing worth understanding first is where the text in those answers comes from. With search enabled, these systems overwhelmingly do not recall your page from training weights. They run a query against a live index, pull back a handful of documents, cut them into passages, and hand a few of those passages to the model as context. The model writes the answer out of that context and attaches a link to whichever passage it leaned on.

That pipeline is why "optimizing for the model" is the wrong frame. There are three separate gates, and they fail independently:

1. Retrieval access. A named crawler has to be allowed to fetch the URL. This is binary and has nothing to do with quality.
2. Passage extraction. Your page gets chopped into chunks before a model sees it. If your chunks are shapeless, none of them scores well against the query.
3. Citation choice. Several passages say roughly the same thing. The one that gets named is the one that made attribution easy and worthwhile.

Getting into training data is a fourth path, but it is slow, unverifiable, and you have almost no control over it. Everything actionable lives in the three gates above.

Most of your existing work carries over — being in a search index, having pages that load, internal links, saying something nobody else says. Four things genuinely change.

The unit is the passage, not the page. SEO for AI search is judged one chunk at a time. A page ranking first with the answer buried in paragraph nine loses the citation to the page ranking seventh that put a self-contained answer under a heading. Position on the SERP is an input to retrieval, not the decision.

Crawler access is a second, separate gate. This is the genuinely new failure mode. Googlebot access does not imply OAI-SearchBot access. A site can be flawlessly indexed by Google, ranking well, and completely absent from ChatGPT search because of one line in robots.txt that someone added in 2023.

There is no single result page to check. The same question produces different citations in ChatGPT, Perplexity and AI Overviews — and different citations across two runs of the same prompt in the same product. There is no rank number. What you can measure is presence across repeated runs, which means you need a sample, not a screenshot.

Clicks stop being the only outcome. A cited answer frequently ends the session. That is uncomfortable, and it is also why brand-level mentions inside answers matter now in a way impressions never did.

Start here, always. It is the cheapest thing to check, it is binary, and it is the only failure that makes every other improvement worthless. Fixing it is one file.

The bots that matter fall into three tiers that behave completely differently. Treating them as one bucket labeled "AI bots" is what causes most of the damage:

The single most expensive confusion in this table is that GPTBot is not OAI-SearchBot. GPTBot collects training text; OAI-SearchBot builds the index ChatGPT searches — OpenAI’s own bot documentation draws exactly this line. Plenty of sites disallowed GPTBot on principle, which is fine, and then copied the same block onto every OpenAI-looking user agent, which quietly removed them from ChatGPT search. The same pair exists at Google — Google-Extended is the training opt-out, and blocking it does not remove you from AI Overviews, because those are served off the ordinary Google index that Googlebot builds — Google’s AI features guide says so in as many words.

A robots.txt that says what you mean

# Groups do NOT stack. A crawler obeys only the most specific
# group matching its name, so every named group has to repeat
# whatever site-wide rules you care about.

User-agent: OAI-SearchBot
User-agent: Claude-SearchBot
User-agent: PerplexityBot
User-agent: DuckAssistBot
Disallow: /admin/
Allow: /

User-agent: ChatGPT-User
User-agent: Claude-User
User-agent: Perplexity-User
Disallow: /admin/
Allow: /

# Training corpora — blocking here costs you nothing in AI search
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: Google-Extended
User-agent: CCBot
Disallow: /

User-agent: *
Disallow: /admin/

Sitemap: https://example.com/sitemap.xml

The comment at the top is the part people get wrong. Under RFC 9309 a crawler picks exactly one group — the one whose user-agent token is the longest match on its name — and ignores every other group in the file, including User-agent: *. So the moment you add a named group for PerplexityBot, all your wildcard rules stop applying to it. Every named group has to restate the rules you actually care about.

Three more things that silently block AI crawlers:

• Rules written years ago for a different bot. A Disallow: /*? added to keep faceted URLs out of Google blocks every AI crawler from every URL carrying a query string. Nobody intended that; it is still the most common cause of a zero on crawler access.
• Your CDN or WAF, not your robots.txt.Bot-manager rules that challenge or drop requests by user agent or ASN are invisible in the file. robots.txt can say Allow while your edge returns 403. Check server logs for the bot names, or fetch a page with the crawler's user agent and read the status code.
• Host and scheme scope. robots.txt applies to one host on one protocol. Rules on example.com say nothing about blog.example.com, which is exactly where the content you want cited usually lives.

On llms.txt: it is a proposed convention for publishing a curated map of your site for language models. No major AI search product has documented using it for retrieval. It costs an hour to add and it is not a substitute for anything on this page — treat it as a cheap bet, not a strategy.

Most AI search optimization advice stops at "write good content". The mechanical layer comes first, because it decides what a retriever is even able to hand the model.

Headings are chunk boundaries. Retrieval splits your page into passages, and headings are the most reliable split points available. A 2,000-word page with two headings produces a few enormous chunks, each covering several topics, each diluted against any specific query. The same 2,000 words under eight question-shaped H2s produce eight tight passages that can win eight different questions.

Answer in the first two sentences under each heading.A chunk is retrieved alone, stripped of everything above it. Anything that opens with "as we saw above" or a pronoun pointing at the previous section arrives unreadable. Lead with the direct answer, then expand — the expansion is for humans, the first two sentences are what gets quoted.

Name the question in the heading. "Pricing" is a label. "How much does a sitemap monitor cost?" matches an actual query. Headings that are puns or one-word nouns cost you the match at the cheapest possible place to win it.

Server-render the content. These crawlers fetch HTML; they generally do not execute JavaScript. If your text only exists after hydration, they receive an empty shell and score you on it. Static generation or SSR for anything you want cited — and if you are not sure which your framework is doing on a given route, fetch the URL with curl and read what comes back.

Mechanical basics that parsers rely on. One <html lang>, one charset declared early in <head>, a viewport meta, and exactly one H1. Two H1s break the outline a summarizer uses to separate "this is the topic" from "this is a sidebar"; a missing lang attribute gets pages misclassified for the language query they should have won.

Structured data, for the boring reason. JSON-LD is not magic input to the model — it is the cheapest unambiguous way to state who published this, when, and what kind of thing it is, so the entity-resolution step does not have to infer it from prose. Organization and WebSite site-wide; Article with datePublished, dateModified and a named author on posts; Product with offers on listings; FAQPage only where a real question-and-answer section exists on the page. Put the date in visible text too, not just in the markup.

Formats that survive chunking. Short paragraphs, definition-style sentences, tables with real headers, and lists come through retrieval intact. Text inside images, numbers only present in a chart, and anything behind a tab that renders on click do not.

Answer engine optimization is the half nobody can automate: given several passages that all answer the question, why would a model name yours? Attribution is a choice, and it gets made on how specific and how attributable your sentence is.

Publish something only you can say. This is the whole game. A model can paraphrase generic advice away without crediting anybody — it appears on forty pages, so it belongs to nobody. It cannot paraphrase away a number that exists in one place. Your own aggregate usage data, a benchmark you ran, a teardown of fifty competitors, results from your own customers: these force the citation, because naming the source is the only way to state the fact.

Specific beats adjectival, sentence by sentence. "Fast and reliable" is unquotable — there is nothing in it to attribute. "Diffs a 12,000-URL sitemap in about 400ms" is a sentence a model can lift whole. Go through a page you care about and count how many sentences contain a number, a date, a name, or a threshold. That count is roughly your citability.

Date things and stand behind them. Visible publish and update dates, a named author with a reason to be credible on the topic, and a source link for every claim you did not generate yourself. Systems that have to decide between two contradictory passages lean on exactly these signals.

Keep your entity consistent off-site. A model assembles a picture of your company from your site, your docs, directory listings, review sites and whoever wrote about you. When those contradict each other, the outlier gets discarded — and the outlier is usually your own newest positioning, because it is the least corroborated. Matching descriptions and consistent claims across those surfaces do more than another blog post.

Get mentioned on the pages that already get cited. Roundups, comparisons and "best X for Y" lists are disproportionately what answers retrieve for commercial questions. Being one of the named options on those pages puts you in the retrieved context without owning the page at all.

Trust plumbing, because it is cheap. A self-referencing canonical means a crawler landing on a parameterized or duplicated copy of your page has a signal pointing back at the real one, rather than citing the copy. Open Graph and Twitter Card tags are what render when an assistant surfaces a card for your link; without them it shows whatever it can scrape, which usually looks broken.

Two things not to do: do not bolt FAQPage markup onto pages with no visible Q&A, and do not plant instructions to the model in hidden text. The first is a structured-data violation, the second gets stripped, and both trade a real asset for a trick with a half-life.

Work in that order — access, then structure, then citability — because each one makes the next measurable. Three instruments cover it:

• Access: grep your server logs for the bot names above. Arrivals are ground truth; robots.txt is only intent. The checker below reads the file and evaluates it per bot, per path, which catches the intent half in seconds.
• Structure: fetch the URL without a browser and confirm the text is in the HTML, then read the heading outline on its own and ask whether each section would still make sense pulled out alone.
• Citations: write down 15–20 questions your buyers actually ask, run them monthly across ChatGPT, Perplexity and Google, and log which domains get cited. Run each prompt more than once — output varies between runs, so a single check tells you nothing. Search Console still matters here: AI Overview impressions land in your ordinary performance data.

Start with the first one now. Paste a URL and we fetch it the way a crawler does, evaluate your robots.txt against each bot by name, and score what a retriever would find on the page:

1. Open robots.txt and check the search and assistant tiers by name, not by wildcard. Repeat your real rules inside every named group.
2. Make sure the text you want quoted is in the HTML before JavaScript runs.
3. Re-cut your best page into question-shaped H2s, each answered in its first two sentences.
4. Publish one number that exists nowhere else, and put it in a sentence that names you.

None of that is a trick, and none of it expires when the next model ships — which is most of the argument for doing it in this order.

Related: the AI visibility checker scores a single URL on these five dimensions · competitor monitoring tells you the day a rival ships the pages that will be cited instead of yours.