Guide · AI search
How to get your business cited by ChatGPT, Perplexity, Gemini and Google AI
A practical, sourced guide: which AI crawlers to allow, what Google officially says you need (and don’t), what the research says gets content cited, and where off-site mentions fit.
Quick answer
Allow the AI search crawlers (OAI-SearchBot, PerplexityBot, Claude-SearchBot and Googlebot) in robots.txt, keep pages indexable, and write clear, factual content. Google says there are no special requirements, no special schema and no AI files to add for AI Overviews, and that llms.txt is ignored by Google Search. Research on generative engines found that adding quotations, statistics and cited sources increased how often content was cited, while keyword stuffing made it worse.
The AI crawlers and what blocking them does
| Crawler | What it does | Source |
|---|---|---|
| OAI-SearchBot (OpenAI) | Builds ChatGPT search results. Sites that opt out won’t appear in ChatGPT search answers, though they can still show as links. | OpenAI |
| GPTBot (OpenAI) | Training crawler. Blocking it keeps content out of model training; it’s independent of OAI-SearchBot. | OpenAI |
| ChatGPT-User (OpenAI) | Fetches pages when a user asks ChatGPT to. Because users trigger it, robots.txt rules may not apply. | OpenAI |
| Claude-SearchBot / Claude-User (Anthropic) | Index pages for Claude’s search and fetch pages on a user’s request. Blocking either may reduce visibility in Claude’s answers. | Anthropic |
| ClaudeBot (Anthropic) | Training crawler. Blocking it signals content should be excluded from training; it doesn’t block the two bots above. | Anthropic |
| PerplexityBot | Surfaces and links sites in Perplexity results. Not used to train foundation models. | Perplexity |
| Perplexity-User | Fetches pages for a user’s request and generally ignores robots.txt. | Perplexity |
| Google-Extended | Controls use for Gemini training and grounding only. Doesn’t affect Google Search inclusion or AI Overviews. |
Crawler names and behaviour are taken from each company’s own documentation (checked 10 November 2026) and can change. Check the current docs before editing robots.txt.
What Google says you need for AI Overviews and AI Mode
Google’s Search Central documentation is unusually direct: “There are no additional requirements to appear in AI Overviews or AI Mode, nor other special optimizations necessary.” Your page needs to be indexed and eligible to show with a snippet, and that’s all.
- No special schema. Google says there’s no special schema.org markup needed for AI features. Structured data is still worth adding for rich results, and it must match what’s visible on the page.
- No new files. You don’t need to create “AI text files” or other machine-readable files to appear.
- Controls are the usual ones. AI Overviews follow Googlebot plus snippet controls such as
nosnippet,max-snippetandnoindex. Blocking Google-Extended does not remove you from AI Overviews. - SEO still applies. Google’s AI optimisation guide says the best practices for SEO continue to be relevant.
In short: for Google, ordinary good SEO is the AI strategy. You can track visibility in Search Console’s generative AI performance report.
What about llms.txt?
llms.txt is a proposal for a Markdown file at your site’s root that summarises your content for AI tools. It’s harmless, but don’t expect it to move rankings: Google’s own guide says llms.txt files “will neither harm nor help your site’s visibility or rankings in Google Search, as Google Search ignores them.” We found no statement from OpenAI or Anthropic that their search systems use it either. Chrome’s Lighthouse has an experimental audit that checks for it, but that relates to browser agents, not search.
Our own site publishes one because it costs nothing, but we treat it as housekeeping, not a growth lever.
What the research says gets content cited
The most-cited academic study is “GEO: Generative Engine Optimization” (Princeton and IIT Delhi researchers, KDD 2024). Testing on a benchmark of queries, it measured how much of an AI answer quoted a source:
- Adding quotations from relevant sources: about +41% visibility.
- Adding statistics: about +31%.
- Citing sources: about +27%.
- Improving fluency: about +28%.
- Keyword stuffing: about −8%, worse than doing nothing.
The study also found smaller, lower-ranked sites benefited most: for the fifth-ranked source, adding citations raised visibility by 115%. Two caveats: the tests used 2023-era engines, and results varied by subject. It’s evidence of what helps, not a guarantee.
What this means in practice: state facts precisely, include real numbers with named sources, quote experts and primary documents, and write clearly. That’s why every guide on this site lists its sources and puts a short direct answer at the top.
How ChatGPT and Copilot find you: Bing matters
OpenAI has said it uses a set of services for ChatGPT search and that “Bing is an important one” (reported from an OpenAI engineering AMA, November 2024). Bing’s webmaster guidelines now say Bing and Copilot search rely on the same core crawling, indexing and ranking foundation (reported by Search Engine Journal, February 2026). In February 2026 Bing also launched an AI Performance report in Bing Webmaster Tools showing how often your pages are cited in Copilot and Bing’s AI summaries.
- Verify your site in Bing Webmaster Tools and submit your sitemap.
- Use IndexNow so Bing and other participating engines hear about new and updated pages quickly. Google doesn’t use IndexNow.
- For local businesses, keep Bing Places details current.
Do mentions on other sites help?
Google’s guidance says “seeking inauthentic ‘mentions’ across the web isn’t as helpful as it might seem,” while noting its AI features can reflect what’s said about a business across the web. Genuine reviews and first-hand content are the valuable kind.
An Ahrefs study of 75,000 brands found branded web mentions correlated with AI Overview mentions more strongly (0.664) than backlinks (0.218). That’s correlation, not proof of cause, and it’s from a vendor’s own tools, but it fits the practical advice: be the kind of business people genuinely mention, review and link to, and make sure your profiles (Google Business Profile, LinkedIn, industry directories) are consistent and complete.
A practical checklist
- Allow the search crawlers you want to appear in (OAI-SearchBot, PerplexityBot, Claude-SearchBot, Googlebot, Bingbot) in robots.txt. Decide separately about training crawlers; that’s a business choice.
- Make sure key pages are indexable, load fast and render without JavaScript.
- Put a direct answer in the first 40–70 words of each page, then support it.
- Use real numbers and name your sources. Quote primary documents.
- Structure pages with clear headings, tables and FAQs: it helps people and extraction.
- Keep content fresh and show update dates.
- Verify Google Search Console and Bing Webmaster Tools, and watch their AI reports.
- Build real off-site presence: reviews, directory profiles and genuine mentions.
- Skip the tricks: keyword stuffing and fake mentions don’t help, and the research shows stuffing hurts.
Want a site built to be found, and cited?
Our websites ship with clean structure, fast pages, sourced content and the technical SEO covered above. Book a free 20-minute call for a fixed quote.
Frequently asked questions
How do I get my business to show up in ChatGPT?
Allow OAI-SearchBot in robots.txt, keep pages indexable, and publish clear, factual content with sources. OpenAI says sites that opt out of OAI-SearchBot won’t appear in ChatGPT search answers. ChatGPT search also draws on Bing, so verify your site in Bing Webmaster Tools.
Do I need special schema or an llms.txt file for Google AI Overviews?
No. Google says there are no additional requirements, no special schema.org markup and no new AI files needed, and that Google Search ignores llms.txt files.
Should I block AI crawlers like GPTBot and ClaudeBot?
It depends on your goals. Training crawlers (GPTBot, ClaudeBot, Google-Extended) are separate from search crawlers. Blocking training bots keeps content out of model training; blocking search bots like OAI-SearchBot removes you from that platform’s search answers.
What kind of content gets cited most by AI?
In the GEO research study, adding quotations, statistics and cited sources increased visibility by roughly 27–41%, while keyword stuffing reduced it. Smaller sites gained the most from citing sources.
Can I track whether AI tools are citing my site?
Yes, partly. Google Search Console has a generative AI performance report, and Bing Webmaster Tools has an AI Performance report showing citations in Copilot and Bing AI summaries. ChatGPT and Perplexity don’t offer equivalent official dashboards.
Sources
- OpenAI: Overview of OpenAI crawlers
- Anthropic: Does Anthropic crawl data from the web?
- Perplexity: Perplexity crawlers
- Google Search Central: AI features and your website
- Google Search Central: Optimizing your website for generative AI features
- Google: Common crawlers (Google-Extended)
- Aggarwal et al.: GEO: Generative Engine Optimization (KDD 2024)
- Bing Webmaster Blog: Introducing AI Performance in Bing Webmaster Tools (Feb 2026)
- Search Engine Roundtable: Bing powers ChatGPT search (Nov 2024)
- Search Engine Journal: Bing adds GEO to official guidelines (Feb 2026)
- llms.txt proposal
- IndexNow
- Ahrefs: AI Overview brand correlation study (May 2025)
Market figures are as published by each source and were checked on 11 November 2026. Sources define categories differently, so treat ranges as a guide rather than a quote.
More guides
- WhatsApp chatbots for businesses in the UAE and Gulf: costs, rules and setup
- Shopify vs WooCommerce for UK small businesses (2026)
- Real estate websites in Dubai: Trakheesi permits, listings and must-have features
Related: Our services · How we work · Pricing · All guides