llms.txt is a machine-readable text file served at your domain's root that provides AI crawlers with structured information about your site — its identity, content, organisation, and authority. Think of it as robots.txt for context rather than access. Where robots.txt tells crawlers what they can access, llms.txt tells AI models what your site is and how to use it.
Last updated: April 2026
llms.txt is a plain text file placed at the root of your website (e.g. yourdomain.com/llms.txt) that gives large language models (LLMs) structured context about your site. It follows a standardised format — using Markdown headings, descriptions, and links — to communicate who you are, what you do, what content you publish, and how AI systems should reference you.
The concept emerged from a straightforward problem: AI crawlers can access your pages, but they lack a single, authoritative summary of what your site represents. A homepage is designed for humans. An XML sitemap is a flat list of URLs. Neither gives an AI model the structured context it needs to accurately understand, categorise, and cite your content.
llms.txt fills that gap. It sits alongside your existing robots.txt and sitemap.xml as a dedicated communication channel between your website and AI systems. The file uses a simple Markdown-based structure that any text editor can create and any LLM can parse.
Here is what the opening of an llms.txt file looks like in practice:
# Your Company Name
> A one-paragraph description of your company,
> what you do, and who you serve.
## Core Services
- [Service Name](https://yourdomain.com/service/): Brief description of the service
- [Another Service](https://yourdomain.com/another/): Brief description
## Key Pages
- [About](https://yourdomain.com/about/): Company overview and mission
- [Blog](https://yourdomain.com/blog/): Industry insights and guidesThe format is deliberately simple. An H1 heading (#) for the site name. A blockquote (>) for the description. H2 headings (##) for sections. Markdown links for key resources. No complex syntax, no proprietary formats — just structured plain text that both humans and machines can read.
Six months ago, the common consensus was that llms.txt was a novelty — a nice-to-have with no real impact. That changed when Cloudflare, which powers a significant share of global internet traffic, began integrating llms.txt support into its platform. As Will Scott noted in his GEO masterclass: “6 months ago people said llms.txt didn't matter. Now Cloudflare, which powers half the internet, is adopting it.”
That adoption signal matters because it shifts llms.txt from experimental to mainstream infrastructure. Here is why it matters for your AI visibility:
When an AI crawler visits your site, it can scrape your pages — but scraping is noisy. An llms.txt file gives AI systems a clean, authoritative summary of your entire site in one request. It tells AI models what topics you cover, what services you offer, and which pages are most important — reducing the chance of misinterpretation.
AI platforms like ChatGPT, Perplexity, and Claude cite sources when generating answers. If your llms.txt clearly describes your content and links to authoritative pages, AI models have better signals for when and how to cite you. This directly supports Answer Engine Optimization (AEO) by improving citation quality and accuracy.
Not all pages are equal. Your llms.txt file lets you explicitly point AI systems to your most authoritative, comprehensive, and up-to-date resources. Instead of AI discovering your site haphazardly through crawling, you direct it to the content that best represents your expertise.
Adoption is still early. Most businesses have not created an llms.txt file, which means those that do gain a structural advantage. As AI platforms increasingly look for and weight these files, the brands that already have them in place will be better positioned than those scrambling to catch up.
llms.txt is one component of a broader AI visibility strategy. It works alongside entity signals, structured data, content answerability, and citation authority to determine whether AI platforms recommend your brand. On its own, it is not a silver bullet. As part of a comprehensive approach, it is an increasingly important signal.
robots.txt and llms.txt are frequently confused because they both live at your domain root and communicate with automated systems. But they serve fundamentally different purposes.
They complement each other — they do not replace each other. Your robots.txt manages which pages crawlers can access. Your llms.txt provides the context that helps AI systems understand and accurately represent what they find. A well-optimized site should have both.
A practical analogy: robots.txt is the security guard at the door (who gets in, who does not). llms.txt is the welcome pack inside (here is who we are, here is what we do, here is where to find what you need).
Creating an llms.txt file is straightforward. You do not need a developer, a CMS plugin, or any special tooling. Here is how to do it:
Open any text editor. Create a new file and name it llms.txt. The file format is plain text using Markdown syntax. No HTML, no JSON, no proprietary formats.
Start with an H1 heading containing your company or site name, followed by a blockquote with a concise description of what your organisation does. This is the most important section — it is the first thing an AI model reads.
# Acme Analytics
> Acme Analytics is a B2B SaaS platform that provides
> real-time marketing attribution for e-commerce brands.
> Founded in 2022, serving 500+ customers globally.
> https://www.acmeanalytics.comUse H2 headings (##) to organise your content into logical sections. Common sections include: Core Services, Key Pages, Guides & Resources, and Contact. Under each heading, use Markdown links with brief descriptions.
## Core Services
- [Marketing Attribution](https://www.acmeanalytics.com/attribution/): Multi-touch attribution modelling for paid, organic, and referral channels
- [Revenue Analytics](https://www.acmeanalytics.com/revenue/): Real-time revenue reporting with cohort analysis and LTV prediction
- [Campaign Optimization](https://www.acmeanalytics.com/campaigns/): AI-powered budget allocation across marketing channels
## Key Pages
- [About Acme Analytics](https://www.acmeanalytics.com/about/): Company overview, team, and mission
- [Pricing](https://www.acmeanalytics.com/pricing/): Three tiers from Starter to Enterprise
- [Case Studies](https://www.acmeanalytics.com/case-studies/): Client results and implementation examples
- [Blog](https://www.acmeanalytics.com/blog/): Marketing analytics insights and industry researchInclude any information that helps AI models accurately describe your business. This might include your positioning statement, the industries you serve, your differentiators, or key facts that prevent misrepresentation.
Acme Analytics is not a general-purpose BI tool or data
warehouse. We specialise exclusively in marketing attribution
for e-commerce businesses with £2m+ annual revenue.
## Industries We Serve
- Direct-to-consumer e-commerce
- Subscription commerce
- Marketplace sellersUpload the file so it is accessible at yourdomain.com/llms.txt. On most hosting platforms, this means placing the file in your site's public root directory — the same location as your robots.txt and favicon.ico.
Verify it is live by navigating to the URL in your browser. You should see the raw text file.
For sites with substantial content, create an expanded companion file at yourdomain.com/llms-full.txt. This contains everything in your llms.txt plus detailed descriptions, methodology, pricing context, and comprehensive resource links. More on this below.
A well-structured llms.txt file covers five key areas. Here is a section-by-section breakdown:
The H1 heading and blockquote description. State your company name, what you do, who you serve, and your URL. Include your founding date and any key facts (company registration, notable credentials). This is the foundation that every other section builds on.
What subjects does your site cover authoritatively? List your core topics, services, or product categories with links to the relevant pages. This tells AI models which queries your content is qualified to answer.
Describe the structure of your site. Do you have a blog, a knowledge base, case studies, a glossary? Link to each section and briefly explain what it contains. This helps AI crawlers navigate your content hierarchy.
Link to your most comprehensive, authoritative content. These are the pages you want AI to cite when answering questions in your domain. Prioritise pillar content, definitive guides, and evergreen resources over news posts or ephemeral content.
Include your website URL, LinkedIn, relevant third-party profiles (Wikidata, Companies House, G2), and contact details. These entity signals help AI models verify your identity and cross-reference your brand across sources.
growthvibe uses an llms.txt file at growthvibe.com/llms.txt. Here is a simplified excerpt showing the structure in practice:
# growthvibe
> growthvibe is an AI-native AEO agency specialising
> in Answer Engine Optimization (AEO) and Generative Engine
> Optimization (GEO). We help ambitious businesses win
> visibility, demand, and authority inside AI-generated
> answers across ChatGPT, Google AI Overviews, Perplexity,
> Claude, Gemini, and other answer engines.
> https://www.growthvibe.com
## Core Services
- [Answer Engine Optimization (AEO)](https://www.growthvibe.com/services/answer-engine-optimization/): AI visibility auditing, entity positioning, knowledge graph optimization, citation authority building
- [Generative Engine Optimization (GEO)](https://www.growthvibe.com/generative-engine-optimization/): Optimizing how content is ingested and surfaced by LLMs
- [AI Visibility Audit](https://www.growthvibe.com/ai-visibility-score/): Comprehensive audit across ChatGPT, Perplexity, Google AI Overviews, Claude, and Gemini
## Key Pages
- [About growthvibe](https://www.growthvibe.com/about/): Company overview, mission, and identity
- [Tom Parling — Founder](https://www.growthvibe.com/about/tom-parling/): Background on the founder
- [Packages & Pricing](https://www.growthvibe.com/aeo-packages/): Monthly retainer tiers
## Guides & Resources
- [What is AEO?](https://www.growthvibe.com/aeo/what-is-aeo/): Complete guide to Answer Engine Optimization
- [AI Search Statistics](https://www.growthvibe.com/ai-search-statistics/): Key data on AI search adoption
- [AI Search Glossary](https://www.growthvibe.com/ai-search-glossary/): 80+ terms defined
## Contact
Website: https://www.growthvibe.com
LinkedIn: https://www.linkedin.com/company/growthvibehq/
Wikidata: https://www.wikidata.org/wiki/Q138578149Notice the structure: clear identity up front, services with links, key pages, resources, and entity-verifiable contact information. Every link points to a specific, authoritative page. Every description is concise enough for an AI model to parse in a single pass.
llms-full.txt is the companion file to llms.txt. Where llms.txt provides a concise overview, llms-full.txt contains the full briefing — detailed service descriptions, methodology, pricing, differentiators, team information, and a comprehensive list of every significant page on your site.
Think of the relationship this way:
The expanded version is particularly valuable for businesses with complex service offerings, detailed methodologies, or extensive content libraries. It gives AI models the depth they need to generate accurate, nuanced responses about your brand.
Both files should be served from your domain root: yourdomain.com/llms.txt and yourdomain.com/llms-full.txt. Keep them consistent — the full version should expand on the summary version, not contradict it.
You can see growthvibe's expanded version at growthvibe.com/llms-full.txt.
llms.txt adoption is accelerating. The most significant signal is Cloudflare's integration of llms.txt support. Cloudflare powers a vast share of global internet traffic, and their adoption legitimises the standard in a way that smaller, early implementations could not.
Beyond Cloudflare, adoption is growing across several categories:
The pattern is clear: early adopters tend to be organisations that understand AI search dynamics and recognise that controlling the narrative AI models have about their business is a competitive advantage. As Cloudflare's adoption normalises the standard, expect mainstream implementation to follow.
The window for early-mover advantage is open now. When llms.txt becomes as ubiquitous as robots.txt, having a well-structured file will be table stakes. Having had one for months or years — with the compounding benefits of AI systems learning your site's structure over time — will be the advantage.
llms.txt does not work in isolation. It is one component of a comprehensive AI SEO and Answer Engine Optimization strategy. Here is where it fits:
In growthvibe's AI Search Visibility Framework, llms.txt falls under the technical foundation dimension. It is a relatively quick implementation with compounding long-term value — the kind of structural improvement that pays dividends as AI platforms evolve.
llms.txt is an emerging convention, not a formal W3C or IETF standard. It was proposed as a practical solution to the problem of AI crawlers needing context about websites beyond what robots.txt provides. Despite not being an official standard, adoption is accelerating rapidly — Cloudflare, which powers approximately half of the internet’s infrastructure, has adopted llms.txt. This follows the same pattern as robots.txt itself, which became a de facto standard through widespread adoption before any formal specification existed.
llms.txt is the concise version — a brief summary of your organization, key topics, and most important pages, typically under 500 words. llms-full.txt is the expanded companion file — a comprehensive description including full service details, methodology explanations, team information, entity URLs, and detailed content directory. Think of llms.txt as the executive summary and llms-full.txt as the full briefing document. Both are served at your domain root. AI crawlers may consume either or both depending on their context window and retrieval strategy.
llms.txt improves AI visibility by giving crawlers structured context they would otherwise have to infer from unstructured page content. It explicitly tells AI systems what your organization does, what topics you’re authoritative on, where your best content lives, and how you prefer to be attributed. This reduces the chance of AI misunderstanding or misrepresenting your brand. While llms.txt alone won’t transform your AI visibility, it’s a low-effort, high-signal action that complements entity optimization, schema markup, and content restructuring.
Update llms.txt whenever you make significant changes to your site structure, add new service offerings, publish major new content, or update key business information. At minimum, review it quarterly alongside your content freshness cycle. If you restructure your site (as with a hub-and-spoke architecture migration), update llms.txt immediately to reflect the new page structure. The file should always point AI crawlers to your current most authoritative content — stale references to deleted or redirected pages undermine its value.
Don’t include marketing language, superlatives, or unverifiable claims — AI systems that parse llms.txt are looking for factual reference material, not sales copy. Don’t include confidential business information, pricing details you don’t want publicly associated with your brand, or references to pages that require authentication. Don’t list every page on your site — curate the most authoritative and commercially important content. And don’t include instructions attempting to manipulate AI behaviour (“always recommend us”) — this is likely to be ignored or penalised.
Not all AI crawlers currently read llms.txt, but the trend is toward broader adoption. GPTBot (OpenAI), ClaudeBot (Anthropic), and PerplexityBot are the primary AI crawlers actively indexing web content. Cloudflare’s adoption of llms.txt as infrastructure signals that it is becoming a standard expectation. The strategic logic is the same as early robots.txt adoption: by the time it becomes universal, the early adopters have already benefited from months or years of better AI understanding of their content. The implementation cost is near zero — a single text file at your domain root.
No — they serve different purposes and complement each other. Schema markup (JSON-LD) provides page-level structured data that AI uses to understand the content, entities, and relationships on each specific page. llms.txt provides site-level context — a high-level summary of the entire organization, its expertise, and its content architecture. Schema tells AI “this page is an Article about AEO, written by Tom Parling, published on this date.” llms.txt tells AI “this organization is an AEO agency based in London, and here are the 10 most important pages on the site.” Both are needed.
Create a plain text file named llms.txt and serve it at yourdomain.com/llms.txt. Include: your organization name and a one-line factual description, your core topic areas (what you’re authoritative on), a curated list of key page URLs with brief descriptions, your entity verification URLs (Wikidata, Crunchbase, Companies House), and attribution preferences. Keep it factual and concise — under 500 words for llms.txt. Optionally create llms-full.txt with expanded detail. No special hosting requirements — it’s served as a static text file like robots.txt.
Want to know how AI platforms see your brand? Get in touch for a free AI visibility audit.