llms.txt: what it is and whether your site needs one

llms.txt is a community proposal, not a web standard: a Markdown file at /llms.txt that offers LLM-friendly content. Google Search does not use it, so you don't need one to appear in Google. For other AI tools the evidence that they read it is still weak, which makes adding one a cheap option and nothing more.

• llms.txt is a Markdown file at /llms.txt. The proposal was published on 3 September 2024 and last modified on 10 August 2026.
• Google Search does not use it, so it has no bearing on whether you appear there.
• An SE Ranking study found no link between llms.txt and more AI citations.
• Only 8.7% of the world's top 1,000 websites published one in June 2026, per Rankability.

What is llms.txt?

Jeremy Howard wrote the proposal. It asks websites to add a /llms.txt file, and the file can also sit at subpaths such as /docs/llms.txt, according to llmstxt.org.

The stated purpose is to give LLMs content they can read easily. The file is used most heavily for software documentation, where coding agents follow it to find API references and tutorials. The same structure can describe a business's structure and policies, but that is an extension of the main use.

GitBook calls it an emerging convention with no single fixed schema. For a longer walkthrough, see Citable's llms.txt explainer.

What does the format require?

Very little. The only required element is an H1 with the name of the site or project. A blockquote summary, H2 link lists and an "Optional" section are conventions, as the format description explains.

A minimal file looks like this:

# Example Site

> A short summary of what the site offers.

## Key pages

Adding one is cheap. The same proposal says many documentation platforms and CMSs can generate the file automatically, including Wix, Yoast SEO and AIOSEO for WordPress.

Does Google use llms.txt?

No. Google's guidance, via TechWyse, says site owners do not need to create new machine-readable or Markdown files to appear in Google Search, because Google Search itself doesn't use them.

Google also says that creating and maintaining llms.txt files is fine for other services or systems that use them. Keeping one neither helps nor harms Google Search visibility.

That position covers Google only. It says nothing about how other AI tools behave.

Do ChatGPT, Claude, Gemini and Perplexity read it?

The evidence does not settle it.

"Whether LLMs.txt affects ChatGPT, Perplexity, or other platforms remains an open question." — Search Engine Land

An SE Ranking machine-learning study of about 300,000 domains, reported by OrganiKPI, found that llms.txt did not correlate with being cited more often by AI systems. Removing it improved the model's accuracy.

Server logs from one site point the same way. Prosopo published llms.txt from May 2026 and reviewed four days of IP-verified data. Verified AI crawlers fetched robots.txt 120 times and sitemaps 43 times, but llms.txt and llms-full.txt zero times. That is one site over a short period.

Ahrefs notes that Anthropic publishes its own llms.txt but does not say its crawlers use the standard. Creating the file is not the same as crawlers reading it.

How many sites already publish one?

Three studies give different numbers because they sample different sites:

Source	Sample	Date	Share with llms.txt
Rankability	World's top 1,000 websites	June 2026	8.7%
Underneath	Tranco top 10,000, live sites	September 2026	11.5% (15.2% of the top 1,000)
LLM Visibility Tools	219 hosts, weighted to developer tools, SaaS and AI companies	3 August 2026	51.8%

The 51.8% panel figure is about six times the 8.7% tracker figure, because of sample choice, and each is correct for its own sample. For general-web claims, the relevant numbers are 8.7% of the top 1,000 (June 2026) and 11.5% of the top 10,000 (September 2026). The developer-heavy panel describes a different group of sites.

These figures show how common the file is. They say nothing about whether it improves citations.

Can llms.txt replace robots.txt or block crawlers?

No to both. According to the proposal, robots.txt states what access is acceptable, while llms.txt is used on demand when an agent needs information about a topic. The two files have different purposes.

If you want to control what AI bots can reach, robots.txt and your access rules still do that job. To see whether your robots.txt blocks ChatGPT, Perplexity or Claude, you can run an AI crawler checker.

Do you need an llms.txt file?

The file costs little and has no proven upside. That makes the decision about fit, and three questions help:

• Audience: If your readers or their tools are developers and coding agents, the proposal's main use case matches your site.
• Content: If you keep clean, readable versions of your key pages, the file has something good to point to.
• Upkeep: If someone will update the file when pages change, it won't send agents to outdated URLs.

If all three fit, adding the file is low-risk. If none do, your time is better spent on the pages themselves.

How can you test whether it matters?

Start with your server logs. Count requests to /llms.txt and /llms-full.txt over a fixed period and compare them with robots.txt and sitemap requests, as Prosopo did.

Then measure what AI tools say about your brand before and after any change, across several prompts. This guide to checking ChatGPT explains why one prompt isn't enough.

Next steps

• Check your server logs for requests to /llms.txt over the last 30 days, and compare them with requests to robots.txt.
• Open your robots.txt and confirm it allows the AI crawlers you want to reach your site.
• Check whether your CMS or SEO plugin (Wix, Yoast SEO, AIOSEO) can generate the file before you write one by hand.
• Ask ChatGPT, Gemini and Perplexity the questions your buyers ask, and record whether you appear on the shortlist. Repeat after any change.
• If you prefer a generator, Citable has an llms.txt generator among its tools.

Frequently asked questions

Does llms.txt affect ChatGPT, Claude, Gemini or Perplexity citations?

The evidence is weak. An SE Ranking study of about 300,000 domains, reported by OrganiKPI, found no correlation between llms.txt and being cited more often. Search Engine Land calls the question open.

Is llms.txt the same as robots.txt, and can it block AI crawlers?

No. Robots.txt states what access is acceptable, while llms.txt is used on demand when an agent needs information about a topic, per the llms.txt proposal. It cannot block crawlers.

Can my CMS generate an llms.txt file for me?

Often, yes. The proposal says many documentation platforms and CMSs generate the file automatically, including Wix, Yoast SEO and AIOSEO for WordPress.

How can I check whether AI crawlers request my llms.txt file?

Search your server logs for requests to /llms.txt and /llms-full.txt, and compare them with robots.txt and sitemap requests. Prosopo did this with four days of IP-verified data and found zero llms.txt fetches from verified AI crawlers.

Does Google use llms.txt to show my site in Search?

No. Google's guidance, via TechWyse, says Google Search itself doesn't use these files, and you don't need new Markdown or machine-readable files to appear in it.

Sources

• The /llms.txt file, v2 – llms-txt
• What is llms.txt? Why it's important and how to create it for your docs – GitBook
• Google Says llms.txt Will Not Help Rankings – TechWyse
• The Markdown Myth: What Google Actually Said About llms.txt and AI SEO
• Does your website need an llms.txt file? – Search Engine Land
• llms.txt in 2026: Adoption Data and When to Use It – OrganiKPI
• llms.txt: we publish one, and no AI crawler has ever read it – Prosopo
• What is llms.txt? – Ahrefs
• LLMS.txt Adoption: 8.7% of the Top 1,000 – Rankability
• How many websites have an llms.txt file? 2026 adoption data – Underneath
• 8.7% or 51.8%? Grading the Two llms.txt Adoption Studies That Disagree by 6x