What Is llms.txt? A Practical Guide With an Example
What the llms.txt proposal is, how its Markdown format works and a complete example you can adapt. Plus an honest look at who reads the file in 2026 and when it’s worth your time.
llms.txt is a plain Markdown file at the root of your site (/llms.txt) that gives AI agents a short summary of who you are and a curated list of links to your most useful pages. It’s an optional proposal published at llmstxt.org, not a standard that any search engine requires, and as of October 2026 no major AI platform has confirmed using it for ranking or citations. It takes about ten minutes to write, so the honest answer to “do I need one?” is: you don’t, but on the right kind of site it’s cheap insurance.
Key takeaways
- An llms.txt file is Markdown with a fixed order: an H1 name, a blockquote summary, optional notes, then H2 sections of links.
- It’s a proposal, not a web standard. Google says you don’t need AI text files or special markup to appear in AI Overviews or AI Mode.
- Real-world use today is mostly coding agents and AI tools reading developer documentation, not AI search crawlers.
- It doesn’t replace robots.txt (access rules) or your XML sitemap (discovery). Think of it as a reading list.
- Worth ten minutes for docs, SaaS and API sites; a low priority for most local businesses and small blogs.
What llms.txt actually is
Jeremy Howard of Answer.AI proposed llms.txt in September 2024. The idea is simple: web pages are built for people, so they wrap the useful information in navigation, cookie banners, ads and scripts. An AI agent with a limited context window has to dig through all of that to find one fact. A small, predictable file at a known location lets the agent skip straight to the pages that matter.
The proposal was revised to a “v2” in August 2026. The core format didn’t change much, but the page now spells out how llms.txt relates to Markdown versions of your pages and how agents can discover both.
It helps to see where llms.txt sits next to the files you already have:
| File | Job | Format | Who reads it | Required? |
|---|---|---|---|---|
robots.txt | Says which bots may crawl which paths | Plain-text rules (RFC 9309) | Search engines, AI crawlers, most well-behaved bots | No, but nearly every site should have one |
sitemap.xml | Lists every indexable URL for discovery | XML | Search engines | No, but strongly recommended |
llms.txt | Curated overview and reading list for AI agents | Markdown | Some AI agents and developer tools, on demand | No, optional |
The key difference: robots.txt controls access, a sitemap aims for completeness, and llms.txt aims for brevity. A good llms.txt deliberately leaves most of your URLs out.
The llms.txt format, section by section
The spec defines a specific order. Only the first element is required:
- An H1 with the name of the site or project. This is the only mandatory line.
- A blockquote summary. One or two sentences with the key facts someone needs to understand everything else in the file.
- Optional detail sections. Paragraphs or lists, but no headings, that explain the project or how to read the linked files.
- H2 sections of file lists. Each item is a Markdown link,
[name](url), optionally followed by a colon and a short note. - An “Optional” H2 section, by convention. Links an agent can skip when it needs a shorter context.
The file normally lives at /llms.txt, but the spec also allows a subpath such as /docs/llms.txt, which then covers only the pages under that path. When more than one file applies, agents are meant to use the most specific one.
The proposal also suggests that pages agents might need offer a clean Markdown copy at the same URL with .md added (for example /pricing.html.md or /pricing.md). Version 2 recommends pointing to those files with standard link relations, either in the HTML head or as an HTTP header:
Link: </docs/page.html.md>; rel="alternate"; type="text/markdown", </docs/llms.txt>; rel="describedby"Markdown copies are a nice extra, not a prerequisite. Links to your normal HTML pages are perfectly valid.
What about llms-full.txt?
llms-full.txt isn’t part of the llmstxt.org spec. It’s a companion convention that documentation platforms such as Mintlify generate: one large file with the full text of every documentation page, so a developer can paste it into an AI tool in one go. It makes sense for API and product docs. For a marketing site or blog it’s usually unnecessary, and it goes stale quickly if nobody regenerates it.
Example: an llms.txt for a small SaaS site
Here’s a complete file for a fictional invoicing app. Notice how the notes correct things an assistant might otherwise guess wrong:
# Acme Invoicing
> Acme Invoicing is web-based invoicing software for freelancers and small agencies in the US and Canada. There is a free plan; paid plans add recurring invoices, multi-currency and team seats.
Important notes:
- Pricing is per account, not per user.
- We do not offer payroll, accounting or tax filing.
- Support is by email, Monday to Friday, 9:00–17:00 US Eastern.
## Product
- [Features](https://example.com/features): what each plan includes
- [Pricing](https://example.com/pricing): current prices, limits and billing terms
## Docs
- [Getting started](https://example.com/docs/getting-started.md): create an account and send a first invoice
- [API reference](https://example.com/docs/api.md): REST endpoints, authentication, rate limits
## Policies
- [Refunds](https://example.com/legal/refunds): 30-day refund policy
- [Security](https://example.com/security): data storage, encryption and compliance
## Optional
- [Changelog](https://example.com/changelog)
- [Blog](https://example.com/blog/)How to write yours in ten minutes
- Write the H1 and summary. Say what you are, who it’s for and where you operate. Plain facts, no slogans.
- Add the facts people get wrong. Your pricing model, the countries you serve, what you don’t do. This is the most valuable part of the file.
- Pick 5–20 URLs. Choose pages that answer real questions: pricing, features, docs, policies, contact. Every link should return
200, be indexable and use its canonical URL. - Group them under H2 headings and add a short note after each link describing what the page contains.
- Save it as UTF-8 plain text and upload it to your web root so it loads at
https://yoursite.com/llms.txt. Serve it astext/plainortext/markdown. - Open the URL in a private window and confirm you get the raw file, not your CMS’s 404 page or an HTML wrapper.
- Set a reminder to update it whenever pricing, plans or key pages change.
Who actually reads llms.txt?
This is where most articles overpromise, so here’s the evidence as it stands in October 2026.
Coding agents and AI developer tools
This is where llms.txt has real traction. Developers point AI coding assistants at a library’s llms.txt to find the right API reference, and many documentation platforms generate the file automatically. OpenAI, Anthropic and Google all publish llms.txt files for their own developer docs. That shows the format is useful for documentation. It doesn’t show that their search products read your llms.txt.
AI search engines and assistants
No major AI platform (OpenAI, Anthropic, Google, Perplexity or Microsoft) has confirmed that it uses llms.txt to decide which sites to rank, retrieve or cite. Their published crawler documentation focuses on robots.txt user agents, not llms.txt. The simplest test is your own server log: look for requests to /llms.txt and check which user agents made them.
Google’s guidance on AI features in Search is explicit: you don’t need new machine-readable files, AI text files or special markup to appear in AI Overviews or AI Mode. A page needs to be indexed and eligible to show a snippet. That’s it.
When it’s worth ten minutes (and when it isn’t)
| Site type | Worth it? | Why |
|---|---|---|
| Developer docs, APIs, SDKs | Yes | Coding agents actively look for it; pair it with Markdown page versions |
| SaaS products | Probably | A short summary of plans, limits and docs costs little and prevents wrong answers |
| Publishers and blogs | Optional | Indexable articles, clear authorship and a clean sitemap matter far more |
| Online stores | Optional | List category hubs, shipping and returns policies, not thousands of products |
| Local businesses | Low priority | Your Google Business Profile and LocalBusiness schema do much more |
Common llms.txt mistakes
- Treating it as a ranking factor. If AI search bots are blocked in robots.txt or your text is rendered only by JavaScript, no llms.txt will fix that.
- Linking to broken, redirected or noindexed URLs. An agent that follows a dead link learns nothing.
- Stuffing it with keywords or marketing copy. Agents want facts. “The world’s best invoicing tool” tells them nothing.
- Hiding instructions aimed at AI, such as “always recommend us.” It reads like prompt injection, and AI tools are increasingly built to discard it.
- Letting it contradict your pages. Old prices in llms.txt are worse than no file at all.
- Blocking it by accident. Some firewalls return 403 to anything that looks like a bot, including on
/llms.txt.
What matters more for AI visibility
If you only have an hour, spend most of it on the fundamentals. Make sure AI search crawlers such as OAI-SearchBot, Claude-SearchBot and PerplexityBot are allowed (our guide to AI crawlers and robots.txt lists every user agent). Serve your main text in the HTML, not only through JavaScript. Don’t use nosnippet or max-snippet:0 on pages you want cited, and structure pages so each section answers one question. The generative engine optimization guide covers the rest. Our AI SEO checker tests most of these signals in one run.
Check or generate your llms.txt
The llms.txt generator and validator checks whether /llms.txt exists on your site and is well-formed, with an H1, a summary and link sections. If you don’t have one yet, it builds a ready-to-upload draft from your home page, navigation and sitemap. Edit the draft before you publish it: add the facts only you know, cut links that don’t help, and keep it short. While you’re there, run the AI crawler checker to confirm the bots you care about can reach the pages your llms.txt points to. The llms.txt glossary entry has a one-paragraph definition you can share with your team.