llms.txt, ai.txt and Markdown pages

Every help center publishes what AI tools and crawlers need to find and read it: an llms.txt index, an ai.txt with its AI usage preferences, content signals in robots.txt, an API catalog, Link headers, and a Markdown version of every page. There is nothing to set up. This page describes each one for developers who build agents, crawlers or integrations.

Owners switch these off in Settings. See How search engines and AI crawlers read your help center.

At a glance

They all work on the help center's own address, including its custom domain. On a private, password-protected or IP-restricted help center, an anonymous request is sent to sign in or to the password page, like any page.

llms.txt

curl -s https://help.example.com/llms.txt

The response is plain text (text/plain; charset=utf-8) in the llms.txt format:

# Acme Help Center

> Answers about your Acme account, billing and invoices.

Every page listed below also serves clean Markdown at the same URL when
requested with `Accept: text/markdown`.

AI assistants can also search and read this help center live through its
read-only MCP server at https://help.example.com/mcp (Streamable HTTP,
no authentication; tools: `search`, `fetch`, `list_categories`).

## Account

Sign-in, passwords and your profile.

- [Reset your password](https://help.example.com/content/reset-your-password): Set a new password from the sign-in page when you cannot remember yours.
- [Close your account](https://help.example.com/content/close-your-account)
- [Change your email address](https://help.example.com/content/change-your-email-address)

## Billing

Plans, invoices and payment methods.

- [Change your plan](https://help.example.com/content/change-your-plan)

### Invoices

Find, download and share your invoices.

- [Download an invoice](https://help.example.com/content/download-an-invoice): Download the PDF invoice for any payment from the Billing page.

## Optional

- [Acme Help Center](https://help.example.com): Help center home
- [All categories](https://help.example.com/categories): Every category in this help center
- [Sitemap](https://help.example.com/sitemap.xml): XML sitemap of every public page

• Title and summary. The heading is the help center's default page title and the quote its description, from its SEO settings.

• The MCP paragraph appears only while the help center has a public MCP server.

• One section per category, nested as the categories are (## for top-level categories, ### for their subcategories, and so on), each with its description and its articles. An article's link is followed by its SEO description when it has one. Articles in no category are listed under ## Other articles.

• The default language only. Other languages are reachable as Markdown pages and through the public MCP server.

• What anyone can read, nothing more. Only published articles that an anonymous visitor can open: no drafts, private or team-only articles and categories, link-only articles, unreleased translations, or categories filed under a hidden parent.

An anonymous visitor's copy is sent with Cache-Control: public, max-age=3600. A team member who is signed in sees their team's content too, so their copy is sent with private, no-store. With Search engine indexing off, /llms.txt answers 404. There is no /llms-full.txt: that address redirects to the home page.

ai.txt

curl -s https://help.example.com/ai.txt

It states that AI assistants may read and cite the help center and that training models on it is not permitted, and points to llms.txt and the public MCP server:

# ai.txt for help.example.com
# AI content-usage preferences for this help center.
# Machine-readable index of this help center: https://help.example.com/llms.txt
# Read-only MCP server for AI assistants (no authentication): https://help.example.com/mcp
# Content signals — https://contentsignals.org
Content-Signal: search=yes, ai-input=yes, ai-train=no

# This help center's content belongs to its publisher. AI assistants are
# welcome to crawl these pages and cite them when answering questions about
# this product. Training models on them is not permitted.

User-agent: *
Allow: /

Sitemap: https://help.example.com/sitemap.xml

The MCP line appears only while the help center has a public MCP server. With Search engine indexing off, the file says so and asks every crawler to stay out:

# ai.txt for help.example.com
# AI content-usage preferences for this help center.
# This help center has opted out of search engines and AI crawlers.
Content-Signal: search=no, ai-input=no, ai-train=no

User-agent: *
Disallow: /

Both are cached like llms.txt.

robots.txt

A help center's robots.txt has one set of rules for every crawler (User-agent: *). With indexing on, its directives are (comment lines left out):

User-agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=no
Disallow: /article/
Disallow: /_hc/
Sitemap: https://help.example.com/sitemap.xml

/article/ and /_hc/ are not pages: they receive comments and ratings and issue form tokens. With Search engine indexing off, its directives become:

User-agent: *
Content-Signal: search=no, ai-input=no, ai-train=no
Disallow: /

The Content-Signal line says whether search engines may index the pages (search), whether AI assistants may read them to answer questions (ai-input), and whether they may be used to train AI models (ai-train, always no). Like the rest of robots.txt, these are instructions that crawlers choose to follow.

API catalog

curl -s https://help.example.com/.well-known/api-catalog

The catalog is a linkset (application/linkset+json; profile="https://www.rfc-editor.org/info/rfc9727") with two contexts: the help center, and its public MCP server.

{
  "linkset": [
    {
      "anchor": "https://help.example.com/",
      "service-doc": [
        { "href": "https://help.example.com/", "type": "text/html", "title": "Acme Help Center" }
      ],
      "service-desc": [
        { "href": "https://help.example.com/sitemap.xml", "type": "application/xml", "title": "Sitemap" },
        { "href": "https://help.example.com/llms.txt", "type": "text/plain", "title": "llms.txt" }
      ]
    },
    {
      "anchor": "https://help.example.com/mcp",
      "service-desc": [
        { "href": "https://help.example.com/mcp/server-card", "type": "application/mcp-server-card+json", "title": "MCP server card" }
      ],
      "service-doc": [
        { "href": "https://help.example.com/mcp", "type": "text/html", "title": "Connect an AI assistant" }
      ]
    }
  ]
}

Without indexing, the llms.txt entry is left out; without a public MCP server, the second context is. The catalog is cached like llms.txt.

Link headers

Every page of the help center, as HTML or Markdown, points agents to the catalog and the sitemap with two Link headers:

Link: <https://help.example.com/.well-known/api-catalog>; rel="api-catalog"
Link: <https://help.example.com/sitemap.xml>; rel="describedby"; type="application/xml"

Markdown for any page

Ask for Markdown with the Accept header and a page answers in Markdown at the same URL. It works on the home page, /categories, category pages and article pages. There are no separate .md addresses.

curl -s -H "Accept: text/markdown" https://help.example.com/content/reset-your-password

The article comes back as Markdown:

# Reset your password

_Category: Account_

If you forgot your password, set a new one from the sign-in page. It takes about a minute.

## Reset it from the sign-in page

1. On the sign-in page, click **Forgot password?**
2. Enter the email address you sign in with and click **Send reset link**.
3. Open the email and click the link in it. The link works for 60 minutes.
4. Choose a new password and click **Save**.

## No email arrived?

Check your spam folder, and make sure you typed the address you sign in with. You can ask for a new link at any time.

The response has these headers:

A page answers in Markdown only when text/markdown (or text/x-markdown) ranks above text/html in Accept:

• Articles start with # Title and a _Category: Name_ line, then the body converted from the article's public HTML. Scripts, styles and internal notes are left out.

• The home page lists the categories with their descriptions and article links, /categories is a nested list of categories, and a category page lists its articles and subcategories.

• The same rules as the page apply. A draft or private article answers 404 to an anonymous request, as its HTML page does. Reading an article as Markdown counts as a view of it, like a visit in a browser.

Download an article as Markdown

Readers can download an article from the Markdown option of its Export menu. The download is also a URL you can call:

curl -s -OJ https://help.example.com/content/reset-your-password/export/markdown

It answers with Content-Type: text/markdown; charset=UTF-8 and Content-Disposition: attachment; filename="reset-your-password_en.md", a name made of the slug and the language. The file holds the same Markdown as the article page. On a help center published in several languages, the address includes the language: /de/content/{slug}/export/markdown.

The address works only while the article page has an Export menu with the Markdown option on, and answers 404 otherwise. For a program, the Accept: text/markdown request above is the better choice: it works on every public page.

Turn them off

• Search engine indexing off (Settings, General card): robots.txt asks every crawler to stay out and sets all three signals to no, llms.txt answers 404, ai.txt states the opt-out, and the public MCP server turns off. Markdown pages keep answering, like the pages themselves.

• Allow AI assistants to connect off (Settings, Public MCP server card): only the public MCP server turns off, and the other files stop mentioning it.

• The template editor's no-index switch adds a noindex tag to the pages and changes none of the files above.

The steps are in How search engines and AI crawlers read your help center and Hide your help center from search engines.

Related

• The public MCP server of a help center

• The HelpCenter.io MCP server