# llms.txt, ai.txt and Markdown pages

_Category: AI agents and MCP_

Every help center publishes what AI tools and crawlers need to find and read it: an `llms.txt` index, an `ai.txt` with its AI usage preferences, content signals in `robots.txt`, an API catalog, `Link` headers, and a Markdown version of every page. There is nothing to set up. This page describes each one for developers who build agents, crawlers or integrations.

Owners switch these off in Settings. See [How search engines and AI crawlers read your help center](https://self.helpcenter.io/content/llms-txt-and-ai-crawlers).

## At a glance

| Address | What it is | When it answers |
| --- | --- | --- |
| `/llms.txt` | A Markdown index of the help center's categories and articles, in its default language | Only while **Search engine indexing** is on; `404` otherwise |
| `/ai.txt` | The help center's AI usage preferences | Always. With indexing off, it states the opt-out. |
| `/robots.txt` | Crawler rules and content signals | Always |
| `/.well-known/api-catalog` | A linkset (RFC 9727) of the help center's machine-readable resources | Always |
| Any page, with `Accept: text/markdown` | The page as Markdown, at the same URL | Home page, `/categories`, category and article pages |
| `/content/{slug}/export/markdown` | An article as a Markdown file download | Only when the article page's Export menu offers Markdown |
| `/mcp` | The read-only MCP server | See [The public MCP server of a help center](https://developers.helpcenter.io/content/public-mcp-server) |

They all work on the help center's own address, including its custom domain. On a private, password-protected or IP-restricted help center, an anonymous request is sent to sign in or to the password page, like any page.

## llms.txt

```
curl -s https://help.example.com/llms.txt
```

The response is plain text (`text/plain; charset=utf-8`) in the llms.txt format:

```
# Acme Help Center

> Answers about your Acme account, billing and invoices.

Every page listed below also serves clean Markdown at the same URL when
requested with `Accept: text/markdown`.

AI assistants can also search and read this help center live through its
read-only MCP server at https://help.example.com/mcp (Streamable HTTP,
no authentication; tools: `search`, `fetch`, `list_categories`).

## Account

Sign-in, passwords and your profile.

- [Reset your password](https://help.example.com/content/reset-your-password): Set a new password from the sign-in page when you cannot remember yours.
- [Close your account](https://help.example.com/content/close-your-account)
- [Change your email address](https://help.example.com/content/change-your-email-address)

## Billing

Plans, invoices and payment methods.

- [Change your plan](https://help.example.com/content/change-your-plan)

### Invoices

Find, download and share your invoices.

- [Download an invoice](https://help.example.com/content/download-an-invoice): Download the PDF invoice for any payment from the Billing page.

## Optional

- [Acme Help Center](https://help.example.com): Help center home
- [All categories](https://help.example.com/categories): Every category in this help center
- [Sitemap](https://help.example.com/sitemap.xml): XML sitemap of every public page
```

- **Title and summary.** The heading is the help center's default page title and the quote its description, from its SEO settings.
- **The MCP paragraph** appears only while the help center has a public MCP server.
- **One section per category**, nested as the categories are (`##` for top-level categories, `###` for their subcategories, and so on), each with its description and its articles. An article's link is followed by its SEO description when it has one. Articles in no category are listed under `## Other articles`.
- **The default language only.** Other languages are reachable as Markdown pages and through the public MCP server.
- **What anyone can read, nothing more.** Only published articles that an anonymous visitor can open: no drafts, private or team-only articles and categories, link-only articles, unreleased translations, or categories filed under a hidden parent.

An anonymous visitor's copy is sent with `Cache-Control: public, max-age=3600`. A team member who is signed in sees their team's content too, so their copy is sent with `private, no-store`. With **Search engine indexing** off, `/llms.txt` answers `404`. There is no `/llms-full.txt`: that address redirects to the home page.

## ai.txt

```
curl -s https://help.example.com/ai.txt
```

It states that AI assistants may read and cite the help center and that training models on it is not permitted, and points to llms.txt and the public MCP server:

```
# ai.txt for help.example.com
# AI content-usage preferences for this help center.
# Machine-readable index of this help center: https://help.example.com/llms.txt
# Read-only MCP server for AI assistants (no authentication): https://help.example.com/mcp
# Content signals — https://contentsignals.org
Content-Signal: search=yes, ai-input=yes, ai-train=no

# This help center's content belongs to its publisher. AI assistants are
# welcome to crawl these pages and cite them when answering questions about
# this product. Training models on them is not permitted.

User-agent: *
Allow: /

Sitemap: https://help.example.com/sitemap.xml
```

The MCP line appears only while the help center has a public MCP server. With **Search engine indexing** off, the file says so and asks every crawler to stay out:

```
# ai.txt for help.example.com
# AI content-usage preferences for this help center.
# This help center has opted out of search engines and AI crawlers.
Content-Signal: search=no, ai-input=no, ai-train=no

User-agent: *
Disallow: /
```

Both are cached like llms.txt.

## robots.txt

A help center's `robots.txt` has one set of rules for every crawler (`User-agent: *`). With indexing on, its directives are (comment lines left out):

```
User-agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=no
Disallow: /article/
Disallow: /_hc/
Sitemap: https://help.example.com/sitemap.xml
```

`/article/` and `/_hc/` are not pages: they receive comments and ratings and issue form tokens. With **Search engine indexing** off, its directives become:

```
User-agent: *
Content-Signal: search=no, ai-input=no, ai-train=no
Disallow: /
```

The `Content-Signal` line says whether search engines may index the pages (`search`), whether AI assistants may read them to answer questions (`ai-input`), and whether they may be used to train AI models (`ai-train`, always `no`). Like the rest of `robots.txt`, these are instructions that crawlers choose to follow.

## API catalog

```
curl -s https://help.example.com/.well-known/api-catalog
```

The catalog is a linkset (`application/linkset+json; profile="https://www.rfc-editor.org/info/rfc9727"`) with two contexts: the help center, and its public MCP server.

```
{
  "linkset": [
    {
      "anchor": "https://help.example.com/",
      "service-doc": [
        { "href": "https://help.example.com/", "type": "text/html", "title": "Acme Help Center" }
      ],
      "service-desc": [
        { "href": "https://help.example.com/sitemap.xml", "type": "application/xml", "title": "Sitemap" },
        { "href": "https://help.example.com/llms.txt", "type": "text/plain", "title": "llms.txt" }
      ]
    },
    {
      "anchor": "https://help.example.com/mcp",
      "service-desc": [
        { "href": "https://help.example.com/mcp/server-card", "type": "application/mcp-server-card+json", "title": "MCP server card" }
      ],
      "service-doc": [
        { "href": "https://help.example.com/mcp", "type": "text/html", "title": "Connect an AI assistant" }
      ]
    }
  ]
}
```

Without indexing, the `llms.txt` entry is left out; without a public MCP server, the second context is. The catalog is cached like llms.txt.

## Link headers

Every page of the help center, as HTML or Markdown, points agents to the catalog and the sitemap with two `Link` headers:

```
Link: <https://help.example.com/.well-known/api-catalog>; rel="api-catalog"
Link: <https://help.example.com/sitemap.xml>; rel="describedby"; type="application/xml"
```

## Markdown for any page

Ask for Markdown with the `Accept` header and a page answers in Markdown at the same URL. It works on the home page, `/categories`, category pages and article pages. There are no separate `.md` addresses.

```
curl -s -H "Accept: text/markdown" https://help.example.com/content/reset-your-password
```

The article comes back as Markdown:

```
# Reset your password

_Category: Account_

If you forgot your password, set a new one from the sign-in page. It takes about a minute.

## Reset it from the sign-in page

1. On the sign-in page, click **Forgot password?**
2. Enter the email address you sign in with and click **Send reset link**.
3. Open the email and click the link in it. The link works for 60 minutes.
4. Choose a new password and click **Save**.

## No email arrived?

Check your spam folder, and make sure you typed the address you sign in with. You can ask for a new link at any time.
```

The response has these headers:

| Header | Value |
| --- | --- |
| `Content-Type` | `text/markdown; charset=utf-8` |
| `X-Markdown-Tokens` | An estimate of the tokens in the Markdown: its length in characters divided by 4 |
| `Vary` | `Accept`, so caches keep the HTML and Markdown versions apart |

A page answers in Markdown only when `text/markdown` (or `text/x-markdown`) ranks above `text/html` in `Accept`:

| `Accept` | Answer |
| --- | --- |
| `text/markdown` | Markdown |
| `text/x-markdown` | Markdown |
| `text/markdown, text/html;q=0.9` | Markdown |
| `text/html;q=0.5, text/markdown;q=0.8` | Markdown |
| `text/html, text/markdown` | HTML |
| `*/*`, `text/plain` or no header | HTML |

- **Articles** start with `# Title` and a `_Category: Name_` line, then the body converted from the article's public HTML. Scripts, styles and internal notes are left out.
- **The home page** lists the categories with their descriptions and article links, **`/categories`** is a nested list of categories, and a **category page** lists its articles and subcategories.
- **The same rules as the page apply.** A draft or private article answers `404` to an anonymous request, as its HTML page does. Reading an article as Markdown counts as a view of it, like a visit in a browser.

## Download an article as Markdown

Readers can download an article from the Markdown option of its **Export** menu. The download is also a URL you can call:

```
curl -s -OJ https://help.example.com/content/reset-your-password/export/markdown
```

It answers with `Content-Type: text/markdown; charset=UTF-8` and `Content-Disposition: attachment; filename="reset-your-password_en.md"`, a name made of the slug and the language. The file holds the same Markdown as the article page. On a help center published in several languages, the address includes the language: `/de/content/{slug}/export/markdown`.

The address works only while the article page has an Export menu with the Markdown option on, and answers `404` otherwise. For a program, the `Accept: text/markdown` request above is the better choice: it works on every public page.

## Turn them off

- **Search engine indexing** off (Settings, **General** card): `robots.txt` asks every crawler to stay out and sets all three signals to `no`, `llms.txt` answers `404`, `ai.txt` states the opt-out, and the public MCP server turns off. Markdown pages keep answering, like the pages themselves.
- **Allow AI assistants to connect** off (Settings, **Public MCP server** card): only the public MCP server turns off, and the other files stop mentioning it.
- The template editor's no-index switch adds a `noindex` tag to the pages and changes none of the files above.

The steps are in [How search engines and AI crawlers read your help center](https://self.helpcenter.io/content/llms-txt-and-ai-crawlers) and [Hide your help center from search engines](https://self.helpcenter.io/content/hide-from-search-engines).

## Related

- [The public MCP server of a help center](https://developers.helpcenter.io/content/public-mcp-server)
- [The HelpCenter.io MCP server](https://developers.helpcenter.io/content/the-mcp-connector-for-ai-agents)