How to Validate Your llms.txt File

To validate an llms.txt file, check four things: it opens with an H1, every link resolves to the page its text describes, every package name and version exists on the registry, and nothing in it contradicts your current docs.

A docs tool probably generated your llms.txt during a launch sprint, and nobody has opened it since. Pages have been renamed and packages deprecated in the meantime, and coding agents keep reading the file anyway.

If you need a refresher on the format, start with our llms.txt explainer. This article skips the basics. It walks through the checks, gives you a zero-dependency Node script, and shows how to run it in CI so problems are caught before an agent acts on them.

Why Is a Stale llms.txt a Supply-Chain Problem?

A stale llms.txt is a supply-chain problem because agents treat llms.txt as trusted reference material and often act on it with no human in between. That turns a stale link or a wrong package name into an instruction. A human reader who hits a dead link gives up. An agent with shell access may run npm install on a name that someone else now owns.

The llms.txt supply-chain risk has been demonstrated. TechRadar reported research by Alon Hertz, who scanned 6,214 live domains and found 8,265 llms.txt and llms-full.txt files. By his count, the files on 120 separate sites pointed to packages or domains that nobody had registered. He then claimed some of those free names and published packages that reported back to him whenever someone installed them. According to the researcher, one Fortune 500 company’s systems started calling in within an hour. The report names Claude, Codex and Hermes as agents involved.

Keep this in proportion. It was a controlled experiment, not a record of malware in the wild. It is still a real supply-chain exposure, and a CI job closes it cheaply.

Access control is a separate concern. Blocking AI crawlers with robots.txt tells crawlers which parts of your site they may read. Validating llms.txt controls whether what they read is still correct.

What to Check When You Validate llms.txt

A valid llms.txt has an H1 and correctly formatted links, and every link and package reference points to something that exists and matches its description. The llms.txt proposal does not publish an official validator. The checks below cover what online checkers test, plus registry checks for the packages the file names.

Links Resolve

Every URL in llms.txt should return a 2xx status once redirects are followed. Treat 4xx, 5xx and DNS failures as errors. A domain that no longer resolves is the domain version of an unregistered package: anyone can buy it. Log redirects as well. A redirect that lands on the site root usually means the page was removed. Send HEAD first and fall back to GET, because some servers reject or mishandle HEAD.

Links Return What They Claim

A 200 response proves a URL answers. It does not prove the page still says what the link text promises. Check that the content-type is HTML or Markdown as expected. Compare the page’s <title> or first heading with the link text, and look out for soft 404s, where an error page is served with a 200 status. This check is fuzzy, so run it as a periodic review rather than a build gate.

Packages and Versions Exist on the Registry

Check registry existence for every package your llms.txt tells an agent to install. By hand, npm view <name> shows what the registry holds for that package and errors out if there is no such package. In a script, fetch the package document from registry.npmjs.org. A name the registry doesn’t know is a name someone else can claim. For pinned versions, confirm the exact version appears in the package’s published version list. PyPI, crates.io and other ecosystems offer equivalent lookups.

Markdown Structure Is Valid

The original llms.txt proposal makes only one section mandatory: an H1 naming the project or site. Apart from an optional byte-order mark, the file must open with # Name. The current llms.txt spec says each H2 section holds a Markdown list, and every item in it must start with a [name](url) link. Notes after a colon are optional. Fail the build on a missing H1 or broken link syntax. Don’t fail it for a missing blockquote or H2. Some online checkers flag both anyway, so don’t treat those warnings as spec violations.

The File Doesn’t Contradict Current Docs

Compare the set of URLs in llms.txt with your sitemap.xml or your docs build’s route manifest. Treat any URL that appears in llms.txt but not in the sitemap as suspect. Do the same for packages: every name in the file should match the name field of a package.json you publish, or a dependency you intend to recommend.

A Zero-Dependency Node Script to Validate llms.txt

This script takes a URL or a local path. It checks the H1 and every absolute link, then looks up each package named in an install command on the npm registry. It uses Node’s global fetch, so it runs without any dependencies on a current LTS such as Node 24.

// scripts/check-llms.mjs — usage: node scripts/check-llms.mjs <url-or-path>
import { readFile } from 'node:fs/promises';

const src = process.argv[2];
if (!src) { console.error('usage: node check-llms.mjs <url-or-path>'); process.exit(2); }
const failures = [];

async function load(s) {
  if (!/^https?:\/\//.test(s)) return readFile(s, 'utf8');
  const res = await fetch(s);
  if (!res.ok) { console.error(`✗ ${s} returned ${res.status}`); process.exit(1); }
  return res.text();
}

async function checkLink(url) {
  let res;
  try {
    res = await fetch(url, { method: 'HEAD', redirect: 'follow' });
  } catch {
    // some servers reset the connection on HEAD; fall through to GET
  }
  try {
    if (!res?.ok) res = await fetch(url, { redirect: 'follow' }); // some servers refuse HEAD
    if (res.status >= 400) failures.push(`link ${res.status}: ${url}`);
    else if (res.redirected) console.warn(`↪ ${url} -> ${res.url}`);
  } catch (err) {
    failures.push(`link unreachable (${err.cause?.code ?? err.message}): ${url}`);
  }
}

async function checkPackage(spec) {
  const [, name, version] = spec.match(/^(@?[^@\s]+)(?:@(\S+))?$/) ?? [];
  if (!name) return;
  const res = await fetch(`https://registry.npmjs.org/${name.replace('/', '%2F')}`);
  if (res.status === 404) return failures.push(`npm: "${name}" is not registered`);
  if (!res.ok) return failures.push(`npm: registry returned ${res.status} for "${name}"`);
  const doc = await res.json();
  if (/^\d+\.\d+\.\d+/.test(version ?? '') && !doc.versions?.[version])
    failures.push(`npm: ${name}@${version} does not exist`);
}

const text = await load(src);
if (!/^# \S/.test(text.trimStart())) failures.push('structure: file does not start with an H1');

const links = new Set([...text.matchAll(/\]\((https?:\/\/[^)\s]+)\)/g)].map((m) => m[1]));
const specs = new Set([...text.matchAll(/\b(npx|npm (?:install|i|add)|pnpm add|yarn add)\s+([^\n`]+)/g)]
  .flatMap(([, cmd, args]) => {
    const tokens = args.trim().split(/\s+/).filter((t) => !t.startsWith('-'));
    return cmd === 'npx' ? tokens.slice(0, 1) : tokens;
  }));

for (const url of links) await checkLink(url);
for (const spec of specs) await checkPackage(spec);

failures.forEach((f) => console.error(`✗ ${f}`));
console.log(`${links.size} links, ${specs.size} package refs, ${failures.length} failures`);
process.exit(failures.length ? 1 : 0);

How it works:

• load reads the file from disk or over HTTP. If the llms.txt URL itself fails, the script stops immediately.
• checkLink tries HEAD first and retries with GET if the HEAD request errors out or returns a non-OK status. Redirects are logged as warnings, and network-level failures such as an unresolvable domain are recorded as failures.
• checkPackage looks up each name on the npm registry and flags names that don’t exist, plus exact versions that were never published.
• Requests run one at a time on purpose, to keep load on your servers and the registry low.
• Any failure makes the script exit with code 1.

It has known limits. Relative links are skipped. Prose that follows an install command on the same line can produce false positives. Version ranges are not checked, only exact versions.

A failing run looks something like this (illustrative output with made-up names):

↪ https://docs.example.com/guide -> https://docs.example.com/
✗ link unreachable (ENOTFOUND): https://docs.old.example.com/api
✗ npm: "example-sdk-legacy" is not registered
42 links, 3 package refs, 2 failures

Run the Check in CI on Every Deploy and Every Week

Run the llms.txt validator twice over: once on every deploy against the file you just built, and once on a schedule against the live file. The deploy run catches mistakes you introduce. The scheduled run catches link rot and packages unpublished upstream while your repo stays untouched.

name: Validate llms.txt
on:
  push:
    branches: [main]
  schedule:
    - cron: '0 6 * * 1'   # Mondays 06:00: catches link rot and unpublished packages
  workflow_dispatch:

jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version: 24
      - run: node scripts/check-llms.mjs https://docs.example.com/llms.txt

The schedule trigger takes a standard five-field cron string, and GitHub reads it as UTC unless you add a timezone key with an IANA zone name. workflow_dispatch lets you trigger a run by hand. If your docs deploy lives in the same workflow, add a step before the publish step that points at the build output, for example node scripts/check-llms.mjs dist/llms.txt, so a failing check blocks the release. Leave the scheduled job pointed at the live URL.

Keep llms.txt in Sync by Generating It

If you generate llms.txt from the same source as your docs, the file can’t list a page or package the docs no longer contain. Build it in the same step that renders the docs, from the same page list and frontmatter:

// scripts/build-llms.mjs
import { writeFile } from 'node:fs/promises';
import pages from '../dist/pages.json' with { type: 'json' };

const lines = ['# Example SDK', '', '## Docs', ''];
for (const p of pages.filter((p) => !p.draft)) {
  lines.push(`- [${p.title}](https://docs.example.com${p.path}): ${p.description}`);
}
await writeFile('dist/llms.txt', lines.join('\n') + '\n');

Never hand-edit the generated file. Either commit only the generator, or commit both the generator and its output and have CI fail when regenerating produces a diff. Once generation is in place, the validator acts as a guard on the generator instead of a cleanup job someone has to remember.

Conclusion

An llms.txt file only helps if everything it lists still exists and still matches its description, because agents act on it without checking. Add scripts/check-llms.mjs to your repo, run it once against your live file, fix whatever it reports, and then commit the workflow so the next broken link or orphaned package name fails a build before an agent finds it.

FAQs

Should llms-full.txt be validated the same way as llms.txt?

Yes for links and packages, and more strictly for packages. llms-full.txt inlines full documentation content, so it usually holds far more install commands and code samples than llms.txt, and every one is a package name an agent may run. Structure checks can be looser, because the file is mostly page content rather than a short list of links. Point the same script at both files.

Why does the link check report 403 errors for pages that load fine in a browser?

Bot protection or a web application firewall is probably blocking the automated request. Some CDNs and security layers reject requests that lack browser-like headers or come from CI runner IP ranges. Allowlist your CI runner or the checker's User-Agent for your own docs domain, or send a descriptive User-Agent header with fetch. Don't simply ignore 403s, since agents may be blocked the same way.

How do I check Python package names from llms.txt against PyPI?

Query the PyPI JSON API at pypi.org/pypi/NAME/json. For an existing project it returns metadata plus a releases object keyed by version string, so you can confirm both the name and any pinned version. If the request returns a 404, treat the name as unregistered. Extract names from pip install and uv add commands the same way the script handles npm install.

Does a passing registry check mean a package in llms.txt is safe to install?

No. The check confirms that the name exists, not that the right people own it. Someone may already have claimed a name after the original package was abandoned, and a typo in llms.txt can point at a look-alike package that does exist. For every package you recommend, check that the publisher and repository link match your organisation, and prefer scoped names you control.