Vaulthalla Logo

Dynamic Sitemap Helper

Add docs, static AI routes, and skill artifacts to a Next App Router sitemap.

Dynamic Sitemap Helper

Use getDocsForSitemap from the /next export when a Next App Router site has a dynamic src/app/sitemap.ts file.

The helper reads published docs sets, resolves group paths, includes the generated docs records inside each set, includes synced static assets by default, prepends siteUrl, merges optional static routes, and returns a ready-to-use MetadataRoute.Sitemap array.

1import type { MetadataRoute } from 'next'2 3import config from '@payload-config'4import { getPayload } from 'payload'5 6import { getDocsForSitemap } from '@valkyrianlabs/payload-markdown-docs/next'7 8const siteUrl = process.env.NEXT_PUBLIC_SITE_URL ?? 'https://example.com'9 10export default async function sitemap(): Promise<MetadataRoute.Sitemap> {11  const payload = await getPayload({ config })12  return getDocsForSitemap({13    payload,14    siteUrl,15  })16}

Combine With Other Routes

Most apps also include static routes, Pages collection routes, AI discovery files, or other dynamic content in the same sitemap. Use additionalRoutes for site-relative path entries or absolute url entries.

1import type { MetadataRoute } from 'next'2 3import config from '@payload-config'4import { getPayload } from 'payload'5 6import { getDocsForSitemap } from '@valkyrianlabs/payload-markdown-docs/next'7 8const siteUrl = process.env.NEXT_PUBLIC_SITE_URL ?? 'https://example.com'9 10export default async function sitemap(): Promise<MetadataRoute.Sitemap> {11  const payload = await getPayload({ config })12  const docs = await getDocsForSitemap({13    additionalRoutes: [14      { path: '/llms.txt' },15      { path: '/llms-full.txt' },16      { path: '/plugins/payload-markdown-docs/llms.txt' },17      { path: '/plugins/payload-markdown-docs/llms-full.txt' },18      { path: '/plugins/payload-markdown-docs/skills/codex' },19      { path: '/plugins/payload-markdown-docs/skills/codex/SKILL.md' },20      { path: '/plugins/payload-markdown-docs/skills/claude' },21      { path: '/plugins/payload-markdown-docs/skills/claude/SKILL.md' },22    ],23    payload,24    siteUrl,25  })26 27  return [28    {29      url: siteUrl,30    },31    ...docs,32  ]33}

The sitemap helper dedupes by final URL. When the same URL appears more than once, the newest lastModified value is kept. Output remains sorted by URL.

AI Discovery Routes

sitemap.xml and llms.txt serve different jobs:

  • sitemap.xml is crawler discovery.
  • llms.txt is an AI-readable entrypoint.
  • native skills are agent workflow artifacts.

When payload-markdown-docs push syncs assets, getDocsForSitemap includes stored /llms.txt, /llms-full.txt, and skill artifact routes by default. Set includeAssets: false only when the site wants to manage those entries manually. For static files that are not synced, keep using additionalRoutes.

Use getPayloadMarkdownDocsAiSitemapRoutes to build common AI/static routes:

1import type { MetadataRoute } from 'next'2 3import config from '@payload-config'4import {5  getDocsForSitemap,6  getPayloadMarkdownDocsAiSitemapRoutes,7} from '@valkyrianlabs/payload-markdown-docs/next'8import { getPayload } from 'payload'9 10const aiRoutes = getPayloadMarkdownDocsAiSitemapRoutes({11  includeLlmsFull: true,12  skills: [13    {14      basePath: '/plugins/payload-markdown-docs/skills',15      agents: ['codex', 'claude'],16      files: [17        'SKILL.md',18        'reference/payload-markdown-directives.md',19        'reference/formatting.md',20        'reference/frontmatter.md',21        'reference/workflow.md',22        'reference/sync.md',23        'reference/routing.md',24        'reference/admin.md',25        'reference/troubleshooting.md',26        'examples/docs-page.md',27        'examples/github-actions.md',28      ],29    },30  ],31})32 33export default async function sitemap(): Promise<MetadataRoute.Sitemap> {34  const payload = await getPayload({ config })35 36  return getDocsForSitemap({37    additionalRoutes: aiRoutes,38    payload,39    siteUrl,40  })41}

Skill artifacts can be hosted under plugin docs routes, such as /plugins/payload-markdown-docs/skills/codex/SKILL.md, or under a top-level route such as /skills/payload-markdown-docs/codex/SKILL.md. Set basePath to the public route your site owns.

Serving Synced Assets

The plugin registers Payload-owned GET endpoints for synced AI/static assets:

  • /llms.txt
  • /llms-full.txt
  • <docsSet.routeBase>/llms.txt
  • <docsSet.routeBase>/llms-full.txt
  • <docsSet.routeBase>/skills/<agent>/<path...>

For example, a docs set served at /plugins/payload-markdown-docs exposes /plugins/payload-markdown-docs/llms.txt, /plugins/payload-markdown-docs/skills/codex, and /plugins/payload-markdown-docs/skills/codex/SKILL.md; the Claude skill is available at /plugins/payload-markdown-docs/skills/claude and /plugins/payload-markdown-docs/skills/claude/SKILL.md after the assets are synced. Consuming apps should install the public Next route files that delegate to the package asset route handler:

1pnpm exec payload-markdown-docs install routes --payload-app "src/app/(payload)"

Use --payload-app "app/(payload)" for apps without src/. If the public routes return an asset schema error, migrate the Payload database so the payload-markdown-docs-assets collection table exists.

The /api/... asset URLs are implementation/internal fallback URLs. Public sitemap entries should use the canonical routes outside /api.

Cache Keys And Tags

The helper wraps its read in unstable_cache. Override cacheKey or tags when the app uses different sitemap invalidation tags.

1const docs = await getDocsForSitemap({2  cacheKey: ['sitemap-docs-v2'],3  payload,4  siteUrl,5  tags: ['sitemap', 'docs'],6})

Defaults:

  • cacheKey: sitemap-docs-v1
  • recursive: true
  • tags: sitemap, sitemap:docs

Recursive Docs

By default, docs set indexes and generated child docs are included in the sitemap. Disable recursion only when the app intentionally wants the base docs set URLs.

1const docs = await getDocsForSitemap({2  payload,3  recursive: false,4  siteUrl,5})

Custom Collection Slugs

If the plugin uses custom collection slugs, pass them through collections.

1const docs = await getDocsForSitemap({2  collections: {3    docs: 'knowledge-docs',4    docsGroups: 'knowledge-groups',5    docsSets: 'knowledge-sets',6  },7  payload,8  siteUrl,9})

Paginated Result

Use getPaginatedDocsForSitemap when the app needs the original Payload-style paginated result instead of the mapped Next sitemap array.

1import { getPaginatedDocsForSitemap } from '@valkyrianlabs/payload-markdown-docs/next'2 3const result = await getPaginatedDocsForSitemap({4  payload,5  siteUrl,6})7 8// result.docs: Array<{ url?: string | null; lastModified?: string | null }>