2.17Beginner9 min

XML Sitemaps: Best Practices for Large Websites

Lucas Blochberger·7 April 2026·Updated 11 June 2026

Definition

XML Sitemaps are machine-readable lists of all indexable URLs. In Next.js 15, the sitemap.ts file convention replaces third-party plugins and enables dynamic generation from CMS data.

Key Takeaways

✓Maximum 50,000 URLs or 50 MB uncompressed per sitemap file; beyond that, a sitemap index file is mandatory, which can reference up to 50,000 sitemaps.
✓Only update lastmod for genuine content changes; Google only uses it if it's demonstrably accurate. changefreq and priority are ignored.
✓Only include indexable, canonical URLs: no noindex, no 404, no 301 redirect, no canonicalized URLs.
✓Shard large sites by page type, language, or recency to enable per-segment indexing diagnostics in Search Console.
✓In Next.js, sitemap.ts generates the sitemap programmatically; generateSitemaps() splits large inventories into 50,000-block chunks, ISR keeps them current.
✓Crawl budget becomes relevant above 1 million URLs (weekly changes) or above 10,000 URLs (daily changes); the sitemap directs budget toward important URLs.
✓The key metric is the ratio of submitted to indexed URLs; IndexNow and AI crawler control complement the sitemap for real-time and GEO.

XML Sitemaps are a cornerstone of technical SEOespecially for large websites with dynamic content.

Limits and Format

Maximum 50,000 URLs or 50MB per sitemap file. For larger sites: sitemap index with references to individual sitemaps. Set lastmod only when content actually changes; automatic daily updates without changes dilute the signal.

Next.js Implementation

The sitemap.ts file convention in Next.js App Router replaces third-party plugins and enables dynamic generation from Payload CMS data with ISR revalidation.

Data & Statistics

Eine einzelne XML-Sitemap-Datei darf maximal 50.000 URLs enthalten und 50 MB (unkomprimiert) nicht berschreiten; grere Bestnde mssen aufgeteilt und ber eine Sitemap-Index-Datei zusammengefhrt werden.

Google Search Central - Build and Submit a Sitemap (2025)

Eine Sitemap-Index-Datei darf bis zu 50.000 loc-Tags enthalten, und pro Property lassen sich bis zu 500 Sitemap-Index-Dateien in der Search Console einreichen.

Google Search Central - Build and submit large sitemaps (2025)

Google verwendet den lastmod-Wert nur, wenn er konsistent und nachweislich korrekt ist, etwa durch Abgleich mit der tatschlichen letzten nderung der Seite.

Google Search Central - Build and Submit a Sitemap (2025)

Crawl-Budget-Management wird relevant ab groen Sites mit ber 1.000.000 Seiten (wchentliche nderungen) bzw. ab ber 10.000 Seiten mit tglich sehr schnell wechselndem Content.

Google Search Central - Large Site Owner's Guide to Managing Crawl Budget (2024)

Next.js generateSitemaps() teilt Sitemaps in 50.000er-Blcke auf (App Router); Googles Limit liegt bei 50.000 URLs pro Sitemap, die generierten Dateien sind unter /.../sitemap/[id].xml erreichbar.

Next.js Docs (Vercel) - generateSitemaps (2025)

2024 lieferten 83,9 Prozent der robots.txt-Dateien bei mobilen Abrufen einen 200-Status, 14,1 Prozent einen 404.

Web Almanac 2024 (HTTP Archive) - SEO Chapter (2024)

Der AhrefsBot wurde 2024 in 8,8 Prozent der mobilen robots.txt-Crawls erfasst, gegenber 5,3 Prozent im Jahr 2022.

Web Almanac 2024 (HTTP Archive) - SEO Chapter (2024)

ber IndexNow werden ber 3,5 Milliarden URLs pro Tag eingereicht; 18 Prozent aller neu in den Suchergebnissen geklickten URLs wurden ber IndexNow entdeckt.

Bing Webmaster Blog - IndexNow Expands Adoption Across Industries (2024)

Im Mai 2025 entfielen rund 50 Prozent des KI- und Such-Crawler-Traffics auf den Googlebot, 7,7 Prozent auf GPTBot und 5,4 Prozent auf ClaudeBot; der gesamte Crawler-Traffic wuchs um 18 Prozent gegenber dem Vorjahr.

Cloudflare Blog - From Googlebot to GPTBot: who's crawling your site in 2025 (2025)

9,6 Prozent der Sites nutzten hreflang im Desktop-Bereich im Jahr 2022, gegenber 9,0 Prozent im Jahr 2021.

Web Almanac 2022 (HTTP Archive) - SEO Chapter (2022)

In sterreich wickelten 2023 bereits 31 Prozent der Unternehmen Verkufe ber E-Commerce ab, gegenber 26 Prozent im Jahr 2022.

Statistik Austria - IKT-Einsatz in Unternehmen 2024 (2024)

FAQ

How many URLs can an XML sitemap contain?

A single sitemap file may contain a maximum of 50,000 URLs and must not exceed 50 MB uncompressed. If either limit is reached, you must split the sitemap into multiple files and consolidate them via a sitemap index file. An index file may itself reference up to 50,000 sitemaps, and up to 500 index files can be submitted per property in Search Console.

When is a sitemap index file necessary?

A sitemap index file is necessary as soon as your website has more than 50,000 URLs or a single sitemap would exceed the 50 MB limit. It consolidates multiple partial sitemaps so you only need to submit a single index file to Google. For large sites, additional sharding by page type, language, or recency is recommended to enable per-segment indexing diagnostics.

Should you set changefreq and priority in the sitemap?

No, changefreq and priority are largely ignored by Google today and can be omitted without disadvantage. Only lastmod is relevant, and even that only if the timestamp is consistent and demonstrably accurate. Set lastmod exclusively for genuine content changes, otherwise Google will devalue the signal and ignore it.

How do I create a sitemap in Next.js?

In the Next.js App Router, use the file convention sitemap.ts in the app directory. The function returns an array of objects with url and optionally lastModified, which Next.js renders into valid XML. For large sites, generateSitemaps() splits the inventory into 50,000-block chunks and serves the routes under /.../sitemap/[id].xml. With ISR and a revalidation interval, the sitemap stays current without recalculating on every request.

Which URLs don't belong in an XML sitemap?

Pages with a noindex tag, URLs blocked by robots.txt, 404 pages, redirected URLs with 301 redirects, and pages that point to another URL via canonical do not belong in the sitemap. The sitemap must exclusively contain indexable, canonical URLs with status 200. Any other URL sends conflicting signals and wastes crawl budget.

What does 'Discovered, currently not indexed' mean in Search Console?

This status means Google knows the URL from your sitemap but has deliberately not yet crawled it. On large sites, this often indicates crawl budget constraints or thin content. This is distinct from 'Crawled, currently not indexed', where the page was crawled but deemed not valuable enough for indexing.

Does IndexNow replace the XML sitemap?

No, IndexNow complements the sitemap but does not replace it. IndexNow is a push protocol that actively informs search engines about changed URLs and is supported by Bing and Yandex. For frequently updated large sites, it's an effective real-time channel. As of today, Google does not use IndexNow as a primary discovery signal, so the classic sitemap remains indispensable.

How does your website perform?

Get a free, AI-powered SEO report of your website by email: technical SEO, on-page, keywords & competitors. No obligation.

Get a free SEO audit →

Previous← Hreflang for DACH: AI Platforms Ignore Language Signals NextJavaScript SEO: Rendering and Indexing →