Online Sitemap Generator: Build a Sitemap and Clear Indexing Errors

An online sitemap generator builds an XML file that lists the URLs you want search engines to crawl, usually with optional metadata such as last modification dates or change frequency. You upload that file to the site root and point to it from robots.txt or submit it in Search Console. The generator saves you from writing XML by hand, but it only reflects the URLs you feed it.

In short:

• A sitemap is a hint for crawlers, not a command, so listing a URL does not guarantee indexing.
• One sitemap file holds up to 50,000 URLs and stays under 50 MB uncompressed; larger sites split URLs across files linked from a sitemap index.
• The file belongs at the domain root, for example /sitemap.xml, so crawlers find it without extra setup.
• When Search Console reports errors, inspect the individual URL before you touch the sitemap; fixing the page usually clears the error faster than resubmitting the file.
• A visualization exposes URLs you did not expect, which is often the first sign of filter parameters, staging subdomains or non-canonical pages leaking into the file.

What does an online sitemap generator actually produce?

It produces a single XML document in the sitemap protocol format: a list of URLs, each wrapped in a <url> element, with optional fields for last modification, change frequency and priority. The generator does not discover your site for you unless you give it a source, such as a crawl of your domain or an export of URLs from your CMS.

The practical difference between generators comes down to three things: where the URL list comes from, whether the output respects the protocol limits, and whether it validates the XML before you download it. A tool that pulls URLs from a crawl will include whatever it can reach, including pages you may not want listed. A tool that accepts an uploaded list gives you control but shifts the work of filtering back to you.

Treat the generated file as a draft. The URLs that end up in it should be canonical, indexable and returning a normal status. Everything else adds noise to crawl reports and makes the file harder to read when something goes wrong.

How do you generate a sitemap step by step?

Start from a clean URL list, not from a blind crawl. The order below keeps the file small and the errors easy to trace.

1. Decide which URLs belong. Include canonical, indexable pages that return a normal response. Exclude parameter duplicates, staging hosts, internal search results and anything blocked in robots.txt.
2. Pick your source. Either crawl the site, export URLs from the CMS, or paste a list. Compare the count against what you expect; a large gap usually means filters or pagination crept in.
3. Run the online sitemap generator and set the optional fields. Last modification dates are worth filling from real data. Change frequency and priority are hints that most crawlers treat loosely, so do not spend time tuning them.
4. Validate the output. Check the XML parses, the URL count is under the protocol limit, and the file size is under 50 MB uncompressed.
5. Upload to the domain root and reference it. Place the file at /sitemap.xml, add a line to robots.txt, and submit it in Search Console.
6. Record what you submitted. Keep the URL list and the date so the next comparison shows what changed.

If your site is large, split the list before step 3. Several child sitemaps linked from one index file keep each file inside the limits and make it obvious which section broke when an error appears.

Where should the sitemap file live and how do search engines find it?

Place the file at the root of the domain, for example /sitemap.xml, so search engines can find it without extra configuration. If you use a sitemap index, keep the index at the root and let it point to the child files. You can also declare the location in robots.txt or submit it directly in Search Console.

The root rule matters because crawlers check the default location first. A file buried in a subfolder will not be picked up automatically, and you would have to declare it every time. Keep one canonical location and do not scatter copies across the site.

Two discovery routes work together rather than replacing each other. The robots.txt line covers crawlers that look there, and the Search Console submission gives you a report on what happened after the file was read. Submitting in Search Console does not make the robots.txt line redundant, and the reverse is also true.

What are the protocol limits for a sitemap file?

A sitemap file can list up to 50,000 URLs and stay under 50 MB uncompressed, per the sitemap protocol. Larger sites split URLs across several sitemap files and link them from a sitemap index. Both limits come from the protocol specification, so a generator that ignores them produces a file search engines may reject.

Compression is allowed by the protocol and does not change the URL count limit, only the transfer size. If you are near the size ceiling, splitting is the safer move because it also isolates failures: one broken child file does not invalidate the rest of the index.

How do you read a sitemap visualization?

A sitemap visualization turns the flat XML list into a structure you can scan: URL groups, depth, counts per section, and often a tree that mirrors the site hierarchy. Its value is not decoration. It shows you what the file actually contains, which is frequently different from what you assumed.

Look for these signals first:

• Sections with far more URLs than the site has pages, which usually means parameters or pagination.
• Hosts other than your main domain, such as staging or a www/non-www duplicate.
• Orphan clusters: URLs in the sitemap with no internal links pointing to them.
• Pages you know are marked noindex or blocked, which should not be listed at all.

A visualization is also a fast way to spot a truncated file. If the tree stops mid-section, the generator hit a limit or the crawl was cut short, and the uploaded sitemap is incomplete.

Why does Search Console report indexing errors for URLs in my sitemap?

A sitemap is a hint, not a command, so Google can skip or postpone URLs for many reasons: the page returns an error, is blocked in robots.txt, carries a noindex tag, or duplicates another URL. Check the URL Inspection tool for the specific page before changing the sitemap. Fixing the page itself usually clears the error faster than resubmitting the file.

The error report describes what the crawler saw, not what you intended. That distinction decides where you spend the fix. A server error is a hosting or application problem. A "blocked by robots.txt" status is a rules problem. A "duplicate, Google chose different canonical" status is a signal problem, and editing the sitemap will not resolve it.

Work in this order when an error appears:

1. Open the specific URL in the inspection tool and read the reported status.
2. Fetch the page as the crawler would and confirm the response code and the meta robots tag.
3. Compare the canonical tag against the URL you listed in the sitemap.
4. Fix the page, then request a re-crawl of that URL.
5. Resubmit the sitemap only after the page-level fix is live.

Resubmitting a sitemap repeatedly without changing the page achieves nothing. The file was read; the URL was rejected on its own merits.

What should you check when a visualization shows unexpected pages?

Unexpected URLs usually come from filter parameters, pagination, staging subdomains, or pages linked internally but marked noindex. Open a few of them and compare against your canonical tags and robots rules. Decide whether each one belongs in the sitemap at all, since listing non-canonical or blocked URLs adds noise to crawl reports.

Sort the unexpected URLs into three buckets before you edit anything. Pages that should be indexed but are missing from the file need adding. Pages that exist but should never be indexed need removing and, if they are reachable, a canonical or robots rule to match. Pages that are duplicates of a canonical URL need consolidating rather than listing.

This is the same discipline that keeps keyword and content work honest: decide what the page is for before you promote it. If you are mapping queries to pages at the same time, the guide to keyword research tools and methods that work covers how to keep that mapping from drifting, and Google Keyword Planner: free and paid keyword tools explains where free data ends and paid data begins.

How do you keep a sitemap accurate over time?

Accuracy comes from regeneration on a schedule, not from one careful build. Sites change: pages get added, redirected, unpublished and merged. A sitemap generated once and forgotten slowly turns into a list of dead and duplicate URLs, and the crawl reports fill with errors that describe the past.

Set a cadence that matches how often the site changes, and automate the generation where the CMS allows it. Then compare each new file against the previous one: URLs added, URLs removed, URLs whose status changed. That diff is more useful than the file itself, because it shows what moved.

Two habits prevent most problems. First, exclude by rule rather than by hand: filter parameters, staging hosts and internal search results should be blocked at the source. Second, keep the sitemap and the canonical tags consistent. When they disagree, crawlers follow the canonical tag, and the sitemap becomes a record of your intentions rather than your site.

The same pattern applies to the rest of technical SEO work, where repeated manual checks get replaced by scheduled ones. SEO automation: how AI speeds up research and analysis covers which of those checks are worth automating, and rank tracking: which queries to keep under control explains how to choose the queries you monitor after the technical layer is clean.

How does sitemap work connect to content and reporting?

A clean sitemap makes everything downstream easier to measure. When the URL list matches the pages you actually want indexed, crawl reports, rank tracking and content performance all refer to the same set of pages. When it does not, you end up reconciling three different lists.

For content teams, the sitemap is also a publishing checklist item. New pages should appear in the next generation, and retired pages should disappear. If you write with AI assistance, the piece on SEO copywriting with AI: how to write content that ranks covers how to keep generated drafts aligned with the pages you intend to index.

On the reporting side, the pages that matter are the ones that return a normal status and carry a self-referencing canonical. Tools that track content and SEO signals, such as the content and SEO workspace, help keep the content plan, positions and page-level checks in one place so the sitemap does not drift away from what you publish.

What is the next step?

Generate the file from a filtered URL list, validate it against the protocol limits, place it at the domain root, and submit it once. Then open the visualization and compare what is listed against what you expected. That comparison tells you more about your site than the error report will, and it is the fastest way to find the pages that should never have been in the file.