No sitemap

What a sitemap does for a new site

Search engines discover pages by following links, starting from pages they already know. A brand-new domain has few links pointing at it, and pages that sit deep in a site or are linked only from a menu that scripts build may take a long time to be found. An XML sitemap lists the URLs directly, often with the date each one last changed, so a crawler can find every page on its first visit and notice updates later.

A sitemap does not force anything into the index and does not raise rankings by itself. It removes one excuse for pages being missed, and it is what you submit in Google Search Console and Bing Webmaster Tools to see how many listed pages are actually indexed, which is the quickest way to spot other launch problems such as a leftover noindex.

How Skewcheck detects it

Skewcheck already fetches /robots.txt for the robots check. If that file has a Sitemap: line with a full http or https address, the check passes and names it, without fetching anything else. Otherwise it requests /sitemap.xml on the audited host with HEAD, or a GET whose body is never downloaded if HEAD answers with an error, through the same safety checks as the page fetch and following at most three redirects. A 200 answer with an XML content type passes with a note that the sitemap is not listed in robots.txt. A 200 answer that is an HTML page is a warning: single-page apps often answer every path, including /sitemap.xml, with their home page.

If neither is found, the check is a low-severity warning worth 2 points. Small sites can be crawled perfectly well without a sitemap, so it never weighs heavily. If the request has not finished when the audit's time budget runs out, the check passes as not checked and deducts nothing. Skewcheck confirms that a sitemap exists; it does not read the file or validate the URLs inside it, and a sitemap at another path counts only when robots.txt names it.

curl -s https://example.com/robots.txt | grep -i '^sitemap:'
curl -sI https://example.com/sitemap.xml | head -1

Publishing one

Most platforms generate a sitemap already; the work is making sure it is on the live domain and easy to find. WordPress core serves one at /wp-sitemap.xml, and SEO plugins such as Yoast replace it with their own index. Webflow and Framer can generate one from their SEO settings. In the Next.js App Router, an app/sitemap.ts file that returns the list of URLs is served at /sitemap.xml. Whatever produces it, the URLs inside should be the live https:// addresses, never staging.

Then add a Sitemap: line with the full URL to robots.txt, which every major crawler reads, and submit the same address in Google Search Console once the domain is verified. Keep pages you have marked noindex out of the sitemap, because listing a page while asking for it not to be indexed sends mixed signals.

import type { MetadataRoute } from "next";

export default function sitemap(): MetadataRoute.Sitemap {
  return [
    { url: "https://example.com/", lastModified: new Date() },
    { url: "https://example.com/about", lastModified: new Date() },
  ];
}

Before and after

$ curl -sI https://example.com/sitemap.xml | head -1
HTTP/2 404

$ curl -sI https://example.com/sitemap.xml | head -1
HTTP/2 200