Add lastmod to sitemap

[jdevalk]

🐛 Bug Report

The XML sitemaps currently output loc, changefreq and priority for every url set. I would propose dropping the changefreq and priority fields, as none of the search engines use these, and instead adding the lastmod field, with the last modification date of the file.

Have you read the Contributing Guidelines on issues?

Yes.

To Reproduce

(Write your steps here:)

1. Open any DocuSaurus v2 sitemap :)

Expected behavior

The current output would be:

<url>
        <loc>https://developer.yoast.com/features/canonical-urls/api</loc>
        <changefreq>weekly</changefreq>
        <priority>0.5</priority>
</url>

(Write what you thought would happen.)

Actual Behavior

I propose changing it to:

<url>
        <loc>https://developer.yoast.com/features/canonical-urls/api</loc>
        <lastmod>2020-04-14T11:22:05+00:00</lastmod>
</url>

Your Environment

• Docusaurus version used: v2

[RDIL]

I think this would be a good addition, but I do know web crawlers that use the priority field.

[jdevalk]

@RDIL such as? Honestly I’ve been doing SEO for well over a decade, not seen it used in the last 5 years.

[RDIL]

Fair enough.

[yangshun]

Great idea! Thanks for the suggestion!

[AvroraPolnareff]

Hello! I want to help solve this issue.
As I can see, there are several implementation options here:

1. Should I leave the old tags and add new ones or replace them?
2. Which date should be specified in the "lastmod" tag: the date of the last build of the project or the date of the last page change? If the second, are there any easier ways to do it?

[RDIL]

Most likely the last build time since even just tiny changes end up changing the chunk hashes, so its constantly being modified.

[slorber]

@RDIL FYI Webpack 5 might help to make the js chunks more "stable" (see my recent comment in #3383), we may try to migrate after i18n is ready.

Not sure what we should do for this date. Also not sure how the sitemaps plugin could access the "last modification date" of the page, as this plugin is decoupled from the others.

Is it mandatory to add it to the sitemaps? It could likely be easier to handle this by adding a meta directly on the page, otherwise, we'd have to find a way to provide such metadata per path to the sitemap plugin.

Asking this, because for my work on i18n I'll also have to think about how to set up useful headers for localization (hreflang), and thought about adding them to the page directly instead of the sitemaps.

@jdevalk as it seems you know more about SEO than the rest of us, can you give us some insights?

[jdevalk]

Last modified is somewhat of a must for XML sitemaps indeed.

I think for hreflang I'd go for adding it to the page instead of the XML sitemaps as that makes debugging a lot easier and maybe even makes it accessible to other features within docusaurus, like a language switcher.