NEWS file using h2/h3 hierarchy fails to render well

[MichaelChirico]

See discussion: tidyverse/style#245.

Currently, {pkgdown} fails to recognize this style of NEWS:

[jayhesselberth]

pkgdown can use h2/h3 news files. The reason the data.table news file doesn't render is that it doesn't conform to the suggested pkgdown header forms (the top bulleted list on this page):

https://pkgdown.r-lib.org/reference/build_news.html

[MichaelChirico]

@MLopez-Ibanez can you confirm the page previously rendered as expected, and only stopped working when we changed header styles?

[MLopez-Ibanez]

@MLopez-Ibanez can you confirm the page previously rendered as expected, and only stopped working when we changed header styles?

I only noticed it because I was thinking about implementing this for my package and I wanted to see how it looked.

[jayhesselberth]

This is package-specific issue so I'm closing for now.

[MichaelChirico]

Hi @jayhesselberth I think this was closed in error. Rendering the NEWS fails long before reaching any issue about header styling.

data_news() assumes it will find the sections at ./body/div, which essentially restricts it to finding <h1> tags. But as noted, {data.table} is using <h2> as the primary heading level, which are found at ./body/div/div:

pkgdown/R/build-news.R

Line 148 in 824ab4a

sections <- xml2::xml_find_all(xml, "./body/div")

c.f.

# in the data.table TLD
library(xml2)
library(pkgdown)

# equivalent to the 'xml' object in the body of data_news()
xml <- as_pkgdown('.') |>
  pkgdown:::markdown_body('NEWS.md') |>
  read_html()

xml_find_all(xml, './body/div')
# {xml_nodeset (1)}
# [1] <div id="datatable-news-and-updates" class="section level1">\n<h1>data.ta ...

xml_find_all(xml, './body/div/div')
# {xml_nodeset (9)}
# [1] <div id="datatable-v11799-in-development" class="section level2">\n<h2>da ...
# [2] <div id="datatable-v1170-20-feb-2025" class="section level2">\n<h2>data.t ...
# [3] <div id="datatable-v1164-4-december-2024" class="section level2">\n<h2>da ...
# [4] <div id="datatable-v1162-9-october-2024" class="section level2">\n<h2>dat ...
# [5] <div id="datatable-v1160-25-august-2024" class="section level2">\n<h2>dat ...
# [6] <div id="datatable-v1154-27-march-2024" class="section level2">\n<h2>data ...
# [7] <div id="datatable-v1152-27-feb-2024" class="section level2">\n<h2>data.t ...
# [8] <div id="datatable-v1150-30-jan-2024" class="section level2">\n<h2>data.t ...
# [9] <div id="datatable-v11410-dec-2023-back-to-v1100-dec-2016-has-been-moved- ...

[jayhesselberth]

The issue is that the data.table news file doesn't conform to what pkgdown expects.

pkgdown expects all version headings to be h1 or h2. If they are h1, they are converted (later in build-news.R) to h2 so that only rendered page title is h1.

pkgdown/R/build-news.R

Lines 166 to 169 in 824ab4a

	if (ulevels == "h1") {
	# Bump every heading down a level so to get a single <h1> for the page title
	tweak_section_levels(xml)
	}

So in your specific case you need to remove the h1 at the top of the file, and then you need to edit your h2 lines to conform to the suggested headings at the top of ?build_news(). These are matched by regex here:

pkgdown/R/build-news.R

Lines 218 to 235 in 824ab4a

	news_version <- function(x, pkgname) {
	pattern <- paste0(
	"(?x)
	(?:",
	pkgname,
	"|version|changes\\ in)
	\\s+ # whitespace
	v? # optional v followed by
	(?<version>
	(?:\\d+[.-]\\d+)(?:[.-]\\d+)* # digits, dots, and dashes
	| # OR
	\\(development\\ version\\) # literal used by usethis
	)
	"
	)
	pieces <- re_match(x, pattern, ignore.case = TRUE)
	gsub("^[(]|[)]$", "", pieces$version)
	}

[MichaelChirico]

I maintain this is a {pkgdown} bug -- it is at odds with R itself, which has no problem with data.table's current NEWS structure:

str(tools:::.build_news_db_from_package_NEWS_md("data.table/NEWS.md"))

Classes ‘news_db_from_md’, ‘news_db’ and 'data.frame':  23 obs. of  5 variables:
 $ Version : chr  "1.17.99" "1.17.99" "1.17.99" "1.17.0" ...
 $ Date    : chr  NA NA NA NA ...
 $ Category: chr  "NEW FEATURES" "BUG FIXES" "NOTES" "POTENTIALLY BREAKING CHANGES" ...
 $ Text    : chr  "1.  New sort_by() method for data.tables, #6662. It uses forder() to\n    improve upon the data.frame method an"| __truncated__ "1.  Custom binary operators from the lubridate package now work with\n    objects of class IDate as with a Date"| __truncated__ "1.  Continued work to remove non-API C functions, #6180. Thanks Ivan\n    Krylov for the PRs and for writing a "| __truncated__ "1.  In DT[, variable := value], when value is class POSIXlt, we\n    automatically coerce it to class POSIXct i"| __truncated__ ...
 $ HTML    : chr  "<ol>\n<li>\n<p>New <code>sort_by()</code> method for data.tables, <a href=\"https://github.com/Rdatatable/data."| __truncated__ "<ol>\n<li>\n<p>Custom binary operators from the <code>lubridate</code> package now work with objects of class <"| __truncated__ "<ol>\n<li>\n<p>Continued work to remove non-API C functions, <a href=\"https://github.com/Rdatatable/data.table"| __truncated__ "<ol>\n<li>In <code>DT[, variable := value]</code>, when value is class <code>POSIXlt</code>, we automatically c"| __truncated__ ...

I will also propose extending the version match regex, but first we shouldn't need to remove the <h1> entry. A single <h1> entry in an HTML file is good and recommended, after all!

[jayhesselberth]

I can confirm that the pkgdown NEWS.md is also accommodated by R.

path <- system.file("NEWS.md", package = "pkgdown")

str(tools:::.build_news_db_from_package_NEWS_md(path))
#> Classes 'news_db_from_md', 'news_db' and 'data.frame':   48 obs. of  5 variables:
#>  $ Version : chr  "2.1.3" "2.1.2" "2.1.2" "2.1.1" ...
#>  $ Date    : chr  NA NA NA NA ...
#>  $ Category: chr  "" "New features" "Bug fixes and minor improvements" "" ...
#>  $ Text    : chr  "  - Menu items can now set class and id." "  - ROR IDs in DESCRIPTION are transformed into icons, similar to ORCID\n    IDs. (#2850)\n  - New clean_site(f"| __truncated__ "  - Option pkgdown.internet was removed because CDN file caching now\n    requires an internet connection.\n  -"| __truncated__ "  - Added keyboard shortcut, /, to focus search bar (#2423).\n  - The BugReports field can now be an email (@ca"| __truncated__ ...
#>  $ HTML    : chr  "<ul>\n<li>Menu items can now set <code>class</code> and <code>id</code>.</li>\n</ul>\n" "<ul>\n<li>ROR IDs in <code>DESCRIPTION</code> are transformed into icons, similar to ORCID IDs. (#2850)</li>\n<"| __truncated__ "<ul>\n<li>Option <code>pkgdown.internet</code> was removed because CDN file caching now requires an internet co"| __truncated__ "<ul>\n<li>Added keyboard shortcut, <code>/</code>, to focus search bar (#2423).</li>\n<li>The <code>BugReports<"| __truncated__ ...

^{Created on 2025-06-22 with reprex v2.1.1}

So I'm not sure I'd call this a bug in pkgdown -- moreso an opinion of how a NEWS file should be formatted.

[jayhesselberth]

A possible general solution is to leverage the internal R tool to build the news data table. But that is an unexported function that could change, and it would be a fair bit of work to support a single package.

[MichaelChirico]

It's unexported, but it's the implementation of an exported function: utils::news(). I only used the private function to get at the issue more directly.

https://github.com/r-devel/r-svn/blob/fbea9dac507d9d42dd9b3eb6a47a79202d06de11/src/library/utils/man/news.Rd#L95-L103

File NEWS.md should contain the news... with a primary heading level (e.g., ATX heading # or ##) giving the version number after a prefix followed by a space, and optionally followed by a space and a parenthesized ISO 8601 format date. Where available, secondary headings (i.e., one heading level higher than the primary heading level) are taken to indicate categories.