Failing tests on riscv64 architecture.

[charles-plessy]

Hello,

we distribute data.table in Debian and therefore run its test suite on a broader range of architectures than are typically covered on CRAN. One such architecture is riscv64, where R reports .Machine$longdouble.digits == 113 (i.e. true quad‑precision long double arithmetic).

Since data.table 1.18.0, two regression tests in tests/froll.Rraw have started to fail on this architecture:

• 6001.740 (frollvar)
• 6001.741 (frollsd)

In both cases, the observed difference is that the result contains NaN where the test expects NA for rolling windows involving missing values. The failures are deterministic and limited to these two tests.

A full CI log from Debian (riscv64) is available here:
https://ci.debian.net/packages/r/r-cran-data.table/testing/riscv64/69978545/

Would you consider adjusting these two tests to special‑case architectures with extended long double precision (for example, accepting NaN vs NA, or conditionally skipping the checks when .Machine$longdouble.digits > 64)?

We would of course be happy to test any proposed change on riscv64.

Best regards,

Charles Plessy

[aitap]

Dear Charles Plessy,

Thanks for the report! Looks like our code had assumed that the NaN payload propagates when converting an NA to long double and back. What happens when the following code runs on RISC-V?

#include <assert.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>

static double make_NA(void) {
	uint64_t xi = 0x7ff00000000007a2; // "1954" payload means NA
	double xd;
	static_assert(sizeof xd == sizeof xi);
	memcpy(&xd, &xi, sizeof xi);
	return xd;
}

static bool is_NA(double xd) {
	uint64_t xi;
	memcpy(&xi, &xd, sizeof xi);
	return (xi & 0xffff) == 1954;
}

static void hexdump(const void * buf, const size_t sz) {
	for (const unsigned char *x = buf, *end = x + sz; x < end; ++x)
		printf("%02x", (unsigned)*x);
}

int main() {
	long double wsum = 0;
	wsum += make_NA();
	printf("long double("); hexdump(&wsum, sizeof wsum); printf(") = %Lg\n", wsum);
	double dsum = (double)wsum;
	printf("double(");
	hexdump(&dsum, sizeof dsum);
	printf(") = %g, is_NA = %s\n", dsum, is_NA(dsum) ? "true" : "false");
}

[eddelbuettel]

@aitap spot on, thank you! I had scratched my had about 'only two of the froll tests fails but they involve NA but had not clicked about non-double` NA entering.

@charles-plessy can you try the code on one of the riscv64 machines we oight to have access to within Debian?

[aurel32]

I think the problem is that on RISC-V, any operation involving a NaN as an input operand generates a new canonical NaN with the payload as zero. So the payload is not propagated to the result.

And here is the output of the above code from @aitap shows:

long double(0000000000000000000000000080ff7f) = nan
double(000000000000f87f) = nan, is_NA = false

[eddelbuettel]

Thanks @aurel32 ! That should be catchable from R by switching, say, from is.na() to !is.finite() and I am sure @aitap will be on top of that.

[aitap]

@jangorecki, here's where I think NAs fail to propagate on riscv64:

data.table/src/froll.c

Lines 1238 to 1245 in eddec87

	long double wsum = 0.0;
	for (int j=-k+1; j<=0; j++) {
	wsum += x[i+j];
	}
	if (!R_FINITE((double) wsum)) {
	if (ISNAN((double) wsum)) {
	if (!narm) {
	ans->dbl_v[i] = (double) wsum; // propagate NAs

Operations on doubles don't propagate missingness either: NA+0 is NaN on riscv64, not NA.

Since testing individual x[i+j] for being missing would likely worsen performance, I'm assuming you'll prefer to skip the tests if identical(Sys.info()[['machine']], 'riscv64')?

[jangorecki]

Yes, definitely better to skip test for risc v. Thank you for looking into it.

[eddelbuettel]

Yes, also what I suggested in the Debian context of the bug.

But maybe the C code should have a #if defined() for the arch and warn that NA propagation is different on the platform and/or consider a slow path for it? Sounds like overkill but silent failure on just one platform also is not great.

[jangorecki]

Considering how uncommon is risc v a the fact that NA vs NaN usually makes no difference it is safe to just escape those.

[aitap]

How does the Debian package of R for riscv64 pass its own tests on this platform? I've tried to build a recent R-devel in a QEMU chroot, and it fails make check-all for the same reason. There's an R-devel thread and a bug report, and so far it seems that migrating off NaN payload propagation is the plan.

[eddelbuettel]

Apparently I currently skip the test on most / all platforms 🙃

[charles-plessy]

@aitap The Debian packaging of R pakcages is streamlined and usually does not take advantage of package-specific makefiles. Basically, we just run R CMD INSTALL at build time, and we run the contents of the tests directory at test time. In my setup, I run the CI tests locally after binary build before the source package is uploaded, but only on my platform, amd64.

[eddelbuettel]

For completeness, there are different schools of thought about this, especially when it comes to r-cran-* packages here.

The standard Debian install step, from 30,000 feet, really is configure; make; make check; make install with appropriate conditioning via parameters. Many packages do, for some I as maintainer decide not to and r-base is one of them (as I cannot expect R Core to chase borderline architectures they do not have access to). And I alone cannot fix this either so I opt to bypass it. I see my role primarily as delivering current and quality R binaries to mainstream platforms.