check_check and check_check_export fail on Guix powerpc64le-linux

[marusich]

As described in Guix bug report 47698, "check" fails to build on Guix powerpc64le-linux because its check_check and check_check_export tests fail. The build was conducted in an isolated environment where the following versions of software were used:

• check version: 0.15.2
• libc version: 2.31
• gcc version: 7.5.0
• binutils version: 2.34

Please check out the Guix bug report for more details. In particular, see here for a copy of the test-suite.log file showing the failure.

When the failure occurs, we are building natively using the powerpc64le-unknown-linux-gnu triplet on a system that uses Debian's Linux kernel. The kernel version shown in /proc/version is:

Linux version 5.10.0-1-powerpc64le (debian-kernel@lists.debian.org) (gcc-10 (Debian 10.2.1-3) 10.2.1 20201224, GNU ld (GNU Binutils for Debian) 2.35.1) #1 SMP Debian 5.10.5-1 (2021-01-09)

The failure occurs when attempting to build "check" using Guix (e.g., via a command such as guix build check); the check_check and check_check_export test failures cause the overall build of "check" to fail. Guix builds "check" in an isolated environment where nothing from the underlying Debian system is used; with the exception of the kernel itself, the only software that is used to build "check" and run its tests comes from Guix, not Debian. The kernel from Debian is used for syscalls, file system access, etc., but the build environment has no access to tools in the usual locations like /usr/bin/gcc. This means that, for example, although /proc/version above indicates that Debian is using GCC 10, during the build it is in fact GCC 7.5.0 which is used to compile "check".

Do you have any idea what might be causing these failures, or how I can try to narrow down the problem? I'm having a hard time understanding the intent of the tests, since it isn't clear to me if assertions made during the tests are actually assertions or if they are themselves part of the code under test. Because of this confusion, and because I'm totally unfamiliar with the "check" framework to begin with, it's hard to tell what the intended behavior is, so it's hard to tell what's wrong. I'm hoping maybe you can help point me in the right direction.

This problem does not occur when using Guix on x86_64-linux (i.e., a system building "check" natively using the triplet x86_64-unknown-linux-gnu), so it might be a powerpc64le-specific issue. The only existing bug I saw that seemed possibly relevant was Fedora 28 (x86_64) test fails #157, but there isn't really any additional info in that bug report, I'm afraid.

I'm happy to put in the time to debug this myself - I've already tried putting in a few hours to do so - but I'm a little lost and not sure how to debug further. Any advice on how to get started would be very welcome. Thank you in advance for taking the time to help.

[brarcher]

TL;DR: try pulling in #322 and see if it works.

The logs are not too obvious to look at, because Check has unit tests against itself to make sure that its assertion pass and fail when they expect to. Some of the log is the output of the tests-under-test, so it's not obvious if the failures are expected or not.

To debug these, I typically look at the expected failure messages and see if any are wrong. That is the part of the log that looks like this:

...
check_check_master.c:631:P:Core Tests:test_check_all_msgs:124: Passed
check_check_master.c:685:F:Core Tests:test_check_all_msgs:125: For test 125:Simple Tests:test_ck_assert_ldouble_eq_tol expected 'Assertion 'fabsl(y - x) < t' failed: x == 0.001, y == 0.002, t == 0.001', got 'Passed'
check_check_master.c:631:P:Core Tests:test_check_all_msgs:126: Passed
check_check_master.c:631:P:Core Tests:test_check_all_msgs:127: Passed
check_check_master.c:631:P:Core Tests:test_check_all_msgs:128: Passed
...

Those check the messages reported in the tests against this array:

check/tests/check_check_master.c

Line 68 in 9ae888a

static master_test_t master_tests[] = {

And, if the unit-tests-under-test report back the same result as the expected result (either a pass or a fail) then test_check_all_msgs will say Passed. Otherwise, it says what the mismatch was.

In this case, there are two tests which have a result which differs from the expected result. The two tests which are failing are related to floating point assertions. Here is the analysis of the first one.

test_ck_assert_ldouble_eq_tol expects to fail with this message, however the test passed.

The tolerance is really close, and on PowerPC they do fail. A change was put in after Check's most recent release that changes the test so it can pass on PowerPC:
#322

I think you should pull in that change and see if it fixes the issue.

[marusich]

Thank you for that information! In Guix, I tried applying the change introduced by commit 4fbe702 to the source code in the 0.15.2 release of check, and the tests now pass. The entire build of check 0.15.2 is even reproducible on my powerpc64le-linux machine (as shown by guix build --rounds=2 check). How nice!

My understanding is that the changes for PR #322 have been merged to master, so they will be included in the next check release. At that time, Guix can just use the latest release instead of patching release 0.15.2.

[rhattersley]

FYI, conda-forge is also looking to patch 0.15.2 with 4fbe702 in order to enable the PPC build.

Are there any plans to release a new version of check so we can remove the patch requirement?