building GCCcore/9.3.0 fails in 2021.12 compat layer because "system" GCC is too new

[boegel]

When testing the installation of GCCcore-9.3.0.eb in the 2021.12 compat layer (cfr. EESSI/compatibility-layer#126), I'm hitting two problems:

1. The compilation of GCC itself fails due to too recent Linux kernel headers:

../../../../libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc:160:10: fatal error: linux/cyclades.h: No such file or directory
  160 | #include <linux/cyclades.h>
      |          ^~~~~~~~~~~~~~~~~~
compilation terminated.

This can be fixed fairly easily by using the patch available in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100379, which is done in easybuilders/easybuild-easyconfigs#14453.

2. Another more serious problem emerges later in the build, when GMP is being built. The configure script fails with:

configure: error: could not find a working compiler, see config.log for details

config.log reveals the problem:

/cvmfs/pilot.eessi-hpc.org/2021.12/compat/linux/aarch64/usr/bin/ld.gold: /tmp/ec2-user/easybuild/build/GCCcore/9.3.0/system-system/GCC_stage1_eb/lib64/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /cvmfs/pilot.eessi-hpc.org/2021.12/compat/linux/aarch64/usr/bin/ld.gold)
collect2: error: ld returned 1 exit status

This is basically due to building an old compiler (GCC 9.3) on a system with a very recent GCC (11.2 in compat layer).

To fix this, we should i) install an older version of GCC in the compat layer (by including it to the EESSI 2021.12 sets), and ii) make that the default system compiler instead of using the latest version (via gcc-config, see also EESSI/compatibility-layer#128).

[boegel]

Same problem when trying to build GCCcore-10.3.0.eb when system compiler is GCC 11.2...

[amadio]

One piece of advice from my long experience as a Gentoo user. Upgrading a compiler will rarely give you problems with ABI (notorious exception was the move from GCC 4.x to 5.x). However, downgrading the system compiler can sometimes give you lots of headaches with that. If you plan on using an older compiler as your system compiler in the compatibility layer, I'd make sure that this is done during the bootstrap process. The very last step is to recompile all packages with the system compiler (emerge -e world), so at that point in the script, you should add some changes to install (if needed) and set GCC 9.4 as your default. Then once everything is rebuilt with GCC 9.4, you should be fine with using any GCC >= 9.4 on top of the compatibility layer. If the bootstrap uses GCC 11, and afterwards you build things using an older GCC, it's likely that some incompatibilities will arise.

[boegel]

@amadio Fully agreed, we were already testing exactly that: masking GCC > 10 for the bootstrap, see #140

[boegel]

Fixed by using GCC 9.4 to bootstrapping Prefix for 2021.12 compat layer, see EESSI/compatibility-layer#140