BLD: compiler recognition on MacOS

[tylerjereddy]

Expected behavior

Build system automagically handles both clang and "genuine" gcc on MacOS. This command should work with spack built gcc compiler toolchain loaded on macOS 10.15.7: python setup.py develop

Actual behavior

The build system thinks it is using clang but it is actually using gcc built with clang by spack (not the "alias" you would normally have on a "normal" mac machine).

gcc: error: unrecognized command line option '-stdlib=libc++'
error: command '/Users/treddy/github_projects/spack/opt/spack/darwin-catalina-x86_64/clang-11.0.0-apple/gcc-8.3.0-fxj4m5svuktydrjgtfkbxyifibwf6wsi/bin/gcc' failed with exit status 1

Code to reproduce the behavior

python setup.py develop

Python 3.7.4, though not likely relevant. Latest develop branch.

Hacky work around, that hints at part of code involved:

--- a/package/setup.py
+++ b/package/setup.py
@@ -286,9 +286,9 @@ def extensions(config):
     cpp_extra_link_args=[]
     # needed to specify c++ runtime library on OSX
     if platform.system() == 'Darwin' and using_clang():
-        cpp_extra_compile_args.append('-stdlib=libc++')
+        #cpp_extra_compile_args.append('-stdlib=libc++')
         cpp_extra_compile_args.append('-mmacosx-version-min=10.9')
-        cpp_extra_link_args.append('-stdlib=libc++')
+        #cpp_extra_link_args.append('-stdlib=libc++')
         cpp_extra_link_args.append('-mmacosx-version-min=10.7')

[IAlibay]

@tylerjereddy I think I'm misunderstanding the issue. Is it that somehow the spack built gcc is triggering a return of True for using_clang()?

If so, out of curiosity what's the output of gcc -v?

[tylerjereddy]

@IAlibay Yes, that's what happens--I'd say that ideally the mechanism of compiler detection should not rely on string analysis alone--for example, we just do return 'clang' in compiler_ver in that function, which is obviously true because gcc was built from source using clang and spack will include the string clang in some of the metadata, etc.

Here's a small sample of gcc -v output; just this first line is sufficient to trick our build system already:

Reading specs from /Users/treddy/github_projects/spack/opt/spack/darwin-catalina-x86_64/clang-11.0.0-apple/gcc-8.3.0-fxj4m5svuktydrjgtfkbxyifibwf6wsi/lib/gcc/x86_64-apple-darwin19.3.0/8.3.0/specs

[tylerjereddy]

I wonder if NumPy distutils has something we can borrow for this, probably

[IAlibay]

So probably a very naive question (especially since we go through the trouble of using -v currently), but why aren't we just using the string from compiler.compiler in our distutils compiler check?

I realise folks could technically alias gcc or clang to whatever string value they want, but surely we could be taking the name provided at face value right? If "clang" in compiler.compiler[0] returns True then we should assume intent by naming?

I can see a case of someone calling your above case gcc-clang though...

[tylerjereddy]

I think 'gcc' more often means clang than gcc on MacOS, so we can't really take the name too literally. The string clang will almost always be in the description we get out there because it is basically an alias of sorts to clang. So, I don't think this is just "folks aliasing it," but rather that's the default case on MacOS if you don't change anything so our current code path would likely cover 90%+ of end users/devs.

Indeed, it is me who has changed something by using genuine gcc instead of the "default alias to clang." That should be fair game though.

I'm pretty sure most robust solutions to the problem involve a try/except style approach to seeing what flags can be supported and deciding off that (I think CMake commonly does small try_compile type things, probably NumPy distutils too).

[hmacdope]

Sorry to chime in a bit here but down the road at distopia we use something like:

  #ifdef __clang_major__
    #define DISTOPIA_CLANG 1
  #endif

  #ifdef __INTEL_COMPILER
    #define DISTOPIA_ICC 1
  #endif
  
  #if defined(__GNUC__) && !defined(DISTOPIA_CLANG) && !defined(DISTOPIA_ICC)
    #define DISTOPIA_GCC 1
  #endif

  #if defined(_MSC_VER) && !defined(DISTOPIA_ICC)
    #define DISTOPIA_MSVC 1
  #endif

you could then combine with something like:

static_assert(DISTOPIA_CLANG != 1, "compiler is not clang");

which will fail to compile if not on clang if you want to go for the try_compile idea.