OSX needs OSX 10.9 SDK for conda compilers to function

[rabah-khalek]

After my update yesterday to Mac Mojave, I am not able to set up an nnpdf env with conda.
I redid the full procedure of setting an env and install nnpdf, but I am getting this error when arriving to the step cmake .. -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX :

-- Setting build type to 'Release' as none was specified.
-- The C compiler identification is Clang 4.0.1
-- The CXX compiler identification is Clang 4.0.1
-- Check for working C compiler: /Users/rabah/Documents/miniconda3/envs/nnpdf/bin/x86_64-apple-darwin13.4.0-clang
-- Check for working C compiler: /Users/rabah/Documents/miniconda3/envs/nnpdf/bin/x86_64-apple-darwin13.4.0-clang -- broken
CMake Error at /Users/rabah/Documents/miniconda3/envs/nnpdf/share/cmake-3.12/Modules/CMakeTestCCompiler.cmake:52 (message):
  The C compiler

    "/Users/rabah/Documents/miniconda3/envs/nnpdf/bin/x86_64-apple-darwin13.4.0-clang"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /Users/rabah/Documents/new/nnpdf/conda-build/CMakeFiles/CMakeTmp
    
    Run Build Command:"/usr/bin/make" "cmTC_a2c8d/fast"
    /Library/Developer/CommandLineTools/usr/bin/make -f CMakeFiles/cmTC_a2c8d.dir/build.make CMakeFiles/cmTC_a2c8d.dir/build
    Building C object CMakeFiles/cmTC_a2c8d.dir/testCCompiler.c.o
    /Users/rabah/Documents/miniconda3/envs/nnpdf/bin/x86_64-apple-darwin13.4.0-clang   -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe  -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk   -o CMakeFiles/cmTC_a2c8d.dir/testCCompiler.c.o   -c /Users/rabah/Documents/new/nnpdf/conda-build/CMakeFiles/CMakeTmp/testCCompiler.c
    Linking C executable cmTC_a2c8d
    /Users/rabah/Documents/miniconda3/envs/nnpdf/bin/cmake -E cmake_link_script CMakeFiles/cmTC_a2c8d.dir/link.txt --verbose=1
    /Users/rabah/Documents/miniconda3/envs/nnpdf/bin/x86_64-apple-darwin13.4.0-clang -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe  -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names -Wl,-pie -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs  CMakeFiles/cmTC_a2c8d.dir/testCCompiler.c.o  -o cmTC_a2c8d 
    ld: warning: ignoring file /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/lib/libSystem.tbd, file was built for unsupported file format ( 0x2D 0x2D 0x2D 0x20 0x21 0x74 0x61 0x70 0x69 0x2D 0x74 0x62 0x64 0x2D 0x76 0x33 ) which is not the architecture being linked (x86_64): /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/lib/libSystem.tbd
    ld: dynamic main executables must link with libSystem.dylib for architecture x86_64
    clang-4.0: error: linker command failed with exit code 1 (use -v to see invocation)
    make[1]: *** [cmTC_a2c8d] Error 1
    make: *** [cmTC_a2c8d/fast] Error 2
    

  

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:21 (project)


-- Configuring incomplete, errors occurred!
See also "/Users/rabah/Documents/new/nnpdf/conda-build/CMakeFiles/CMakeOutput.log".
See also "/Users/rabah/Documents/new/nnpdf/conda-build/CMakeFiles/CMakeError.log".

[Zaharid]

A search yields this link:
bigartm/bigartm#910

[wilsonmr]

This looks like a similar issue that I get even on High Sierra if I didn't have the correct SDK. Have you read this? https://conda.io/docs/user-guide/tasks/build-packages/compiler-tools.html

"The macOS compilers require the macOS 10.9 SDK. The SDK license prevents it from being bundled in the conda package."

Easiest way to check if this is the problem: you download the SDK from here https://github.com/phracker/MacOSX-SDKs (follow link for just 10.9) then when you compile you need to:

source activate conda-env
export CONDA_BUILD_SYSROOT=/path/to/MacOSX10.9.sdk
cmake .. 

Once you've done that, try running cmake and see if it still thinks compiler is broken. If that does work then I can recommend a more satisfactory way to have the SDK with having to repeatedly point the environment variable to look at it

[rabah-khalek]

Great ! I will try this asap.
Thanks both for the prompt reply.
I will keep you posted.

[rabah-khalek]

@wilsonmr It works like a charm ! thanks.

[wilsonmr]

Glad it worked but we shouldn't just close this, I think it should be documented.. also there is a more permanent way to fix it

On Tue, 30 Oct 2018, 12:55 Rabah Abdul Khalek, ***@***.***> wrote: @wilsonmr <https://github.com/wilsonmr> It works like a charm ! thanks. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#323 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AgVi--xd3OYCqGPw20wUGPJ2etYdT5j9ks5uqExRgaJpZM4YBgmS> .

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

[rabah-khalek]

@wilsonmr, you proved to be more efficient regarding solving such conda problems, can I leave the fix to you?

[wilsonmr]

yeah no problem

[wilsonmr]

@rabah-khalek So if you want a more permanent fix I recommend the following (which I will add to the correct documentation later):

• Make an apple developer account so you can access this list https://developer.apple.com/download/more/
• Download a version of Xcode with 10.9 SDK, I found that 6.2 explicitly has this, however I believe 6.4 also does because that's what the Travis CI appears to use.
• Once you have downloaded DO NOT move to apps directly, since you already have a newer version of Xcode there, instead drag to desktop to install there
• With Xcode 6.2/6.4 installed to desktop, right click and rename something sensible like Xcode6.x, once renamed it is safe to move it to the Applications folder

finally type sudo xcode-select -s /Applications/Xcode6.x.App this will change the Xcode being used to the older version which has the correct SDK. This means that the correct SDK will automatically be pointed at when you enter a development environment. If you want to compile something with newer version of clang then sudo xcode-select -r resets to the newer version of Xcode. But it means if you're constantly making development environments/using them to develop code then you don't have to export the CONDA_BUILD_SYSROOT everytime. This is what I do, the only downside is that Xcode is like 5Gb which is annoyingly large

[rabah-khalek]

Thanks a lot @wilsonmr for taking care of this,
For now I just downloaded the sdk as you mentioned earlier and I added this to my bashrc:
export CONDA_BUILD_SYSROOT=/path/to/MacOSX10.9.sdk.

[wilsonmr]

I think it won't work in the bashrc because this variable is overwritten when you activate the environment. You could modify the activation script but then if you create a new dev environment you would have to repeat this step

On Wed, 31 Oct 2018, 09:08 Rabah Abdul Khalek, ***@***.***> wrote: Thanks a lot @wilsonmr <https://github.com/wilsonmr> for taking care of this, For now I just downloaded the sdk as you mentioned earlier and I added this to my bashrc: export CONDA_BUILD_SYSROOT=/path/to/MacOSX10.9.sdk. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#323 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AgVi-04C5LCRcRj2cWEGilsLN0kfq4hWks5uqWiGgaJpZM4YBgmS> .

The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

[rabah-khalek]

It doesn't seem to overwrite it, I think CONDA_BUILD_SYSROOT is a static variable since you also can find it outside the env

[wilsonmr]

Have you opened a new shell, activated the environment and ran cmake since adding it to the bashrc?