gcc: fix darwin dispatch/object.h header issue

[goxberry]

Fixes #1203. Apple ships headers in Yosemite (and possibly earlier) that
are gcc-incompatible, but compile fine with clang. The fix is to copy
the offending system header from /usr/include/${REST_OF_HEADER_PATH} to
${GCC_PREFIX}/include/${REST_OF_HEADER_PATH} and replace the non-gcc-
compatible features with gcc-compatible equivalents.

See https://github.com/hashdist/hashstack/pull/771/files for
inspiration, and
http://hamelot.io/programming/osx-gcc-dispatch_block_t-has-not-been-declared-invalid-typedef/
for a description of the header issue.

[goxberry]

Haven't tested it yet, but will tomorrow (gcc is a long build, and so is cmake).

[goxberry]

Fixed regex error in transcribing from sed (POSIX regex) to Python regex. gcc is currently building now, will check the header when it's done (probably around when I wake up), and try installing cmake with gcc.

[goxberry]

Added flake8 fixes.

[adamjstewart]

Does this need a slash after prefix?

[goxberry]

@adamjstewart Pushed some of those fixes, still need to test again.

[goxberry]

Made the code a little more debuggable. The good news: the section of code I added does what it's supposed to, which is to copy the gcc-incompatible header to the correct location and perform the right replacement. The bad news: I couldn't finish the build before I leave work, because building gcc with spack install --verbose --keep-prefix gcc is slow (though building gcc is slow regardless). I will try to test again overnight.

[goxberry]

After several false starts and 2-3h compiles of gcc, the good news is: gcc compiles and the patched header is copied in the right place, so this patch "works".

The bad news is: spack install cmake %gcc still fails on OS X Yosemite, because spack still picks up the header from /usr/include/dispatch/object.h instead of from ${GCC_PREFIX}/dispatch/object.h, so it doesn't actually fix the CMake build bug in #1203.

If the fix from hashdist/hashstack#771 indeed fixes the issue for HashDist, then maybe switching the order of compiler flags will force gcc to pick up the patched version of object.h? Could anyone familiar with the deeper internals of spack point me in the right direction towards where I might look at the order of the compiler flags spack uses? (@tgamblin? @alalazo?)

[alalazo]

@goxberry I think your best try is the compiler wrapper : lib/spack/env/cc. Most of the information passed to the wrapper is set in lib/spack/spack/build_environment.py in particular in set_compiler_environment_variables. Hope this could help a bit...

[goxberry]

@alalazo Thanks! Looking in those places helped, and gave me ideas on testing.

[goxberry]

@tgamblin Narrowed the issue down to the following: the compiler still doesn't seem to pick up the patched gcc header before the non-compliant system header.

So far I've tried:

• bin/spack install cmake cppflags=\"-I${GCC_PREFIX}/include/dispatch\" %gcc to move the patched header to the front of the header search path
• swapped cppflags for cxxflags
• copied the patched object.h to each of the paths mentioned in https://gcc.gnu.org/onlinedocs/cpp/Search-Path.html with libdir in them, where libdir was replaced with ${GCC_PREFIX}/lib64

In all cases, I've encountered the same error. I don't know what else to try at this point; I can't replicate the behavior from hashdist/hashstack#771, so at this point, I need to reach out to Ondrej or Aron and ask them what is actually happening with that patch.

[goxberry]

@tgamblin Okay, something worked: bin/spack install cmake cxxflags=\"-I${GCC_PREFIX}/include\" %gcc, and CMake builds, which is progress.

[goxberry]

@tgamblin @davydden @alalazo Figured out the issue. The patch has been tested and it works; the problem I encountered was that spack won't distinguish between versions of compilers it builds and system compilers.

If I build gcc 6.1.0 with spack and gcc 6.1.0 is installed on my machine, then spack compiler find ~/spack/opt/.../gcc-6.1.0-${HASH} won't detect a new compiler because their version numbers are the same. All of the trouble I've had testing this patch has to do with spack picking up the system-installed (really, Homebrew-installed) gcc 6.1.0 instead of spack-installed gcc 6.1.0. Upgrading the system gcc to 6.2.0 and manually replacing the stale system-installed gcc 6.1.0 information in .spack/compilers.yaml with the appropriate paths of the spack-installed gcc 6.1.0 fixed this issue temporarily and enabled me to confirm that this patch actually works.

Is there any way to distinguish between spack-installed and system-installed compiler versions? Maybe add a spack-gcc, etc.? Perhaps @citibeth has an opinion? If the preferable way to patch the noncompliant OS X system headers is in spack installs of compilers, then there should be clean way of specifying at the command line that a user wants the spack-installed compiler version instead of the system-installed compiler version.

[citibeth]

Perhaps @citibeth https://github.com/citibeth has an opinion?

[Why me? :-]

OK, my opinion... I think this is a non-issue. I never use Spack's
automagic configure stuff anyway, I just edit compilers.yaml myself.
Always worked for me, and there's never an question in my mind which
compilers Spack is using.

On Tue, Sep 20, 2016 at 8:50 PM, Geoffrey Oxberry notifications@github.com
wrote:

@tgamblin https://github.com/tgamblin @davydden
https://github.com/davydden @alalazo https://github.com/alalazo
Figured out the issue. The patch has been tested and it works; the problem
I encountered was that spack won't distinguish between versions of
compilers it builds and system compilers.

If I build gcc 6.1.0 with spack and gcc 6.1.0 is installed on my machine,
then spack compiler find ~/spack/opt/.../gcc-6.1.0-${HASH} won't detect a
new compiler because their version numbers are the same. All of the trouble
I've had testing this patch has to do with spack picking up the
system-installed (really, Homebrew-installed) gcc 6.1.0 instead of
spack-installed gcc 6.1.0. Upgrading the system gcc to 6.2.0 and manually
replacing the stale system-installed gcc 6.1.0 information in
.spack/compilers.yaml with the appropriate paths of the spack-installed
gcc 6.1.0 fixed this issue temporarily and enabled me to confirm that this
patch actually works.

Is there any way to distinguish between spack-installed and
system-installed compiler versions? Maybe add a spack-gcc, etc.? Perhaps
@citibeth https://github.com/citibeth has an opinion? If the preferable
way to patch the noncompliant OS X system headers is in spack installs of
compilers, then there should be clean way of specifying at the command line
that a user wants the spack-installed compiler version instead of the
system-installed compiler version.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1518 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AB1cd7BBDYJ8Xlc6RIVL7MhamabO1QpTks5qsH9RgaJpZM4JkJEl
.

[eschnett]

I regularly use a Spack-installed compiler. I configure Spack manually to do this, appending -spack to the version number there, so that there is no conflict with auto-detected system compilers. See also here.

[goxberry]

Perhaps @citibeth https://github.com/citibeth has an opinion?
[Why me? :-]

@citibeth You're among the most active commenters on the spack repo and also take the lead on a lot of the documentation issues. If the response is mainly, "a user should just manually configure spack to use the installed compiler in case of version collision," then I believe this advice should be added to the documentation.

[goxberry]

I bring up autoconfiguring Spack partly because it would be convenient -- a Spack-installed compiler should be known to Spack -- and partly because debugging this issue was difficult. The incorrect assumption on my part was that installing gcc 6 via Spack meant that when I typed spack compiler find, it would, well, find the Spack-installed compiler. Instead, it found my system compiler first, and used that instead. As a result of that false assumption, I didn't think to check the paths in compilers.yaml until I uncovered this header incompatibility issue, and only after inserting the patched header in a directory Spack should never touch -- /usr/include. All of this behavior in Spack makes sense after the fact, but I'd hate for someone else to encounter this same issue with respect to patching compilers on their operating system/architecture. Plus, it's a manual configuration step that in theory could be automated (though I'm not familiar enough with the internals of Spack yet to know how).

[citibeth]

Clearly, the right automagic is to modify compilers.yaml appropriately
when you run spack install gcc.

On Tue, Sep 20, 2016 at 10:36 PM, Geoffrey Oxberry <notifications@github.com

wrote:

I bring up autoconfiguring Spack partly because it would be convenient --
a Spack-installed compiler should be known to Spack -- and partly because
debugging this issue was difficult. The incorrect assumption on my part was
that installing gcc 6 via Spack meant that when I typed spack compiler
find, it would, well, find the Spack-installed compiler. Instead, it
found my system compiler first, and used that instead. As a result of that
false assumption, I didn't think to check the paths in compilers.yaml until
I uncovered this header incompatibility issue, and only after inserting the
patched header in a directory Spack should never touch -- /usr/include.
All of this behavior in Spack makes sense after the fact, but I'd hate for
someone else to encounter this same issue with respect to patching
compilers on their operating system/architecture. Plus, it's a manual
configuration step that in theory could be automated (though I'm not
familiar enough with the internals of Spack yet to know how).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1518 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AB1cdyPw5Q5vXWcTDx3QUxLxVTkrFvFEks5qsJhDgaJpZM4JkJEl
.

[davydden]

you write Yosemite and earlier, why don't you check for that (looking at versions)? I am not sure this patch will work on elcapitan or sierra. Generally what Apple ships with next versions is unknown. So i would adjust the if so that it is in accordance with the comment.

[davydden]

see #1814

[goxberry]

you write Yosemite and earlier, why don't you check for that (looking at versions)?

@davydden The API for OS versions isn't documented at all, AFAICT. I had some downtime so I spent a couple hours looking at the Spack code and couldn't make much heads or tails of it, so I only added a check for Yosemite. As I'm sure you're aware, the bare string comparison of OS versions will fail (e.g., spec.architecture.platform_os.version < 10.10 is False when the OS version is 10.9), so as you note in #1814, the OS version should really be a Version object so that it can use the comparison operators inherited from that class.

I am not sure this patch will work on elcapitan or sierra.

Also, AFAICT from Google searches, the string replacement doesn't affect other versions of Mac OS prior to Sierra, excluding Yosemite, of course. That said, I haven't tested this code against any versions of OS X other than Yosemite. If someone has access to OS X systems running Sierra/El Capitan/Mavericks/etc. and would be willing to test this patch, I'd appreciate it.

So i would adjust the if so that it is in accordance with the comment.

I have made a change that I think complies with your request.

[davydden]

the way you wright it now is even more readable than what i have thought. Thanks for changing the if.

@tgamblin ping.

[goxberry]

There's probably one caveat, which is that if you're cross-compiling gcc on OS X Yosemite for some other architecture, then there should not be a patched header. However, when I ran Spack in PDB, I couldn't for the life of me figure out why spack.architecture.target_string (or something to that effect) was None. I guess the fix for this case would be to add and spack.architecture.target_string is None to the conditional.

I like the design of Spec objects a lot because it looks very extensible, and for the life of me, I wish I understood design patterns or software design to write software that looks as extensible. It just took a lot of my time to figure out what was going on, having to flip back and forth between source files due to the compositional & inheritance hierarchy. Documentation would really be a huge help here, and I wish I had the bandwidth to tackle it.

[goxberry]

I was wrong; this patch should work because the conditional will not be satisfied in a cross-compile. This patch is ready to merge, @tgamblin.

[tgamblin]

@goxberry: I'm hoping to get specs documented eventually. They're currently kind of extensible, although I'd really like to get the constraint login in there to make it so the concretization code is essentially declarative. Allowing plugins or something for different fields on Specs would be a start to that but I think the more important thing at the moment is making the concretization better... it basically needs a better solver. Thanks for the contributions and sorry this took a while to merge!