Skip to content

CMake: Use LDC as (cross-)archiver for static runtime libs#2319

Merged
kinke merged 1 commit intoldc-developers:masterfrom
kinke:ldcArchiver
Sep 11, 2017
Merged

CMake: Use LDC as (cross-)archiver for static runtime libs#2319
kinke merged 1 commit intoldc-developers:masterfrom
kinke:ldcArchiver

Conversation

@kinke
Copy link
Member

@kinke kinke commented Sep 9, 2017

Attempt to fix #2309.


# Define a 'D' CMake linker language for the static runtime libs, using LDC as archiver.
# LDC handles (cross-)archiving internally via LLVM and supports LTO objects.
set(CMAKE_D_CREATE_STATIC_LIBRARY "${LDC_EXE_FULL} -lib -of=<TARGET> <OBJECTS>")
Copy link
Member Author

@kinke kinke Sep 9, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LINKER_LANGUAGE D => CMake looks up command template CMAKE_<LANG>_CREATE_STATIC_LIBRARY with known placeholders (<TARGET>, <OBJECTS> etc.). Found via Google, StackOverflow and finally looking up a few things in the default CMake modules.

The alternative would be way uglier, as we'd have to compile the C/ASM files manually, add custom commands for the LDC archiving command lines etc. etc.

@kinke kinke changed the title CMake: Use LDC as (cross-)archiver for static runtime libs [WIP] CMake: Use LDC as (cross-)archiver for static runtime libs Sep 9, 2017
@kinke
Copy link
Member Author

kinke commented Sep 9, 2017

LLVM's MSVC archiver/lib.exe driver apparently has issues with objects with debuginfos, leading to MS linker warnings when linking against the static debug libs, e.g.:

druntime-ldc-debug.lib(memory.obj) : warning LNK4255: library contain multiple objects of the same name; linking object as if no debug info
druntime-ldc-debug.lib(hash.obj) : warning LNK4255: library contain multiple objects of the same name; linking object as if no debug info
druntime-ldc-debug.lib(common.obj) : warning LNK4255: library contain multiple objects of the same name; linking object as if no debug info
druntime-ldc-debug.lib(config.obj) : warning LNK4255: library contain multiple objects of the same name; linking object as if no debug info
druntime-ldc-debug.lib(gc.obj) : warning LNK4255: library contain multiple objects of the same name; linking object as if no debug info

At least with the currently used LLVM 4.0.1.

@joakim-noah
Copy link
Contributor

Tested this pull on native Android/ARM and cross-compiling to Android from linux/x64 and Wine/x64, no problem. I updated #2301 to rely on this instead of the Android NDK's ar/ranlib, let me know if this won't go in soon so I can go back to the NDK there if needed.

@kinke
Copy link
Member Author

kinke commented Sep 10, 2017

Thanks for testing. With the debuginfo issue above for MSVC targets, it can't be enabled by default (yet), so maybe introducing a RT_ARCHIVE_WITH_LDC CMake var (defaulting to ON for all but MSVC targets) would be appropriate.

@joakim-noah
Copy link
Contributor

Sounds good.

By default for all non-MSVC targets; overrideable via CMake var
RT_ARCHIVE_WITH_LDC.

LDC/LLVM handles cross-archiving and LTO objects out of the box.
@kinke kinke added this to the 1.4.0 milestone Sep 10, 2017
@kinke kinke changed the title [WIP] CMake: Use LDC as (cross-)archiver for static runtime libs CMake: Use LDC as (cross-)archiver for static runtime libs Sep 10, 2017
Copy link
Contributor

@joakim-noah joakim-noah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, need this for #2301.

@kinke kinke merged commit e31a3d3 into ldc-developers:master Sep 11, 2017
@kinke kinke deleted the ldcArchiver branch September 11, 2017 12:28
@joakim-noah
Copy link
Contributor

It looks like this breaks with llvm 5.0, in a subtle way. If you archive a library once, it works fine. However, if you then go make a change to some file and rebuild the library, any attempt to use it errors out because of repeated symbols in the C/ASM files. Turns out that all the object files have been archived twice, if you run ar t libdruntime-ldc.a. Tweak a file in druntime again and rebuild, everything's there a third time, ie it just keeps adding the same files over and over again.

Looking at the contents of the archive produced by llvm 4.0.1 with this pull, it shows all object files at the top-level, no paths. When you modify a file and re-archive, it simply overwrites the old object file. However, with llvm 5.0, it archives the paths too, ie runtime/CMakeFiles/druntime-ldc.dir/druntime/src/core/stdc/errno.c.o, and keeps adding all the object files each time you rebuild. I wonder if that has anything to do with using the path.

I see this when compiling druntime-test-runner natively on linux/x64 or Android/ARM: tweak some druntime file and I can't rebuild, because of link errors. Let me know if you can reproduce.

@kinke
Copy link
Member Author

kinke commented Sep 17, 2017

IIRC, we use the equivalent of ar rcs ..., where the r stands for 'replace existing objects of the same name'. So even if the objects are now somehow identified by path instead of filename alone, it 'should' still work, provided the paths are the same.
As LLVM's POSIX archiver isn't a LLVM lib (as the lib.exe equivalent for MSVC targets) but a separate tool, I had to integrate a cut-down and slightly customized version of it, based on rather early LLVM 5.0, so I'm wondering why this seems to depend on the LLVM version. See https://github.com/ldc-developers/ldc/blob/master/driver/archiver.cpp.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use LDC for archiving static runtime libs

2 participants