Implement --use-shared-defaultlib switch

[mihails-strasuns]

Partially fixes #1282

Allows to force linking to shared version of standard
library if both lphobos2.so and lphobos2.a are present

Sadly, my cmake knowledge is below zero so I can't tweak build to
generate both targets at the same time but I can workaround it by
building LDC twice in package script.

[mihails-strasuns]

Is the proposed API suitable or shall I change it?

[dnadlinger]

Sorry for the lack of replies here so far, this is pretty much unacceptable from our side. People (including me) are generally quite paranoid about command line argument design, as it is something we tend to be stuck with for a long time.

I think we will probably end up with something like this switch. DMD instead seems to recognise specially formatted defaultlib/debuglib strings (see below), but I don't think requiring the user to repeat the libraries to use from the config files would necessarily be a good idea.

---

        const(char)* libname = global.params.symdebug ? global.params.debuglibname : global.params.defaultlibname;
        size_t slen = strlen(libname);
        if (slen)
        {
            char* buf = cast(char*)malloc(3 + slen + 1);
            strcpy(buf, "-l");

            if (slen > 3 + 2 && memcmp(libname, "lib".ptr, 3) == 0)
            {
                if (memcmp(libname + slen - 2, ".a".ptr, 2) == 0)
                {
                    argv.push("-Xlinker");
                    argv.push("-Bstatic");
                    strncat(buf, libname + 3, slen - 3 - 2);
                    argv.push(buf);
                    argv.push("-Xlinker");
                    argv.push("-Bdynamic");
                }
                else if (memcmp(libname + slen - 3, ".so".ptr, 3) == 0)
                {
                    strncat(buf, libname + 3, slen - 3 - 3);
                    argv.push(buf);
                }
                else
                {
                    strcat(buf, libname);
                    argv.push(buf);
                }
            }
            else
            {
                strcat(buf, libname);
                argv.push(buf);
            }
        }

[mihails-strasuns]

DMD instead seems to recognise specially formatted defaultlib/debuglib strings (see below), but I don't think requiring the user to repeat the libraries to use from the config files would necessarily be a good idea.

Yes, DMD approach is this regard is rather terrible and I would prefer to avoid anything like that if possible.

this is pretty much unacceptable from our side

So what would be your desired CLI? I don't care how it looks as long as it doesn't require developer to mention library name explicitly.

[dnadlinger]

this is pretty much unacceptable from our side

So what would be your desired CLI? I don't care how it looks as long as it doesn't require developer to mention library name explicitly.

Ah, I meant that us leaving this PR without comments is unacceptable, not the PR itself. ;)

Regarding alternatives, I'm not too fond of the use- prefix since it seems to be superfluous, but I don't have a much better proposal. link- maybe?

[mihails-strasuns]

Updated to --link-shared-defaultlib

[kinke]

This surely doesn't work with the MS linker...

[mihails-strasuns]

I neither have any idea how MS linker works nor have access to it. Someone else will need to implement support for it.

[kinke]

The command-line options reference is on MSDN. ;)
I've just mentioned this as that whole approach will not work for the MS linker. I guess we'll have to use a suffix (as the file extension is always .lib) for the shared Windows libs.
What also doesn't work for the MS linker is simply specifying additional lib directories for multilib builds - the linker is extremely dumb and doesn't check the architecture, it just uses the first lib it finds, so the order of the lib directories is crucial.
We'll have to think this through carefully and probably come up with an extended config file...

Edit: We may also switch to the LLVM linker (lld) at some point, at least on Windows, which apparently already supports COFF objects and PE output (but no debug-info PDBs yet). It can't be as dumb as the MS one.

[dnadlinger]

Signing out for now, please ensure that this goes somewhere, @redstar / @kinke / @JohanEngelen. ;)

[mihails-strasuns]

Just in case it wasn't clear - I am not planning work on Windows support (and neither see how it is relevant considering vast amount of differences in linking in general), My only interest is fixing a bug report for Arch Linux package.

[JohanEngelen]

For me, it would be OK to add this without Windows support for now. It'd be a shame if this is lost and perhaps someone in the future will add Windows support. It'd be good to #ifdef the cmdline option out of Windows builds though.

[mihails-strasuns]

Updated:

• version out added code to be posix-only

[JohanEngelen]

@Dicebot I'll see if I can spend some time in updating the CMake scrips for this, so we can automate testing of this PR

[mihails-strasuns]

btw travis build seems to fail with ld: unknown option: -Bstatic, any idea about it?

[JohanEngelen]

@Dicebot Can you rebase onto master? (There is a problem with Travis+LLVM38 that makes all Linux tests fail)

[JohanEngelen]

btw travis build seems to fail with ld: unknown option: -Bstatic, any idea about it?

ld on Mac is different from GNU ld and does not know -Bstatic

[JohanEngelen]

Perhaps the GNU equivalent option -static does the same on Mac.

Edit: with "GNU equivalent" I meant that GNU's ld accepts both -Bstatic and -static and they do the same thing.

[mihails-strasuns]

Rebased.

MacOS looks tricky if http://www.manpages.info/macosx/ld.1.html data is correct. It implies MacOS ld doesn't have any direct analog of -Bstatic / -Bdynamic at all as supposed replacements are defined to be mutually exclusive.

[JohanEngelen]

If it also doesn't work on Mac, I think @kinke is right and it is better solved with an extended config file, with different names for the shared and static libraries so that no special linker flag is needed.

[mihails-strasuns]

Right now ldc config looks like it only provides already existing command-line switches, for example mine looks like this:

default:
{
    // 'switches' holds array of string that are appends to the command line
    // arguments before they are parsed.
    switches = [
        "-I/usr/include/dlang/ldc/ldc",
        "-I/usr/include/dlang/ldc",
        "-L-L/usr/lib", 
        "-L-L/usr/lib32",
        "-L--no-warn-search-mismatch",
        "-defaultlib=lphobos2,ldruntime",
        "-debuglib=lphobos2-debug,ldruntime-debug"
    ];
};

How would you envision changing it?

[JohanEngelen]

I don't know. :/

[kinke]

Iirc, @redstar proposed something in the past. Something like a default section and sections for shared/static stdlibs (druntime & phobos) which override (or specify) the -defaultlib and -debuglib switches. And let's not forget about multilib (-m32/-m64).
We may reduce the config overhead by going with a fixed -debug suffix for the debug libs (and dumping the -debuglib switch). We may also simplify the multilib part by using 2 fixed directories (lib32 and lib64)...

[mihails-strasuns]

I want to resurrect this but would be nice to have approved config file design first because I keep disliking everything I come up with myself :)

So right now I have something along these lines:

default:
{
    switches = [ 
        // ...
    ];
    // debug version are hard-coded to have -dbg suffix  
    staticstdlib = "phobos-ldc";
    sharedstdlib = "sodruntime-ldc,sophobos-ldc";
};

... which seems to allow switching between shared/static modes in 100% cross-platform fashion, but:

1. Is damn ugly
2. I can't decide what would be best semantics for interacting with CLI -debuglib / -debuglib flag (override / conflict / get shadowed)

Any ideas?

[mihails-strasuns]

Anyone? :(

[kinke]

Do we really need the flexibility to specify the names of the stdlibs? I'd rather tend towards using a fixed, hardcoded scheme.
A -debug filename suffix for the debug variants would be fine by me.
For dynamic vs. static, another -shared suffix seems sufficient to me. On POSIX, we'd have libphobos2-ldc[-debug][-shared].(a|so), and on Windows phobos2-ldc[-debug][-shared].lib.

[kinke]

This in combination with an extended commandline interface to select 1 of the 4 stdlib flavours.

[kinke]

The 32/64 bit issue on Windows (can't specify both 32-bit and 64-bit lib directories and have the linker ignore incompatible libs) has been solved by an extended config file, allowing to specify the default commandline options on a per triple basis. E.g., the Windows CI multilib config looks like this:

// This configuration file uses libconfig.
// See http://www.hyperrealm.com/libconfig/ for syntax details.

// The default group is required
default:
{
    // 'switches' holds array of string that are appends to the command line
    // arguments before they are parsed.
    switches = [
        "-I%%ldcbinarypath%%/../include/d/ldc",
        "-I%%ldcbinarypath%%/../include/d",
        "-L-L%%ldcbinarypath%%/../lib64", 
        "-defaultlib=phobos2-ldc,druntime-ldc",
        "-debuglib=phobos2-ldc-debug,druntime-ldc-debug"
    ];
};

i686-pc-windows-msvc:
{
    // 'switches' holds array of string that are appends to the command line
    // arguments before they are parsed.
    switches = [
        "-I%%ldcbinarypath%%/../include/d/ldc",
        "-I%%ldcbinarypath%%/../include/d",
        "-L-L%%ldcbinarypath%%/../lib32", 
        "-defaultlib=phobos2-ldc,druntime-ldc",
        "-debuglib=phobos2-ldc-debug,druntime-ldc-debug"
    ];
};

This could obviously be further extended by pattern-matching/allowing wildcards in the triples, but it's sufficient for now.

[mihails-strasuns]

Do we really need the flexibility to specify the names of the stdlibs?

Not sure. I wouldn't be surprised if some of more strict distributions like Debian require specific naming style for all packaged libraries but I don't know certainly. Maybe @ximion can advise.

A -debug filename suffix for the debug variants would be fine by me.

How would that interact with existing -debuglib switch?

[kinke]

How would that interact with existing -debuglib switch?

It could be deprecated (and ignored).

[mihails-strasuns]

Sounds good to me but I'd like to hear confirmation from at least one more person from LDC team before proceeding with deprecating (any) functionality :)

[ximion]

@Dicebot

Not sure. I wouldn't be surprised if some of more strict distributions like Debian require specific naming style for all packaged libraries but I don't know certainly. Maybe @ximion can advise.

We usually don't require any specific naming (unless of course there are name clashes between two projects). Since D doesn't have an ABI GDC and LDC both obey, we do need different names for all D libraries depending on whether they are compiled with LDC or GDC (which really sucks a lot).
Fortunately, LDC names its Phobos correctly already, so, at least on the library side, there is no issue here.

[mihails-strasuns]

Closing in favor of #1735