fix Issue 10711 - shared phobos library should not depend on _Dmain

[WalterBright]

http://d.puremagic.com/issues/show_bug.cgi?id=10711

This moves code that was formerly in druntime's src/rt/entrypoint.d into the compiler itself.

[WalterBright]

Once it's pulled, I'll remove entrypoint.d from druntime.

[jacob-carlborg]

Alternatively we could compile entrypoint.d as a separate object file/library which is only linked when an executable is built. I don't know which approach is best.

[WalterBright]

Alternatively we could compile entrypoint.d as a separate object file/library which is only linked when an executable is built.

I felt that having yet another library would be confusing to users.

[yebblies]

Another library might be, but a single object file inside druntime.lib should do this correctly without any hardcoding in the compiler.

The linker should be capable of only pulling in the object file if main is needed, right?

[WalterBright]

Also needs dlang/druntime#572

[WalterBright]

Another library might be, but a single object file inside druntime.lib should do this correctly without any hardcoding in the compiler. The linker should be capable of only pulling in the object file if main is needed, right?

That works fine as long as druntime is a library. When it's a DLL, then that code is always there, and as the bug report says, _Dmain will be undefined if you attempt to load it from a C++ program.

[yebblies]

Doesn't that mean you just exclude the object file when building druntime as a dll? We have to link it differently anyway, so there it doesn't need to be the exact same objects. (I could be wrong here)

[WalterBright]

Doesn't that mean you just exclude the object file when building druntime as a dll?

Then it doesn't work with a D main(). libphobos.so must work when loaded by both a C++ program or a D program.

[jacob-carlborg]

Then it doesn't work with a D main().

Why not?

[ibuclaw]

Doesn't that mean you just exclude the object file when building druntime as a dll? We have to link it differently anyway, so there it doesn't need to be the exact same objects. (I could be wrong here)

So you are proposing there should be two libraries... libdruntime.so and libdruntime_nocmain.so?

Throwing in a third possibility. I had initially envisioned that cmain would be in it's own object file that gets pushed into the linker command at the end of compilation, eg:

dmd -v foo.d
=> dmd -c foo.d -o foo.o
=> dmd -I/path/to/dmd/private/stuff cmain.o foo.o -lrt -lphobos -o foo

This wouldn't be different from gcc with compiling in crtstart.o and crtend.o into C programs.

[WalterBright]

Why not?

Because it says "_Dmain is undefined" when you try to link a C++ program that does not have a D main() in it to libphobos.so, because libphobos.so will have a reference to _Dmain. Look at entrypoint.d, it references _Dmain.

[jacob-carlborg]

@ibuclaw That's exactly what I suggested, or at least tried to.

[jacob-carlborg]

@WalterBright The C main and D main functions should be in the same object file.

[WalterBright]

The C main and D main functions should be in the same object file.

That's what this pull does.

[jacob-carlborg]

That's what this pull does.

_Dmain is still in druntime. Actually, as far as I can see, with this pull request and dlang/druntime#572 you don't have a reference to _Dmain at all.

[jacob-carlborg]

We have a couple of uses case here:

1. Building a D executable that is statically linked
2. Building a D executable that is dynamically linked
3. Building a D executable that is linked dynamically to a D library
4. Building a C executable that links with a static D library
5. Building a C executable that links with a dynamic D library

(Have I missed a uses case?)

Necessary libraries/object files:

• libphobos.a doesn't contain any main functions
• libphobos.so doesn't contain any main functions
• entrypoint.o only containing C and D main functions, including reference to _Dmain. This object file is built from a D module in druntime

This is how it will link with the different use cases:

1. Link with libphobos.a and entrypoint.o
2. Link with libphobos.so and entrypoint.o
3. The dynamic library (foo) is linked with libphobos.so. The executable is linked with libphobos.so, entrypoint.o and libfoo.so
4. Link with the D library and libphobos.a. The C code provides the C main function that will call rt_init
5. The dynamic library (foo) is linked with libphobos.so. The executable is linked with libphobos.so and libfoo.so. The C code provides the C main function that will call rt_init

[WalterBright]

_Dmain is still in druntime. Actually, as far as I can see, with this pull request and dlang/druntime#572 you don't have a reference to _Dmain at all.

Exactly, which is why it fixes the problem.

[jacob-carlborg]

Exactly, which is why it fixes the problem.

What happens then when you try to build an executable without providing a D main function?

[WalterBright]

The executable is linked with libphobos.so, entrypoint.o

I chose a method that makes it unnecessary to deal with entrypoint.o. The idea is to minimize user-facing problems.

[WalterBright]

What happens then when you try to build an executable without providing a D main function?

When linking with libphobos.so, you get a lovely message "_Dmain is undefined" from the linker, as http://d.puremagic.com/issues/show_bug.cgi?id=10711 shows.

[jacob-carlborg]

You get a lovely message "_Dmain is undefined" from the linker.

How is that possible when _Dmain isn't referenced anywhere? I thought the reason for the linker error was this:

https://github.com/D-Programming-Language/druntime/blob/master/src/rt/entrypoint.d#L27

If you remove that you won't get any linker errors?

[WalterBright]

Note the use case - writing a C++ program and loading "plugin" DLLs written in D. There is no D main() or D entry point - yet it must still work.

[jacob-carlborg]

See my comment above: #2476 (comment)

[WalterBright]

How is that possible when _Dmain isn't referenced anywhere?

It's referenced from entrypoint.d.

If you remove that you won't get any linker errors?

And then your D programs all fail because libphobos2.so is never initialized and your D main() never gets called.

[WalterBright]

See my comment above: #2476

And my reply: I chose a method that makes it unnecessary to deal with entrypoint.o. The idea is to minimize user-facing problems.

[jacob-carlborg]

It's referenced from entrypoint.d.

You are removing it with this pull request: dlang/druntime#572 and I can't see it being added by this one.

And then your D programs all fail because libphobos2.so is never initialized.

Yes, exactly, but as I said above, you're removing it.

[WalterBright]

@jacob-carlborg I just don't know what you're getting at. Yes, this pull and the druntime one work together to REMOVE the entrypoint code from the library and puts it in the compiler. That way it's there when it's needed and not there when it isn't.

[jacob-carlborg]

And my reply: I chose a method that makes it unnecessary to deal with entrypoint.o. The idea is to minimize user-facing problems.

If it wasn't obvious, the compiler will automatically link with entrypoint.o if it builds an executable. I don't see what the user-facing problems are.

[jacob-carlborg]

I just don't know what you're getting at. Yes, this pull and the druntime one work together to REMOVE the entrypoint code from the library and puts it in the compiler.

1. I just suggested an alternative, avoiding having the compiler generating the main functions
2. As far as I can understand this pull request and the one for druntime will cause a crash at application start up if you fail to provide a main function for a D executable. Yes, I've tried this before using this: https://github.com/D-Programming-Language/druntime/blob/master/src/rt/dylib_fixes.c

[WalterBright]

1. I just suggested an alternative, avoiding having the compiler generating the main functions

Yes, and I replied: I chose a method that makes it unnecessary to deal with entrypoint.o. The idea is to minimize user-facing problems.

The user-facing problems is he has to add entrypoint.o in his makefiles (if he is linking with other than dmd) and entrypoint.o must be generated in all its incarnations, and inserted into the right locations on the user's machine.

1. As far as I can understand this pull request and the one for D main will cause a crash at application start up if you fail to provide a main function for a D executable

On Windows:

OPTLINK (R) for Win32  Release 8.00.13
Copyright (C) Digital Mars 1989-2010  All rights reserved.
http://www.digitalmars.com/ctg/optlink.html
OPTLINK : Warning 23: No Stack
OPTLINK : Warning 134: No Start Address

No crash.

[yebblies]

This sizeof is not auto-portable to D. sizeof(code) / sizeof(code[0]) will work, or strlen(code)

[WalterBright]

I'm not aware of any case where sizeof(code) will not work, as 'char' in any usable (with D) C++ compiler has size 1.

[yebblies]

In D sizeof(code) will give 4, as code is a char *.

[ibuclaw]

sizeof(code) -> code.length :o)

[WalterBright]

Another user facing problem is linking:

dmd a.o b.o

Does dmd add in entrypoint.o or not? dmd doesn't look inside .o files that it is linking to see if there's a _Dmain in them. Those files may contain a C main() and the user is doing his own initialization. If those files are creating a plugin, adding in entrypoint.o will cause errors.

[jacob-carlborg]

Ok, I got a linker error for the C main function:

Undefined symbols for architecture x86_64:
  "_main", referenced from:
      start in crt1.10.6.o
     (maybe you meant: _D4core6thread6Thread7sm_mainC4core6thread6Thread)
ld: symbol(s) not found for architecture x86_64
collect2: ld returned 1 exit status
--- errorlevel 1

I guess that's ok.

[WalterBright]

I guess that's ok.

It's the same as with dmd before this change.

[jacob-carlborg]

It's the same as with dmd before this change.

No, currently, without this change, it complains about an undefined _Dmain symbol.

[WalterBright]

On windows I get (with the old dmd) the same no start address message. I suppose Linux is different.

[ibuclaw]

@WalterBright

1. With this and the other pull, I guess it would be an error message like:

crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
collect2: error: ld returned 1 exit status

Which instantly tells you 'whoops' I forgot to put in an int/void main.

2. Whether or not the compiler adds entrypoint.o should depend on whether or not we are building a library, which can be done in two ways.

a. Static library => dmd -c a.d b.d; ar a.o b.o -o mylib.a; ranlib mylib.a
As the compiler is not doing the linking, entrypoint.o is not included.

b. Shared library => dmd -shared a.d b.d -o mylib.so
As the compiler is not producing an executable, entrypoint.o is not included.

Hypothetically there could be a rare instance when the user for whatever reason puts in their own extern(C) main inside a source file (best example I can think of is an exokernel - which there are two or three that exist written in D), for this case we could either say that:

• They shouldn't be doing this (probably not a great idea to educate the user).
• Have some sort of -noentrypoint style switch which also omits entrypoint.o from being linked in to the executable.

[WalterBright]

@ibuclaw or we could have the D compiler itself generate the entrypoint code, and then there's no need to add a compiler switch, add more documentation explaining it, manage entrypoint.o, add complex logic to the compiler to decide whether or not to add in entrypoint.o, tell people they have to modify their makefiles, answer user questions about what entrypoint.o is and why their setup can't find it, etc. It's 3 lines of code.

[WalterBright]

@ibuclaw dmd can generate library files directly - there's no need to use ar or ranlib.

Also, shared libraries certainly can have a D main() in them. Granted, this is unusual.

I develop with my own extern(C) main in a D program when I am porting D to a new platform. It's usually because there's no phobos library to link with yet.

As for dmd generating functions, it already does that in several other cases. Why not add the necessary wrapper code if a D main() is encountered?

[ibuclaw]

@WalterBright - don't doubt you for a moment there. I've never honestly looked at what dmd can/can't do in link.c - infact I've never even looked at link.c...

Shared libraries could have a D main, but D main does not depend on entrypoint.o, so there is no problem with this. A program written in C/C++ could have the following which calls the D main from the library.

#include <druntime.h>
int main()
{
  rt_init();
  Dmain();
  rt_term();
}

However if the compiler generates and emits a C main upon seeing a D main...

When porting to a new platform, there's chance that there's no druntime library to link with either - For example I started off on ARM this way before getting things to a usable level, then added in druntime, then phobos into the library. In Debian ports, there are also no library included with the D compiler for them (SPARC, IA64, PPC, MIPS), and I expect a similar thing to be done when we finally get round to porting druntime/phobos to these platforms. So automatically generating a C main becomes unhelpful in the event that no _d_run_main exists in the library/no library to link to.

I assume other cases you are referring to array ops / typeinfo (eg: core functionality)?

[MartinNowak]

It should take the address of _Dmain not main. Also the declaration extern(C) int _Dmain(char[][] args); is missing.

[WalterBright]

It should take the address of _Dmain not main

The symbol is referred to as "main" inside the D code. _Dmain is unknown.

Also the declaration extern(C) int _Dmain(char[][] args); is missing.

No, it's not, as this code is compiled in the context of D main().

[MartinNowak]

Ah, just saw that while testing. Works fine.

[9rnsr]

This change broke my pull#2130. In the context of the module which declares D main(), &main is essentially ambiguous between extern(C) main and extern(D) main.
I'm trying to fix the issue in #2130. @WalterBright , could you please see my fix?

[9rnsr]

@WalterBright I opened a PR #2540 for better C-main addition.

[MartinNowak]

Does dmd add in entrypoint.o or not? dmd doesn't look inside .o files that it is linking to see if there's a _Dmain in them. Those files may contain a C main() and the user is doing his own initialization. If those files are creating a plugin, adding in entrypoint.o will cause errors.

Similar argument, when you compile an object file with a D main function but use a different linker then entrypoint.o would be missing, i.e. we would force people to use dmd for linking which would likely break some build tools.

When porting to a new platform, there's chance that there's no druntime library to link with either.

Then you can't have a D main but have to start with a C main.

Despite the issue the change looks good.

[ibuclaw]

Then you can't have a D main but have to start with a C main.

-Wl,-e_Dmain :o)

[MartinNowak]

-Wl,-e_Dmain :o)

Then you'd miss all kind of startup code and C initialization.

So automatically generating a C main becomes unhelpful in the event that no _d_run_main exists in the library/no library to link to.

If you don't have a runtime you can easily start from an earlier entry point like C's main or define a custom one.

[ibuclaw]

Change is still borderline compiler specific as it depends on that the rt/ modules match what dmd comes shipped with, with a leniency towards that we'll be linking against a shared library (something that for the time being, absolutely zero of your work on shared library/dso has been merged in and it will never be merged in as far as the forseeable future holds).