[WIP] Generate C++ header files from public extern (C++) declarations

[yebblies]

If I can get this to work, it means no more maintaining header files for the frontend.

This generates one mega-header, since it needs to reorder declarations to avoid forward references. I need to work out what to do about declarations that can't be automatically converted but are referenced from the converted ones, like Array and RootObject.

[ibuclaw]

While on the one hand I see this as not being friendly with dmd's c++ mangling rules.
#define _d_long long long

I see this as a recipe to break existing ports of D.
#define _d_int int

Not that they weren't already broken because of the switch. But I think the correct thing to do is to map all types to fixed sized integers. That should guarantee that data passing between the two sides should work.

[yebblies]

Defining them to anything other than what dmd uses for mangling will break (d)dmd, so it's not going to happen. It's only set up for win32 until I get around to fixing the other platforms. win32 is currently working perfectly.

[ibuclaw]

Yep. I'm probably thinking too hard about this, but perhaps I should get round to adding some kind of Target::mangleType hook to take care of the special cases. Ignoring all basic types - off the top of my head, ARM EABI mangles va_list as if it was in the C++ std namespace, ugh! :-D

[JohanEngelen]

Forgot to modify the comment

[yebblies]

Yup.

[yebblies]

@ibuclaw Ping on the gdb failures

[ibuclaw]

@yebblies - not related to gdb, but FYI.

c++ -c -Wno-deprecated -Wstrict-aliasing -fno-exceptions -fno-rtti -D__pascal= -DMARS=1 -DTARGET_LINUX=1 -DDM_TARGET_CPU_X86=1 -m64 -std=gnu++98 -Iroot -Itk -Ibackend -MMD -MF e2ir.deps e2ir.c
In file included from e2ir.c:19:0:
frontend.h:10617:0: warning: "UINT64_MAX" redefined
 #define UINT64_MAX 18446744073709551615LLU

In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/7.0.0/include/stdint.h:9:0,
                 from root/port.h:17,
                 from root/outbuffer.h:17,
                 from root/root.h:23,
                 from e2ir.c:16:
/usr/include/stdint.h:169:0: note: this is the location of the previous definition
 # define UINT64_MAX  (__UINT64_C(18446744073709551615))

This is because of the following in intrange.d

enum UINT64_MAX = 0xFFFFFFFFFFFFFFFFUL;

This should be removed. (You have ulong.max in D :-)

[ibuclaw]

I can reproduce gdb failure on gdb-7.11.

[ibuclaw]

Here is your problem:

Contents of the .debug_info section:

  Compilation Unit @ offset 0x0:
   Length:        0xfd (32-bit)
   Version:       3
   Abbrev Offset: 0x0
   Pointer Size:  8
 <0><b>: Abbrev Number: 1 (DW_TAG_compile_unit)
    <c>   DW_AT_producer    : Digital Mars D 2.0xx�runnable/gdb4149.d
    <34>   DW_AT_language    : 47   (Unknown: 2f)
    <35>   DW_AT_name        : home/ibuclaw/src/dmd/dmd/test
    <53>   DW_AT_comp_dir    : 
    <54>   DW_AT_low_pc      : 0x0
    <5c>   DW_AT_entry_pc    : 0x0
    <64>   DW_AT_ranges      : 0x0
    <68>   DW_AT_stmt_list   : 0x2000000
 <1><6c>: Abbrev Number: 103

[ibuclaw]

After that:

objdump: Warning: DIE at offset 0x6c refers to abbreviation number 103 which does not exist

[wilzbach]

If I can get this to work, it means no more maintaining header files for the frontend.

I don't follow dmd for that long, but this seems to me a serious improvement as forgetting to update the header files seems to be a common problem around here ;-)
@yebblies

1. What's your status on this PR? What's blocking/missing? (apart from the current merge conflict)
2. How will you ensure that new features in D don't accidentally break the automatic header generation? (I couldn't find any tests, but apart from testing linking with LDC I can't think of a good way to test?)

[yebblies]

It's not quite that big a deal, because the autotester catches most broken changes.

This PR is (or was) mostly complete, but is a little messy when it comes to platform differences. There is also hard-coded exclusion of ddmd.root which needs to be re-thought.

I don't know what you mean by new features breaking the generation. The generator ignores everything except extern(C/C++) declarations which generally aren't affected by new features. As well as that, the autotester would catch it if it started generating broken C++ code (hopefully).

[UplinkCoder]

hmm .... C header generations seems useful, I might adopt this one.

[yebblies]

It's pretty messy - I'm hoping to push this out to an external compiler-as-a-library tool.

[UplinkCoder]

hmm ... it only needs to be concerned with extern C/C++ stuff.

[yebblies]

Yeah, and it can be done pretty well without semantic analysis. I mean, I did it like that in the other direction, and this doesn't need to do statements/expressions. The part I'm not so keen on having in the compiler is the special cases and rewrites that inevitably come up. Eg which modules to export, things to exclude, translations for variable sized types, automatic header includes, etc

[UplinkCoder]

I hear you.
The special cases suck the life right out of you.
Still I think it's not necessary to deal with those immediately.
Having some c-hdrgen is even if incomplete is better then the status quo.

[RazvanN7]

I'm currently working on this using the parser library I just merged. I'm starting with the simple case: top level functions, classes, structs and variables that are declared extern(C++). This should be extremely simple. After that, I'm going to tackle more complex cases like having an extern function in a class that's not extern.

Ideas and help would be appreciated. @UplinkCoder

[UplinkCoder]

@RazvanN7 If you push your progress to a visible branch, I can have a look.
It is my believe that the header-generation can mostly stay clear of complicated things.
Since you would not be able to interface with those using C anyway.

[ibuclaw]

Hmm, no need to do anything that isn't toplevel. The only complex things that need handling are:

• Type conversion (D long -> C long)
• Templates (template FPTypeProperties(T) -> template <typename T> struct FPTypeProperties)
• Forward declarations.

And even then, they aren't even that complex to handle IMO. Just make sure you emit them in the correct order. :-)

[ibuclaw]

Revamped this in #8591