[WIP] Generate C++ header files from public extern (C++) declarations by ibuclaw · Pull Request #8591 · dlang/dmd

ibuclaw · 2018-08-21T23:45:45Z

Takes #5082 and converts it into a standalone program.

Still needs a bit of work, documentation, command-line option handling maybe to turn on/off certain parts.

Some bugs that I reported are still present, such as #5082 (comment)

Using the auto-generated header in the C++ unittest works at least.

dlang-bot · 2018-08-21T23:45:47Z

Thanks for your pull request, @ibuclaw!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub fetch digger
dub run digger -- build "master + dmd#8591"

jacob-carlborg · 2018-08-22T07:01:48Z

I would recommend using the Dub package. Also building the new tool as a Dub package.

wilzbach

Yeah it would save a lot of code in the main method if this would use the DMD dub package.

wilzbach · 2018-08-22T10:17:36Z

src/dmd/dtoh.d

+    "safe.d", "sapply.d", "semantic2.d", "semantic3.d", "sideeffect.d", "statement.d",
+    "statementsem.d", "staticassert.d", "staticcond.d",
+    "target.d", "templateparamsem.d", "tokens.d", "traits.d", "typesem.d", "typinf.d",
+    "utf.d", "utils.d",


Can't we use globbing here like we do for the dub package or build.d, s.t. we have less files to update whenever a file gets moved?

Maybe, this part was carried over from the old PR.

For sure am open to design improvements - the most important part is the blacklisting of certain modules that we retain manual updates for.

src/dmd/dtoh.d

wilzbach · 2018-08-22T10:23:54Z

src/dmd/dtoh.d

+                // Generate special static inline function.
+                if (cd.isIdentifierClass())
+                {
+                    buf.writestring("    static inline Identifier *idPool(const char *s) { return idPool(s, strlen(s)); }\n");


Why this special casing?

Because this doesn't exist in D for obvious reasons.

Oh I was more asking: shouldn't we do this in a more general-purpose way.

src/dmd/dtoh.d

wilzbach · 2018-08-22T10:31:26Z

src/dmd/dtoh.d

+    global.params.isLP64 = global.params.is64bit;
+
+    global.path = new Strings();
+    global.path.push(__FILE_FULL_PATH__.dirName.buildPath("../../../druntime/src/").toStringz());


This is fragile if druntime is checkout out in a different directory (I think the LDC people do this).

Yes our path is $LDCROOT/runtime/druntime.

wilzbach · 2018-08-22T10:34:08Z

SemaphoreCI failure is due to the dub package trying to include dtoh:

../../src/dmd/dtoh.d(103,23): Deprecation: Using super as a type is deprecated. Use typeof(super) instead
../../src/dmd/dtoh.d(131,23): Error: function `dmd.dtoh.genCppFiles.ToCppBuffer!(ASTCodegen).ToCppBuffer.visit` conflicts with alias dmd.dtoh.genCppFiles.ToCppBuffer!(ASTCodegen).ToCppBuffer.visit at ../../src/dmd/dtoh.d(103,9)
../../src/dmd/dtoh.d(1140,19): Error: template instance `dmd.dtoh.genCppFiles.ToCppBuffer!(ASTCodegen)` error instantiating

jacob-carlborg · 2018-08-22T19:14:24Z

Related [1]. Perhaps join forces?

[1] https://forum.dlang.org/thread/oikrrejcengbjivsyzws@forum.dlang.org

jacob-carlborg · 2018-08-22T19:20:45Z

@TurkeyMan this should be interesting for you.

ibuclaw · 2018-08-22T23:32:11Z

@jacob-carlborg this differs in that it is not intended to be a general purpose tool. Same as the tool used to convert dmd to D. This handles only dmd specific style and quirks.

kinke · 2018-08-23T22:13:08Z

Nice job Iain, thanks!

PetarKirov · 2018-08-29T10:19:26Z

It would be better to make the hard-coded lists and paths command-line / or environment parameters.

src/dmd/intrange.d

src/dmd/dtoh.d

TurkeyMan · 2018-09-29T21:00:17Z

What's going on with this? I could really use this like, yesterday.

ibuclaw · 2018-09-29T23:01:07Z

@TurkeyMan - it's hardly meant for general purpose, there are many special cases that look for dmd-specific code.

If it gets used at all, it'll be part of the build process for generating headers that gdc and ldc can include. No need to check what is and isn't in sync.

TurkeyMan · 2018-09-29T23:49:52Z

Right... I want to do an experiment where I define some struct's and functions in D, but they are extern(C++), and the .h file needs to be emit by the D compiler. Is that the goal here? It sounds like the same goal; keeping the C++ headers in sync.
This would be a massive enabler for my current project.

thewilsonator · 2018-09-30T07:11:51Z

@TurkeyMan I'll do this up as a dub package once #8776 goes in.

WalterBright · 2018-10-26T09:57:55Z

Is this still a WIP?

thewilsonator · 2018-10-26T10:46:33Z

This needs:

the druntime path to be fixed for LDC
made to work for non-linux, and 32-bit

It would also be good to have the DMD-specific stuff factored out.

ibuclaw · 2018-10-26T11:14:47Z

Is this still a WIP?

Yes, as I have done no testing whatsoever against gdc and auto-generated headers.

I've been too busy with work on other things to look back at this (you should know what other things are :-)

The cxxfrontend tests that are in dmd only cover the surface of the C++ bindings. That the generated header is compilable and the running program passes all tests is promising though.

As I've said before, this not meant to be general purpose, I would suggest maybe dpp is better suited for you than this utility tool.

thewilsonator · 2018-10-26T11:21:33Z

As a general purpose dub-package-to-be: https://github.com/thewilsonator/dtoh

N.B. also completely untested.

jacob-carlborg · 2018-10-26T13:15:52Z

I would suggest maybe dpp is better suited for you than this utility tool.

dpp is for the other way.

TurkeyMan · 2018-10-26T17:21:16Z

As I've said before, this not meant to be general purpose, I would suggest maybe dpp is better suited for you than this utility tool.

dpp is the wrong way around; we want to emit .h files with the extern(C++) symbols like .di files work today... in this case, the D code should be the authoritative code. I don't want C++ injecting noise into this particular codebase.

Is there a path to making this work general purpose in the way I say?
extern(C++) doesn't support anything that C++ doesn't support; the mapping should be cleanly possible.

For DMDFE as a library so that tools can populate `global.params` correctly. Required for https://github.com/thewilsonator/dtoh the dub package of dlang#8591

Factor out compiler specific parts Attempt to make cross platform

wilzbach · 2019-02-25T01:12:05Z

src/dmd/dtoh.d

+    }
+    // Read the configurarion file
+    auto inifile = File(global.inifilename);
+    inifile.read();


I think most of this code in this main method could be saved by using DMD's dmd.frontend:

https://github.com/dlang/dmd/blob/master/src/dmd/frontend.d

https://github.com/dlang/dmd/blob/master/test/dub_package/frontend.d

thewilsonator · 2019-06-10T04:26:32Z

There is #9971 now, can this be closed?

ibuclaw · 2019-06-10T04:53:10Z

If they're willing to take the baton in this relay race.

wilzbach · 2019-06-10T07:21:23Z

It's Edi's GSoC project, so he'll be working full-time on this over the next weeks.

ibuclaw added the Review:WIP Work In Progress - not ready for review or pulling label Aug 21, 2018

ibuclaw requested review from MartinNowak, RazvanN7, dnadlinger and wilzbach as code owners August 21, 2018 23:45

ibuclaw mentioned this pull request Aug 21, 2018

[WIP] Generate C++ header files from public extern (C++) declarations #5082

Closed

ibuclaw force-pushed the dtoh branch 3 times, most recently from fd44e79 to 2f24603 Compare August 22, 2018 00:05

wilzbach reviewed Aug 22, 2018

View reviewed changes