Revive Generate C++ header files from public extern (C++) declarations by edi33416 · Pull Request #9971 · dlang/dmd

edi33416 · 2019-06-04T16:53:47Z

This PR takes #8591, rebases with master and converts it into a compiler switch.
This is a separate PR as I don't have the rights to push to existing dmd PRs.

Credit to @ibuclaw for the actual work.

dlang-bot · 2019-06-04T16:53:50Z

Thanks for your pull request and interest in making D better, @edi33416! We are looking forward to reviewing it, and you should be hearing from a maintainer soon.
Please verify that your PR follows this checklist:

My PR is fully covered with tests (you can see the coverage diff by visiting the details link of the codecov check)
My PR is as minimal as possible (smaller, focused PRs are easier to review than big ones)
I have provided a detailed rationale explaining my changes
New or modified functions have Ddoc comments (with Params: and Returns:)

Please see CONTRIBUTING.md for more information.

If you have addressed all reviews or aren't sure how to proceed, don't hesitate to ping us with a simple comment.

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub fetch digger
dub run digger -- build "master + dmd#9971"

andralex · 2019-06-04T17:09:11Z

This is for GSoC - @edi33416 please reach out to @ibuclaw about what's still to be done.

andralex · 2019-06-04T17:11:12Z

Bootstrapping is an issue now - we need to compile the compiler before using it to generate the headers. @edi33416 please make sure the compiler does not need the headers.

jacob-carlborg · 2019-06-04T17:41:03Z

I guess the same comments that were made in the original PR apply here as well.

wilzbach · 2019-06-04T18:34:39Z

Bootstrapping is an issue now - we need to compile the compiler before using it to generate the headers.

That is no concern. D already requires a host D compiler (>= 2.079) to be built and has no dependence on C++ anymore.

src/dmd/dtoh.d

ibuclaw · 2019-06-10T04:51:59Z

As far as I'm aware, no testing of this has been done beyond the cxxfrontend unittests, which only covers a small amount of the actual use of the c++ api. So integration with gdc and ldc - as well as bootstrapping - would also be required as a preliminary requirement.

Secondly, we should have a way to red flag any prs that modify the output of the c++ headers, so that extra checking can be done to verify its correctness.

src/dmd/compiler.d

wilzbach · 2019-07-16T13:59:09Z

test/compilable/extra-files/dtoh-postscript.sh

+
+diff -pu --strip-trailing-cr ${EXTRA_FILES}/${TEST_NAME}.out ${OUTPUT_BASE}.out
+
+rm -f ${OUTPUT_BASE}.out


Is there really a need for removing this file? Might be useful for debugging when the test failed?

The test suite will complain if any files are left over with the shell tests (which I despise). Also this should be made in to a D test rather than a shell one. The less shell tests the better.

No it won't. It will only complain if the files aren't placed in the test_results directory.

src/dmd/dtoh.d

wilzbach · 2019-07-16T14:03:54Z

src/dmd/dtoh.d

+//}
+
+/****************************************************
+ */


actual description

src/dmd/dtoh.d

wilzbach · 2019-07-16T14:10:10Z

Secondly, we should have a way to red flag any prs that modify the output of the c++ headers, so that extra checking can be done to verify its correctness.

This could be done by simply checking in the generated DMD frontend C++ headers into the source code and in one CI checking whether the output has been changed. This way all future changes to the C++ headers will be visible and I'm sure we can either make the test work with AUTO_UPDATE=1 or probably a simple update command, s.t. it's no big hassle for people when they touch the headers.
Additionally, this would make bootstrapping for LDC/GDC even easier.

TurkeyMan · 2020-02-16T22:34:23Z

Let's make this PR happen people!
Needs a rebase... are we happy to merge it in prototype form, then it can be improved incrementally going forward? I would contribute to this if it were merged.

MoonlightSentinel · 2020-02-16T22:51:24Z

The initial version could be limited to the test suite without the changes to existing C++ headers/Tests. That would yield a much smaller change and avoid breaking stuff while being WIP.

@disable

The -HC switch let's dmd output header files from existing extern(C++) files to ease C++ interoperation. Use -HCd=<directory> / -HCf=<file> to select the directory/file where the header should be generated. The following commits were squashed from dlang#9971 without the changes to the existing C++ interface/test suite: Testing D to H tool Remove unused and redundant dmd headers Improve dtoh Factor out compiler specific parts Attempt to make cross platform Fix stuff Add dtoh as a compiler switch Update code after rebase Compile frontend srcs and write to flie Add dtoh enum test Add free functions dtoh test Add unittest block dtoh test Add VarDeclaration dtoh test Add AliasDeclaration dtoh test Add AnonDeclaration dtoh test Add StructDeclaration dtoh test Add ClassDeclaration dtoh test Add TemplateDeclaration dtoh test Add debug prints Fix struct field void initializer Add ignoredModule tmp helper Add tmp workaround for alias forward decl and >> template issue Add support for enum base type Add support for alignment of pow2 Fix alignment issues Generate enums based on -extern-std version Update tool auto-gen message Update tests Code cleanup CPP does not support covariant const methods Correctly write ctor declarations Generate default args for literals Generate dtor and copy ctor declarations Rebase with master Add compiler switch to cli.d Update tests Add support for @disable Improve default ctor generation Update struct decl test Use member initializer list for default ctor definition Co-Authored-By: Eduard Staniloiu <edi33416@gmail.com> (Rebase + Fixup) Co-Authored-By: MoonlightSentinel <moonlightsentinel@disroot.org>

@disable

The -HC switch let's dmd output header files from existing extern(C++) files to ease C++ interoperation. Use -HCd=<directory> / -HCf=<file> to select the directory/file where the header should be generated. The following commits were squashed from dlang#9971 without the changes to the existing C++ interface/test suite: Testing D to H tool Remove unused and redundant dmd headers Improve dtoh Factor out compiler specific parts Attempt to make cross platform Fix stuff Add dtoh as a compiler switch Update code after rebase Compile frontend srcs and write to flie Add dtoh enum test Add free functions dtoh test Add unittest block dtoh test Add VarDeclaration dtoh test Add AliasDeclaration dtoh test Add AnonDeclaration dtoh test Add StructDeclaration dtoh test Add ClassDeclaration dtoh test Add TemplateDeclaration dtoh test Add debug prints Fix struct field void initializer Add ignoredModule tmp helper Add tmp workaround for alias forward decl and >> template issue Add support for enum base type Add support for alignment of pow2 Fix alignment issues Generate enums based on -extern-std version Update tool auto-gen message Update tests Code cleanup CPP does not support covariant const methods Correctly write ctor declarations Generate default args for literals Generate dtor and copy ctor declarations Rebase with master Add compiler switch to cli.d Update tests Add support for @disable Improve default ctor generation Update struct decl test Use member initializer list for default ctor definition Co-Authored-By: Eduard Staniloiu <edi33416@gmail.com> (Rebase + Fixup) Co-Authored-By: MoonlightSentinel <moonlightsentinel@disroot.org>

@disable

The -HC switch let's dmd output header files from existing extern(C++) files to ease C++ interoperation. Use -HCd=<directory> / -HCf=<file> to select the directory/file where the header should be generated. The following commits were squashed from dlang#9971 without the changes to the existing C++ interface/test suite: Testing D to H tool Remove unused and redundant dmd headers Improve dtoh Factor out compiler specific parts Attempt to make cross platform Fix stuff Add dtoh as a compiler switch Update code after rebase Compile frontend srcs and write to flie Add dtoh enum test Add free functions dtoh test Add unittest block dtoh test Add VarDeclaration dtoh test Add AliasDeclaration dtoh test Add AnonDeclaration dtoh test Add StructDeclaration dtoh test Add ClassDeclaration dtoh test Add TemplateDeclaration dtoh test Add debug prints Fix struct field void initializer Add ignoredModule tmp helper Add tmp workaround for alias forward decl and >> template issue Add support for enum base type Add support for alignment of pow2 Fix alignment issues Generate enums based on -extern-std version Update tool auto-gen message Update tests Code cleanup CPP does not support covariant const methods Correctly write ctor declarations Generate default args for literals Generate dtor and copy ctor declarations Rebase with master Add compiler switch to cli.d Update tests Add support for @disable Improve default ctor generation Update struct decl test Use member initializer list for default ctor definition Co-Authored-By: Eduard Staniloiu <edi33416@gmail.com> (Rebase + Fixup) Co-Authored-By: MoonlightSentinel <moonlightsentinel@disroot.org>

@disable

The -HC switch let's dmd output header files from existing extern(C++) files to ease C++ interoperation. Use -HCd=<directory> / -HCf=<file> to select the directory/file where the header should be generated. The following commits were squashed from dlang#9971 without the changes to the existing C++ interface/test suite: Testing D to H tool Remove unused and redundant dmd headers Improve dtoh Factor out compiler specific parts Attempt to make cross platform Fix stuff Add dtoh as a compiler switch Update code after rebase Compile frontend srcs and write to flie Add dtoh enum test Add free functions dtoh test Add unittest block dtoh test Add VarDeclaration dtoh test Add AliasDeclaration dtoh test Add AnonDeclaration dtoh test Add StructDeclaration dtoh test Add ClassDeclaration dtoh test Add TemplateDeclaration dtoh test Add debug prints Fix struct field void initializer Add ignoredModule tmp helper Add tmp workaround for alias forward decl and >> template issue Add support for enum base type Add support for alignment of pow2 Fix alignment issues Generate enums based on -extern-std version Update tool auto-gen message Update tests Code cleanup CPP does not support covariant const methods Correctly write ctor declarations Generate default args for literals Generate dtor and copy ctor declarations Rebase with master Add compiler switch to cli.d Update tests Add support for @disable Improve default ctor generation Update struct decl test Use member initializer list for default ctor definition Co-Authored-By: Eduard Staniloiu <edi33416@gmail.com> (Rebase + Fixup) Co-Authored-By: MoonlightSentinel <moonlightsentinel@disroot.org>

moon-chilled · 2020-02-23T02:29:07Z

Superseded by #10801, so this should probably be closed.

CC @edi33416 @wilzbach

wilzbach · 2020-02-23T08:44:22Z

Yes, though I will let Edi close or update his PR as it contains a few more things. Most importantly the setup to emit the header files for DMD itself

RazvanN7 · 2021-01-12T12:31:09Z

I'm gonna close this since the superseded #10801 got merged and since then it was continuously improved. If any of the features in this PR are needed, the PR is already linked to the one that got in.

edi33416 requested review from Geod24, MartinNowak, WalterBright, ibuclaw, rainers and wilzbach as code owners June 4, 2019 16:53

dlang-bot added the Review:WIP Work In Progress - not ready for review or pulling label Jun 4, 2019

thewilsonator reviewed Jun 4, 2019

View reviewed changes

src/dmd/dtoh.d Outdated Show resolved Hide resolved

src/dmd/dtoh.d Outdated Show resolved Hide resolved

thewilsonator mentioned this pull request Jun 10, 2019

[WIP] Generate C++ header files from public extern (C++) declarations #8591

Closed

edi33416 force-pushed the pr/8591 branch from 1d5f426 to 032d60a Compare June 13, 2019 12:48

thewilsonator reviewed Jul 15, 2019

View reviewed changes

src/dmd/compiler.d Show resolved Hide resolved

wilzbach reviewed Jul 16, 2019

View reviewed changes

src/dmd/dtoh.d Outdated Show resolved Hide resolved

wilzbach reviewed Jul 16, 2019

View reviewed changes

edi33416 force-pushed the pr/8591 branch from bc18193 to 40478e6 Compare August 7, 2019 13:07

This was referenced Aug 7, 2019

Rename Dsymbol.namespace to cppnamespace #10287

Merged

Make RootObject equals and toChars const #10293

Merged

This was referenced Aug 19, 2019

Make Type hierarchy methods const #10319

Closed

Declare frontend structs as extern cpp #10321

Merged

edi33416 force-pushed the pr/8591 branch from 40478e6 to 25d1fa9 Compare August 21, 2019 12:50

edi33416 mentioned this pull request Aug 21, 2019

Remove const qual from covariant methods in Type hierarchy #10330

Closed

wilzbach changed the title ~~[WIP] Revive Generate C++ header files from public extern (C++) declarations~~ Revive Generate C++ header files from public extern (C++) declarations Oct 19, 2019

wilzbach removed the Review:WIP Work In Progress - not ready for review or pulling label Oct 19, 2019

edi33416 added 17 commits November 14, 2019 15:14

Update tool auto-gen message

9a3ce62

Update tests

7072bdc

Fix outdated code in cxxfrontend test

d17d199

Code cleanup

54e7e18

CPP does not support covariant const methods

0ef5acf

Correctly write ctor declarations

77b584a

Generate default args for literals

a548da6

Generate dtor and copy ctor declarations

3e75514

Rebase with master

6e052f8

Add compiler switch to cli.d

70b2f7f

Update tests

ca370fb

Add support for @disable

1eba99b

Improve default ctor generation

65b05f4

Cleanup cxxfrontend.c

c3f2a24

Update struct decl test

00d65f8

Use member initializer list for default ctor definition

144c85d

Add frontend.h dep to build.d

341de35

edi33416 force-pushed the pr/8591 branch from a67353f to 341de35 Compare November 14, 2019 15:08

MoonlightSentinel mentioned this pull request Feb 18, 2020

Add C++ header generation #10801

Merged

ibuclaw added the Feature:dtoh C++ header generation label Sep 28, 2020

RazvanN7 closed this Jan 12, 2021


		diff -pu --strip-trailing-cr ${EXTRA_FILES}/${TEST_NAME}.out ${OUTPUT_BASE}.out

		rm -f ${OUTPUT_BASE}.out

Uh oh!

Conversation

edi33416 commented Jun 4, 2019

Uh oh!

dlang-bot commented Jun 4, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bugzilla references

Testing this PR locally

Uh oh!

andralex commented Jun 4, 2019

Uh oh!

andralex commented Jun 4, 2019

Uh oh!

jacob-carlborg commented Jun 4, 2019

Uh oh!

wilzbach commented Jun 4, 2019

Uh oh!

Uh oh!

Uh oh!

ibuclaw commented Jun 10, 2019

Uh oh!

Uh oh!

wilzbach Jul 16, 2019

Choose a reason for hiding this comment

Uh oh!

thewilsonator Jul 18, 2019

Choose a reason for hiding this comment

Uh oh!

wilzbach Jul 18, 2019

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wilzbach Jul 16, 2019

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wilzbach commented Jul 16, 2019

Uh oh!

TurkeyMan commented Feb 16, 2020

Uh oh!

MoonlightSentinel commented Feb 16, 2020

Uh oh!

moon-chilled commented Feb 23, 2020

Uh oh!

wilzbach commented Feb 23, 2020

Uh oh!

RazvanN7 commented Jan 12, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

dlang-bot commented Jun 4, 2019 •

edited

Loading