build d_do_test with stable compiler

[MartinNowak]

• so that the test-suite can still run when some bug breaks d_do_test

Let's see whether that works on Windows/Cygwin.

[dlang-bot]

Thanks for your pull request, @MartinNowak!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub fetch digger
dub run digger -- build "master + dmd#7345"

[WalterBright]

That was quick! Nice. And it means a separate PR can be done to ratchet the STABLE compiler used forward, so just that can be tested in isolation.

[WalterBright]

BTW, don't pull this just yet. I have a couple ideas I want to try tomorrow first to deal with https://issues.dlang.org/show_bug.cgi?id=17997

[MartinNowak]

BTW, don't pull this just yet. I have a couple ideas I want to try tomorrow first to deal with https://issues.dlang.org/show_bug.cgi?id=17997

I'd guess that the bug also manifests in some test, not only in d_do_test itself. But sure we can wait, not sure whether mktemp and unzip will properly work on Cygwin anyhow.

[braddr]

Why not just use the same dmd used to build dmd? It's a specific stable version and is guaranteed to already exist. And I am really against the make file downloading and installing a compiler in the test makefile (or any other part of the build process for that matter). I know I'm in the minority on this, but it's just a fundamental violation of a basic build tenant.

[WalterBright]

As long as it is not the dmd being tested, I'm good with that. Looking through the code of d_do_test.d, a very large percentage of dmd/druntime/phobos has to be bug free for it to work at all.

I don't know what order things are done, but it should be:

1. build dmd
2. build libraries
3. run tests
4. run library unit tests

(3) is before (4) because the tests are designed to make bugs easily tracked down. It's a LOT more work to track down a compiler bug starting from library unit tests. Ideally, the library unit tests should never fail due to compiler bugs if (3) passes.

Also, are the library unit tests counted in the coverage report? They shouldn't be.

[WalterBright]

Oh, and as a step (5), build the compiler again with itself, and once again, then do a binary compare. It should match exactly.

[MartinNowak]

Oh, and as a step (5), build the compiler again with itself, and once again, then do a binary compare. It should match exactly.

We already do that in the Travis-CI test.

[MartinNowak]

Why not just use the same dmd used to build dmd? It's a specific stable version and is guaranteed to already exist. And I am really against the make file downloading and installing a compiler in the test makefile (or any other part of the build process for that matter). I know I'm in the minority on this, but it's just a fundamental violation of a basic build tenant.

Using a system wide dmd is a recipe for confusion and issues, relying on you to update all testers has been a serious bottleneck over the past years. Being able to update such dependencies with code is a proven success (hence docker).
Indeed we should try to bundle this with the auto-bootstrap compiler.

Any suggestion for a cacheable directory to avoid repeated downloads? We could install this to a user-wide location, e.g. install.sh's default ~/dlang folder. In fact we might just use install.sh.

[braddr]

I didn't suggest system wide. I suggested using the exact same mechanism, that already exists on all the testers, used during the build: HOST_DC points to a specific, intended to be used compiler. It's perfect for this use case.

I tend to be put in the position of pointing out why requests for changes are either out right broken or are only partially ready. That doesn't make me a bottleneck but rather a proper gatekeeper. Can we keep on topic here please?

[MartinNowak]

I didn't suggest system wide. I suggested using the exact same mechanism, that already exists on all the testers, used during the build: HOST_DC points to a specific, intended to be used compiler. It's perfect for this use case.

Yes, we can try that (including the auto_bootstrap mechanism), still has the problem that people might use different versions.
The d_do_test tool is a lot more stable than e.g. dpl-docs with more all of it's dependencies, so compiler updates should be rare.

I tend to be put in the position of pointing out why requests for changes are either out right broken or are only partially ready. That doesn't make me a bottleneck but rather a proper gatekeeper. Can we keep on topic here please?

Well, at least perceived, some contributors don't know how to do updates on the auto-tester (or whom to contact), thus tend to avoid them altogether or work around them.
Being able to automate such dependencies not only reduces manual maintenance (for you to update 17 hosts), but also better documents the setup.
For that reason I prefer to download the compiler, so it's quite on topic :).
Of course downloading with every test should be avoided.
In any case I will try to reuse the src/posix.mak HOST_DMD mechanism for now.

[WalterBright]

BTW, don't pull this just yet.

I withdraw that. Ready to pull.

[wilzbach]

Also, are the library unit tests counted in the coverage report? They shouldn't be.

No, they aren't. The only command that is executed during the DMD code coverage build is this one:

make -j1 -C test MODEL=$MODEL ARGS="-O -inline -release" DMD_TEST_COVERAGE=1

[ibuclaw]

Is this wise? I mean d_do_test itself doubles as a compilable test. On the flip side though it does prevent people making sweeping changes to it just for using their new .

[wilzbach]

As we're trying to move to test/run.d I'm not sure that this will help us in the future.
Also as auto-tester recently got upgraded to 2.079.0, we might not need this PR anymore.

[MoonlightSentinel]

Implemented in #10950. The makefile now uses HOST_DMD to build run,d, d_do_test.d_, ... if possible.