rdmd: compile the root module in the same step as getting the dependencies

[aG0aep6G]

As suggested by Andrei in a comment on #191. Consequently, this is an alternative to #191.

Little benchmark

Used bash's builtin time with TIMEFORMAT=%R to get simple output.
All times are the best of 3 runs.
rdmd_master is the current master (1260719).
rdmd_single_out_root is the one from this PR.
foo.d is http://sprunge.us/ZZhR, taken from #191.

single file

$ time ./rdmd_master --force foo.d > /dev/null
1.142
$ time ./rdmd_single_out_root --force foo.d > /dev/null
0.747

Good improvement here.

trivial root, complex import

bar.d is import foo;.

$ time ./rdmd_master --force bar.d > /dev/null
0.935
$ time ./rdmd_single_out_root --force bar.d > /dev/null
0.937

As expected, no change here.

complex root, trivial import

baz.d is foo.d plus import qux;.
qux.d is an empty file.

$ time ./rdmd_master --force baz.d > /dev/null
1.144
$ time ./rdmd_single_out_root --force baz.d > /dev/null
0.763

This is improving too, because the root isn't compiled again when the import is compiled.

reuse cached executable

$ time ./rdmd_master foo.d > /dev/null
0.013
$ time ./rdmd_single_out_root foo.d > /dev/null
0.012

No bad surprise here.

[dlang-bot]

@aG0aep6G, thanks for your PR! By analyzing the annotation information on this pull request, we identified @CyberShadow, @andralex and @NilsBossung to be potential reviewers. @CyberShadow: The PR was automatically assigned to you, please reassign it if you were identified mistakenly.

(The DLang Bot is under development. If you experience any issues, please open an issue at its repo.)

[wilzbach]

As suggested by Andrei in a comment on #191. Consequently, this is an alternative to #191.

Oh nice - I did the same too, but didn't publish it yet because I wanted to test it further & do more cleanup ;/

[AndrejMitrovic]

Is there a difference between using .byValue.any and .length > 0 ?

[aG0aep6G]

myDeps is a string[string]. .byValue.any is false when there are values but they're all null. Null values are set for non-source dependencies (lines 656-683).

But maybe this should check against emptiness instead. It's easy to forget that null has a different truthiness than empty strings.

[wilzbach]

That's quite a clever trick to check for "external" source files!

But maybe this should check against emptiness instead. It's easy to forget that null has a different truthiness than empty strings.

Yep better use .any!"a !is null", then it's at least consistent with the query below.

[aG0aep6G]

Yep better use .any!"a !is null", then it's at least consistent with the query below.

I think that's the same as just .any. But it's clearer with an explicit !is null, so I changed it. I did o => o !is null though, because I don't like string predicates.

[AndrejMitrovic]

Cool stuff! It gets my LGTM.

[wilzbach]

I used exeBasename as the filename is shown in the error message

[aG0aep6G]

I used exeBasename as the filename is shown in the error message

Seems like a good idea. Done.

The problem I had worked around here was that dmd would also write to foo.o when -main is passed in the linking command.

I've now changed it to pass -main when compiling the root module. But there seems to be a bug in dmd with -main -c -of: issue 16440. So I've reverted to rdmd's old style of doing --main: stubmain.d.

[wilzbach]

It's too late today for me to really look at rdmd.d anymore. Yours here seems to be more extensive than mine. So you may have caught details that I missed.

In the short version: You directly modified the source dependency checking function to emit an object file, whereas I went a rather long road that removes this function completely and refactored quite a bit on the way.

PR duel!

After comparing our approaches I think yours is better because it touches a lot less lines - great job!
If I find time, I will port my refactorings to the future master.

I already added a couple of comments, but there is one interesting thing in the benchmarks I did at #195 - your binary is 0.1MB larger and it takes 1ms longer to run on all benchmarks.
I store the information whether there are other source files directly while dependency file is read, might this explain the 1ms difference?

edit: just to clarify - the approach #195 is the same and thus I can backup the benchmarks and improved performance.

[aG0aep6G]

I already added a couple of comments, but there is one interesting thing in the benchmarks I did at #195 - your binary is 0.1MB larger and it takes 1ms longer to run on all benchmarks.
I store the information whether there are other source files directly while dependency file is read, might this explain the 1ms difference?

Are you sure that you can detect a 1ms difference? On my machine, two runs of the same binary usually differ by more than 1ms.

I've ran your benchmark script a couple times, and the results weren't really conclusive. Sometimes #195 was faster, other times #194 was faster. The difference was always only a couple ms. Two runs of the same binary had a similar variance.

[andralex]

spaces around ~ please

[andralex]

Good work, I approve modulo a few nits. Thanks!!

[aG0aep6G]

Good work, I approve modulo a few nits. Thanks!!

Added spaces around ~ (but only at the ones I'm introducing or touching anyway). That was the only nit, right?

[andralex]

thx!

[andralex]

This is definitely worth a changelog entry and hopefully also a blog entry.

[wilzbach]

This is definitely worth a changelog entry and hopefully also a blog entry.

Ping @mdparker - we had quite a fun weekend here. Maybe you can interview @aG0aep6G?

[mdparker]

If @aG0aep6G sends me an email (I can't find any contact info), we can start talking about a blog post. I'm eager to do so.

[aG0aep6G]

If @aG0aep6G sends me an email (I can't find any contact info), we can start talking about a blog post. I'm eager to do so.

My email is ag0aep6g@gmail.com. Though, I'm not sure if I can give much interesting insight. After all, combining -c with -v was Vladimir's/Andrei's idea.

[CyberShadow]

This pull request introduced a regression:
https://issues.dlang.org/show_bug.cgi?id=16978