[WIP] Rework -unittest flag

[alexandrumc]

The idea of this PR is to decrease the compilation time when the -unittest switch is used.

This has to do with the way the current algorithm that generates code for template instantiations works. The algorithm has two branches:

• If -unittest is used, then do code generation for every template that is instantiated and used in a root module, even if the same template is instantiated in some non-root module that is imported.

• Else (unittests not enabled) prefer instantiations from non-root modules, to minimize object code size. If a template is ever instantiated by non-root modules, do not generate code for it, because it will be generated when the non-root module is compiled.

So there are 3 options in my opinion:

1. the first one is to eliminate the branching and to adopt in every case the same behaviour performed when the -unittest switch is used: this might decrease the compile-time and will generate huge binaries because for almost every template instantiation code will be generated, but there will be a unified behaviour in the compiler

2. make the non-unittest behaviour the default, keep binaries small, decrease compilation time when -unittest is enabled but break a lot of code and force people to change the way they compile and write their code

3. make a better code generation algorithm for templates (that will use some kind of analysis to determine on which template instantiations code generation should be called and on which shouldn’t be called)
   1. this probably will lead to an increase in the binary size (not a huge one, but most likely we won’t be able to do something fine-grained because of the amount of work/time needed)
   2. and probably will break some code but not that much and will also decrease the -unittest compilation time

So far I've eliminated the branching from the algorithm and I've made the "if -unittest is used" behaviour the default one. The next step is to do some measurements regarding the compilation time and based on them to decide what should be done further. I'll update this description with the measurements when I'll have them.

[dlang-bot]

Thanks for your pull request and interest in making D better, @alexandrumc! We are looking forward to reviewing it, and you should be hearing from a maintainer soon.
Please verify that your PR follows this checklist:

• My PR is fully covered with tests (you can see the coverage diff by visiting the details link of the codecov check)
• My PR is as minimal as possible (smaller, focused PRs are easier to review than big ones)
• I have provided a detailed rationale explaining my changes
• New or modified functions have Ddoc comments (with Params: and Returns:)

Please see CONTRIBUTING.md for more information.

---

If you have addressed all reviews or aren't sure how to proceed, don't hesitate to ping us with a simple comment.

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub fetch digger
dub run digger -- build "master + dmd#10759"

[MoonlightSentinel]

It should be possible to revert to the old behavior. Maybe extend the -unittest flag to take an optional value, e.g. -unittest=all and -unittest=rootonly

[RazvanN7]

@alexandrumc Please add an explanation for all of these changes so that we don't have to speculate what the intent is. I assume that you haven't done so because this is a work in progress project, but I suggest that you do it anyway so that reviewers/community members are able to understand what you are trying to do and offer support.

[alexandrumc]

@alexandrumc Please add an explanation for all of these changes so that we don't have to speculate what the intent is. I assume that you haven't done so because this is a work in progress project, but I suggest that you do it anyway so that reviewers/community members are able to understand what you are trying to do and offer support.

Done.

[Geod24]

You might also want to test what effect your change have on some sample code, e.g. the ones at https://perf.dlang.io/

[RazvanN7]

How about creating a file __tempinstances.d that will be viewed as a root module where all the template instances from non-root modules go? This file is generated by the compiler and it imports the non-root modules that were the source of a template instantiation. All the root modules that instantiate templates from non-root modules will import __tempinstances. This way, every time a template is instantiated from a non-root module, it is going to have code generated for it in an auxiliary file that is going to be linked by default.

Destroy.

[atilaneves]

How about creating a file __tempinstances.d that will be viewed as a root module where all the template instances from non-root modules go? This file is generated by the compiler and it imports the non-root modules that were the source of a template instantiation. All the root modules that instantiate templates from non-root modules will import __tempinstances. This way, every time a template is instantiated from a non-root module, it is going to have code generated for it in an auxiliary file that is going to be linked by default.

Destroy.

Importing __tempinstances might cause name clashes. I'm also unsure of what this strategy would accomplish.

[RazvanN7]

Importing __tempinstances might cause name clashes.

Why would you get name clashes?

I'm also unsure of what this strategy would accomplish.

You will generate code for each template instance exactly once no matter in what module the instantiation is used + you get uniformity for the compilation with -unittest and without it. Plus the temp file can be cached. For example, if you have 2 files a.d and b.d that both use the same template instance from the non-root module c.d and you want to compile separately, first a and then b, once you compile b the template instance from a is already there in __tempinstances.

[RazvanN7]

This seems to be obsolete now, given the recent template emission fixes. @alexandrumc please reopen if anything new is added.