@noobject on module decalaration to prevent importing object.d

[marler8997]

An alternative to #10184

This PR allows modules to specify the @noobject attribute on their module declartion to disable importing the implicit object module, i.e.

@noobject
module foo;

// the object.d module will not be imported
// because @noobject was specified on the module declaration

PR #10184 instead added a new command-line switch to disable importing the object module. I created this one as an alternative because it's the source code that knows whether or not the object module should be imported. This solution is less brittle as the one compiling the module doesn't need to know whether or not the command-line switch is needed. It also allows the compiler to compile both kinds of modules in the same invocation since each module configures itself instead of having one global configuration that must apply to every module being compiled in the same invocation.

The use case for this pointed out by @JinShil is to be able to compile modules that don't require the object.d module without having to create a dummy object module. Note that I don't have a good idea of how useful this feature will be, however, I would prefer this implementation over a command-line switch if it is accepted.

[dlang-bot]

Thanks for your pull request and interest in making D better, @marler8997! We are looking forward to reviewing it, and you should be hearing from a maintainer soon.
Please verify that your PR follows this checklist:

• My PR is fully covered with tests (you can see the coverage diff by visiting the details link of the codecov check)
• My PR is as minimal as possible (smaller, focused PRs are easier to review than big ones)
• I have provided a detailed rationale explaining my changes
• New or modified functions have Ddoc comments (with Params: and Returns:)

Please see CONTRIBUTING.md for more information.

---

If you have addressed all reviews or aren't sure how to proceed, don't hesitate to ping us with a simple comment.

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub fetch digger
dub run digger -- build "master + dmd#10188"

[ghost]

What happens if I have multiple files?
Need to add @noobject in each one?

[marler8997]

Yeah. You add it to the files that you don't want to use object.d.

Just like you have to add "import" statements for each module you need, this is the inverse where you must add an attribute to exclude the object module from being imported. This maintains that the module itself is the one who "owns" which modules to import and/or not import since the module implementation itself depends on a particular set of imports.

[JinShil]

Despite being a beneficiary of such a feature, I don't think this is the best approach because it is much too specialized. I would prefer we aim for more general-purpose, composable features.

If I could have my druthers I would require users to explicitly import object; (or rename object.d to package.d and import druntime;). However, that would be too radical, and my other attempts to help make the compiler more pay-as-you-go w.r.t. object.d have not been well-received (#7825, #9814).

My suggestion, given the history of previous attempts, is to add the ability to implicity import from the compiler command-line. We already have the -i=-foo switch that allows us to exclude implicity importing certain modules. Maybe we can exapand that with a + operator so users can implicitly import modules from the command line with -i=+foo. Then we can add -i=+object to the dmd.conf file and remove the hard-coded logic in the compiler to implicitly import object.d. If that abuses the intent of the -i switch, let's make a new one (e.g. -implicitlyImport=object).

Such a feature will resolve the issue at hand (having to create phony source files just to get a build), will remove some hard-coded logic from the compiler (which we should be aiming for anyway), and will be more generally useful to everyone. Also, it makes the compiler more "additive" (for lack of a better word); instead turning things OFF, we only need to turn things ON, which is more in the spirit of "pay-as-you-go". If one wants something ON by default, they can add it to the dmd.conf file.

A couple of other use cases:

Implicity importing runtime compiler assets
We're currently converting many runtime hooks to templates that need to be publicly imported into object.d because it is the only tool we have at our disposal to accomplish this task:
https://github.com/dlang/druntime/blob/7d962321244ae1cb03a6c9f25ff85ece83f28277/src/object.d#L41-L63

This is somewhat unfortunate because object.d is a publicly documented module, and none of the _d_xxx should be explicitly employed by the user, but there they are, font-and-center, in the only place they can be at the moment. If we had a way to tell the compiler to implicitly import certain modules, we can hide these symbols in a dark corner of the runtime and implicitly import them in dmd.conf.

Specialized builds
Users may also find the feature useful when doing specialized builds. For example, importing an instrumentation framework into all files only for a debug build, or importing specialized platform assets when building the same software for different platforms. Yes, there are other ways to do this in D, but this option may be preferrable over existing methods that could easily turn into version-hell.

[marler8997]

The -i option does not change what modules are imported, it only changes what modules are "included" in compilation. It sounds like you understand that but I'm just clarifying. Specifying -i=-foo does not mean the foo module will not be imported, it means it won't be compiled to object code.

The module itself is the sole owner and maintainer of its imports. Whether or not a module imports the object module is fundamental to the module itself, changing it would require a refactor of the module in question. It's not something you can enable/disable from the command-line and have both versions work. This makes the flag meaningless since the module will only compile one or the other, but not both, meaning it's not really a command-line option is it? It's really a part of the module definition rather than a command-line option.

All this being said, if you want to implicitly add modules to every module you can do this already. Just create your object object module that publicly imports any modules you want by default. Although, I'm not sure if you could publicly import the original object.d from druntime if you did this....I'd have to play with it to find out a way to do that.

[JinShil]

@marler8997 I don't disagree with anything you've said. But the fundamental problem is that the compiler is implicitly importing object.d when it shouldn't be implicitly importing anything. As you said the module is the sole owner and maintainer of its imports, so only the module should specify that it wants to import object.d.

Since we agree that the module is the sole owner of its imports and the compiler is partially assuming ownership of the module's imports, it would be prudent to return the compiler to a more fundamentally correct state by turning off the implicit import of object.d. Unfortunately, that is not a practical option, so we are left with the next best thing which is a configuration option for the compiler to return it to a more fundamentally correct state.

Correction:
Actually what I'm proposing is that we return the compiler to a more fundamentally correct state, where it does not assume partial ownership of the module's imports, but allows the user to revert to the existing, fundamentally incorrect compiler behavior of implicitly importing object.d, through the use of a compiler switch, to avoid breaking existing code.

Edit:
Note that I tried to be more precise about disabling this incorrect compiler behavior in #9814, but it was not well-received, so I'm reaching for the next best thing.

[marler8997]

I can't say for sure but it may have been better for object.d to have been explicitly imported. Of course like you say it's probably too late to change that. But I don't see it as a big problem either way.

I've read #9814 and I lean towards agreeing with @andralex comment in saying that we should focus us making object.d pay-for-what-you-use rather than trying to provide extra features to mitigate our inability to do so within the language with the current implementation. The D Language should be all we need to implement a pay-for-what-you-use runtime.

Let me supplement this by clarifying a bit about how I see the object module. The fundamental reason that object.d is needed right now is because the compiler defers some of its code generation to it. So, rather than generating the AST inside the compiler for some of its code, like other compilers would do, it just generates references to symbols within object.d and puts the implementation in there. However, this primary reason for why object.d is necessary gets conflated since object.d also contains alot of code that isn't referenced by code intrinsically generated by the compiler.

So in my mind, any code that could be referenced by compiler-generated code should always be imported/available in every module, and furthermore, isn't specific to the module but is actually specific to the compiler (so I'm changing what I said about the module owning all imports). This means you can still compile your module with a completely different object.d implementation so long as it maintains the same symbols required by the compiler and maintains the same public-facing API that is available to the module.

As I write this out, it makes me think that splitting up object.d could be an improvement to this situation. If the compiler intrinsic functions were deployed along-side the compiler binary in a module, say "intrinsics.d". This way we could keep the intrinsic functions but disable everything else in object.d.

Anyway I've gone off on a bit of a tangent. I still think that if you want to disable the object module, then it should be done inside the module. If a module needs to be compiled a certain way (i.e. a compiler that does not assume object is imported) then the module should declare that requirement rather than forcing the user to specify it as a compiler option and then failing with some seemingly unrelated compiler errors when then don't.

[JinShil]

I've given this some thought, but I can't support it.

We don't need another specialized, niche feature added to the compiler to workaround fundamental problems. Instead, we should be making fundamental improvements or adding transitional features that help us migrate towards those fundamental improvements.

This feature doesn't offer any practical benefit to the user over simply creating an empty object.d file, yet it arguable makes matters worse by adding more hard-coded logic to the compiler when we should be working to remove such logic from the compiler.

[Geod24]

I second @JinShil here. I can't support this.

[marler8997]

@Geod24 are you in favor of the command-line option instead? If so why?

I personally don't see much justification for this change. The only reason for creating it is to create what I think to be a better alternative should #10184 be accepted.

[marler8997]

@JinShil

Based on your response it sounds like you think that #10184 will help transition the language to more fundamental improvements? Can you elaborate on that?

Also how do you handle intrinsics? We are currently using object.d for code that the compiler would otherwise generate. Are you saying you want to move code from object.d into the compiler by generating AST during compile time? That would be alot harder to maintain right?

[Geod24]

@Geod24 are you in favor of the command-line option instead? If so why?

I'm still thinking about it, but I'm completely sure I don't want us to introduce a language feature for this.

[marler8997]

This isn't a language feature, it's using a user-defined attribute. It's already part of the language. There's no lexical or grammar changes. See the description for why we would want this to be configured in the module rather than a command-line option. Of course, that's assuming we want the feature in the first place.

[JinShil]

Based on your response it sounds like you think that #10184 will help move transition the language to more fundamental improvements?

See my comments at #10180 (comment) for some more detail about my perspective.

In general, I don't see -dnostdlib as a transitional feature. A transitional feature would be one that we could eventually do away with or one that is provided primarily for backward compatibility.

Also how do you handle intrinsics? We are currently using object.d for code that the compiler would otherwise generate? Are you saying you want to move code from object.d into the compiler by generating AST during compile time? That would be alot harder to maintain right?

All great questions. D is a little different from C/C++ in that it has part of its implementation in the runtime itself, that many would want implicitly imported for convenience. I don't think it's too much to ask to have users explicitly import the language features they intend to use (they can all be aggregated into one package.d if you want all that D has to offer), but that's already proven to be an unpopular opinion.

As I mentioned in the other PRs and on Slack, I would prefer to have this logic delegated to the toolchain configuration (i.e. dmd.conf). I believe it would only require one transitional compiler switch (e.g. -implicitlyImport=object), but I haven't explored it completely yet. A -implicitlyImport compiler switch would also allow us to separate the runtime hooks, intrinsics, and other "compiler-only" mechanisms into separate files form object.d and implicitly import them through dmd.conf. This will be good for some of the refactoring work that is currently going on in druntime. I also think it would empower users with other use cases including the use cases that prompted this PR (bare metal, embedded systems, operating systems, drivers, etc...) where one may not want object.d or may choose to organize a custom druntime a little differently. For the latter use case, they would invoke the compiler with dmd -conf= or provide their own custom dmd.conf file specific to their use case.

[marler8997]

Ok well it sounds like we want the same things, just need to work out the best way to get there. Let's keep thinking on this and hopefully we'll hear back from @WalterBright on #9936 and #9944 soon. If he rejects them then we'll have to rethink the strategy of moving things into dmd.conf.

[marler8997]

@JinShil you keep bringing up the point that the language shouldn't implicitly import anything. This is something I've kept note of as well. I've been playing around and exploring the zig language which uses this strategy. It only imports what you explicitly import and only links to what you tell it to link to. This means the helloworld example is a bit longer since you always need an import, but it makes bare-metal/alternate runtimes more straightforward. Might be worth looking into if you're interested. I've ported a few of my D projects to it so I could really learn it and I think it's got alot of potential. It is still pretty new though and there's a few big changes coming down the pike soon but still very usable right now.

Some other things I like about it.

• no dependency on libc (but you can still use libc if you want)
• all intrinsics are easy to distinguish as they are all prefixed by an @ symbol
• creator is engaged in discussion

All that being said it has some different core pholosophies from D. It prioritizes one way of doing things. It doesn't have overloads (which is a big change). It's a much simpler language and I'm surprised with how much I can do with it. So it's pretty interesting.

[JinShil]

@marler8997 I'm aware of the Zig language and have been checking in on it periodically. If it had a statically-checked memory safety mechanism and a better way to model object composition, I'd be more interested. I think the @reify feature is supposed to help with the latter, but I don't know if there are any plans for statically-checked memory safety.

[marler8997]

That's funny because I just added a proposal for @reify earlier today, not knowing it was already proposed. I seem to have convinced Andrew to implement at least a subset of it for simple types, so hopefully that gets implemented soon.

What kind of statically-checked memory safety are you looking for? Does D have some checking that zig doesn't?

[JinShil]

What kind of statically-checked memory safety are you looking for? Does D have some checking that zig doesn't?

Yes, @safe, scope and return lifetime attributes, and hopefully someday this. I'm not aware of any such objective in Zig.

[marler8997]

Whoa this is a big proposal/change to the D language! Very cool stuff. Can't wait to see what happens with it.

[WalterBright]

The need for such a feature is so specialized that the programmer doing it shouldn't have any difficulty creating a dummy object.d with nothing in it to import.

I don't think this feature pulls its weight.

[marler8997]

@WalterBright fair enough. Do you feel the same way about #10184?

[WalterBright]

@marler8997 yes.

[marler8997]

Closing since proper justification for the feature has not been presented.