Running run-make tests performs unnecessary rebuilds

[Kobzol]

With the latest master, when I run a run-make test twice in a row, bootstrap rebuilds lld-wrapper and rustdoc:

./x test tests/run-make/CURRENT_RUSTC_VERSION/
./x test tests/run-make/CURRENT_RUSTC_VERSION/
<lld-wrapper + rustdoc rebuilt>

This has to be a recent regression, it didn't happen until ~yesterday. Confirmed with @jieyouxu that it also happens for them.

[Kobzol]

Sounds suspiciously similar to #122491.

[Kobzol]

Yeah this is again caused by llvm-bitcode-linker being enabled 😆 @kjetilkjeka Do you think that there's some more general solution to the rebuild problem?

[kjetilkjeka]

Thanks for notifying me about this. I am travelling this weekend, meaning i might not have a chance to thoroughly investigate before Monday. (But I will at the latest be able to look on Monday).

The last time it was due to dependency on the same crates with different rustflags. One from llbc-linker and one from some other tool (rustdoc?). The two different invocations maps to the same cache entry but is invalid between each other and thus triggers a rebuild.

The solution last time was to mimic the rustflags being used other places in llbc-linker. That might be a good short term solution now as well.

I don't know about the long term solution but it seems like either caching multiple versions of a crate or unifying the rustflag somewhere might be alternatives?

[Kobzol]

Thanks for notifying me about this. I am travelling this weekend, meaning i might not have a chance to thoroughly investigate before Monday.

Of course, there's no rush, I just wanted to let you know, given that you already have some experience in debugging this 😆

This seems rather brittle though, as you mention. I wonder what's so unique about this tool that it causes these rebuilds so often.

@onur-ozkan You mentioned here that we cannot share flags between all Step invocations. But maybe it would make sense to better share flags between at least this tool and other steps, rather than hardcoding selected flags? (as was done last time with the parallel compiler).

[Kobzol]

Actually, I wonder if we really need to enable this linker by default in all four bootstrap profiles.

[onur-ozkan]

@onur-ozkan You mentioned #122491 (comment) that we cannot share flags between all Step invocations. But maybe it would make sense to better share flags between at least this tool and other steps, rather than hardcoding selected flags? (as was done last time with the parallel compiler).

Sure, seems fine to me. I didn't expect it to be a problem this frequently.

[Kobzol]

#126479 posted instead to disable this flag by default, I don't think it should be enabled for most contributors.