Repeated rebuilds of Hadrian (GHC build system)

[ndmitchell]

Using

$ stack --version
Version 1.10.0, Git revision d1f0fde156245ddbb6656461f3f3531ea84488a2 (dirty) x86_64 hpack-0.31.1

Given GHC as checked out with git clone https://gitlab.haskell.org/ghc/ghc.git --recursive at SHA
ef25b59a97f419a2ad6457f696e32aef9ffb3a0f, I do:

cd hadrian
stack build hadrian
.. takes a while and works ..
stack build hadrian 2> log.txt
.. rebuilds a large chunk of things and gives lots of error messages ..

I would expect that building twice in a row does little to nothing the second time around. Instead, it biulds a whole heap of stuff. The log is attached: log.txt

I note the line Exception ignored when attempting to load C:\Users\ndmit_000\AppData\Local\Temp\stack7772\hashable-1.2.7.0\.stack-work\dist\7d103d30\stack-config-cache: C:\Users\ndmit_000\AppData\Local\Temp\stack7772\hashable-1.2.7.0\.stack-work\dist\7d103d30\stack-config-cache: openBinaryFile: does not exist (No such file or directory)

Note sure if that is related.

[qrilka]

@ndmitchell we've just today changed the way Stack is supposed to work with GHC boot libraries (in #4547), and from a quick glance into your log it looks like it could be quite related - Stack complains a lot about mismatching libraries. I'll try to find some time to check out building Hadrian with the latest master - it looks to be a good test case (though a lengthy one )

[ndmitchell]

Note that running stack build hadrian just builds Hadrian, not GHC itself, so only a few minutes.

I upgraded to LTS 13.6 and removed the text local dependency, which cuts the rebuild down to just happy. New log attached for that case: log2.txt

[qrilka]

@ndmitchell it looks like we do some extra rebuilds even using the latest master - https://gist.github.com/qrilka/1ee438e352aa8b1539b63bd60483a056 but I get no errors at least.
Checking out if we could have a workaround for boot libraries rebuilds needs further investigation

[ndmitchell]

Looking at it, if I remove happy as a build-tool of hadrian, the spurious rebuilds go away. I suspect the issue is that happy has a custom-setup dependency on Cabal of < 2.4, and we're building with a local Cabal 2.5.

[dbaynard]

Is this resolved, then? It would be good to get an integration test on this. We have 4001-excess-recompilation from #4003 but (as I understand it) it only checks stack doesn't unregister anything when

1. test/bench dependencies change, and
2. only library is targeted.

[ndmitchell]

Definitely not resolved, since I can reproduce it today using Stack from HEAD.

[qrilka]

@ndmitchell with fixes in #4588 I see no more repeated rebuilds of Hadrian or its dependencies

[ndmitchell]

Confirm it's gone now, so closing. Thanks for all the fixes!