Add flag to print GHC output to stdout

[nh2]

Having streaming compilation output is super useful to ensure that things compile at the right speed, especially when piping into something like ts.

#426 added a flag that can stream to a file, but that still doesn't provide streaming terminal output.

Of course interleaved output makes things messy, so it would make sense for this flag to work only for -j1.

Note that (quote @mgsloan)

If it's a local package and the last thing to build with -j1, it should output the log

but it would be great to have it also for snapshot packages.

[recursion-ninja]

Our project recently moved to a multi-package build setup, and we immediately encountered this defect. I see that this is currently a low-priority issue, but I'd argue that it's actually high-priority, as it's an essential function of a build system.

Every other build system or compiler I've used does it by default, e.g. gcc, clang, Java, the Python and Perl interpreters, and of course both GHC and cabal. But rather than just appealing to authority, here's a concrete example of why it's important. Recently one of our test suites regressed from running in about a minute to taking an hour. I had no idea why stack had hung. It could have been (off the top of my head) any permutation of:

• The compiler running into an undecidable instance.
• A test suite gone awry.
• A benchmark executing an infinite loop.

Without output streaming to the terminal this is next to impossible to diagnose.

And then, of course, sometimes you need to see a warning to determine the cause of an error that appears later in the build process, but that's not possible under either of these scenarios.

I've read through all of the threads regarding this subject, just a few of which are:

• Warnings aren't visible #298
• stack build --ghc-output (or similar) to print GHC output during a build #426
• Avoid hiding output due only to parallelism #642
• output ghc warnings for multi-package build #896

I don't find any of the suggested workaround satisfactory, but I'd like to take special exception to the suggestion that I build with --pedantic, which is all but worthless. Aside from that fact that erroring out is for errors and not for warnings, it means that the compiler stops even for warnings that it over zealously complains about, such as type family deductions and over-determined constraints.

Is it possible to stream to the terminal in multi-package builds the output from GHC compilation, test suites, benchmarks, etc just like what occurs in a single package build, regardless of whether the packages are being built in parallel or not. While I think this should be the default and users should need to opt-out of this information, I'd settle for an opt-in flag for streaming the output of my packages building to the terminal.

[mgsloan]

Is it possible to stream to the terminal in multi-package builds the output from GHC compilation, test suites, benchmarks, etc just like what occurs in a single package build, regardless of whether the packages are being built in parallel or not.

PRs appreciated! This is quite a lot trickier than it would seem.

[recursion-ninja]

@mgsloan The onus is not on me, the user of stack, to implement essential functionality of stack as a build system.

Can a collaborator reopen this issue and add it to a milestone, preferably P1: Must.

[mgsloan]

@recursion-ninja This issue is not closed, and I've added it to the P2 milestone because things work fine without this nicety. Note that logs do get streamed into .stack-work/logs, so you can look at those if you need to know what's going on.

It is unclear how this streaming output should behave. In the presence of concurrent builds, which one gets streamed to the output? Perhaps the output should get interspersed with some indication of which process it comes from? Not saying there aren't good answers to this, but in addition to implementation there is the problem of determining what should be implemented.

The onus is not on me, the user of stack, to implement essential functionality of stack as a build system.

Stack is an opensource project.

[nh2]

Perhaps the output should get interspersed with some indication of which process it comes from?

@mgsloan The way nixops does it works quite well and looks like this:

server1> Stopping service nginx
server2> Stopping service nginx
server1> Starting service apache
server3> Stopping service nginx
server2> Starting service apache
server3> Starting service apache

where the server3> bits have some nice different terminal colours so you can easily distinguish what belongs to what visually (and the colours make it strangely satisfying to look at).

This approach still gives a very good indication of progress, and is copy-paste friendly.

And, even more conveniently, you can sort the output by the prefix up to > to recover each process's individual output in one block.

[mgsloan]

Yeah, that sounds good to me @nh2 . Ok, so we've decided on the goal behavior, so that part is done :)

[snoyberg]

Relevant PR: #4097