Nix integration errors

[davidar]

I'm seeing some strange errors when using Stack with Nix integration enabled.

Globally:

$ stack --version
Version 1.0.1 x86_64
$ stack setup # doesn't make a difference if this line is executed or not
$ stack --nix install happy # won't work
Run from outside a project, using implicit global project config
Using resolver: lts-4.0 from implicit global project's config file: /home/david/.stack/global-project/stack.yaml
No compiler found, expected minor version match with ghc-7.10.3 (x86_64) (based on resolver setting in /home/david/.stack/global-project/stack.yaml).
Try running "stack setup" to install the correct GHC into /home/david/.stack/programs/x86_64-linux-nix/
$ stack --nix setup # doing what stack told me to do...
Run from outside a project, using implicit global project config
Using resolver: lts-4.0 from implicit global project's config file: /home/david/.stack/global-project/stack.yaml
Preparing to install GHC to an isolated location.
This will not interfere with any system-level installation.
Already downloaded.                 
Unpacking GHC into /run/user/1000/stack-setup9737/ ...
<snip>
/nix/store/60zf1xjmhwbf4g42724wrrj6ydy45cch-gnutar-1.28/bin/tar: ghc-7.10.3/utils/count_lines/ghc.mk: Cannot write: No space left on device
/nix/store/60zf1xjmhwbf4g42724wrrj6ydy45cch-gnutar-1.28/bin/tar: ghc-7.10.3/utils/compare_sizes/ghc.mk: Cannot write: No space left on device
/nix/store/60zf1xjmhwbf4g42724wrrj6ydy45cch-gnutar-1.28/bin/tar: ghc-7.10.3/utils/compare_sizes/dist-install/package-data.mk: Cannot write: No space left on device
/nix/store/60zf1xjmhwbf4g42724wrrj6ydy45cch-gnutar-1.28/bin/tar: ghc-7.10.3/utils/compare_sizes/dist-install/build/.depend.haskell: Cannot write: No space left on device
/nix/store/60zf1xjmhwbf4g42724wrrj6ydy45cch-gnutar-1.28/bin/tar: ghc-7.10.3/utils/compare_sizes/dist-install/build/.depend.c_asm: Cannot write: No space left on device
/nix/store/60zf1xjmhwbf4g42724wrrj6ydy45cch-gnutar-1.28/bin/tar: ghc-7.10.3/mk/project.mk: Cannot write: No space left on device
/nix/store/60zf1xjmhwbf4g42724wrrj6ydy45cch-gnutar-1.28/bin/tar: ghc-7.10.3/mk/install.mk.in: Cannot write: No space left on device
/nix/store/60zf1xjmhwbf4g42724wrrj6ydy45cch-gnutar-1.28/bin/tar: ghc-7.10.3/bindist.mk: Cannot write: No space left on device
/nix/store/60zf1xjmhwbf4g42724wrrj6ydy45cch-gnutar-1.28/bin/tar: ghc-7.10.3/libraries/gen_contents_index: Cannot write: No space left on device
/nix/store/60zf1xjmhwbf4g42724wrrj6ydy45cch-gnutar-1.28/bin/tar: ghc-7.10.3/libraries/prologue.txt: Cannot write: No space left on device
/nix/store/60zf1xjmhwbf4g42724wrrj6ydy45cch-gnutar-1.28/bin/tar: Exiting with failure status due to previous errors

Note that there is plenty of space left on all my partitions, and I can reproduce this on another computer.

Within a project:

$ stack new test-project --resolver lts-3.0
$ cd test-project
$ stack --nix setup
stack will use the GHC on your PATH
For more information on paths, see 'stack path' and 'stack exec env'
To use this GHC and packages outside of a project, consider using:
stack ghc, stack ghci, stack runghc, or stack exec
$ stack --nix install happy
happy-1.19.5: configure
/run/user/1000/stack10039/happy-1.19.5/.stack-work/dist/x86_64-linux-nix/Cabal-1.22.4.0/setup/setup: singleBuild: runInteractiveProcess: exec: permission denied (Permission denied)

CC: @bitemyapp

Edit: Note that I'm using a standalone Nix install on Ubuntu

[borsboom]

@YPares, any ideas?

[mboes]

I think what's happening here is that with Nix support turned on, GHC is being unpacked in the local directory when you do stack setup (@manny might this behaviour be inherited from the Docker support?). So I suspect you're running out of RAM, not disk space, as the /run/user/1000/... paths tend to suggest (/run is a tmpfs on most systems).

Further, I cannot reproduce the "within a project" behaviour. It works as expected on my system (Happy installs fine). Have you tried doing everything on-disk rather than in-memory?

Finally, using the --nix flag outside of a stack project is not a supported use case. I'm not sure that it ever makes sense to do so. Perhaps we should yield an error in that case. @YPares thoughts?

[davidar]

@mboes Yeah, it's the within-project behaviour that I'm most concerned about, which I can reproduce on multiple computers.

Have you tried doing everything on-disk rather than in-memory?

Possibly stupid question, but how do I tell it to do that?

[mboes]

This issue looks very similar to this one: #841. Perhaps the same cause.

@davidar to make sure, are you at least creating your project somewhere in your home directory, rather than in some folder on tmpfs? GHC itself might still be writing some temporary files to tmpfs, but it should keep it at that. How much RAM do you have? And have you ever seen the above error with Stack even without --nix?

[borsboom]

You should be able to override the location of any temporary files using the TMPDIR environment variable.

[YPares]

@davidar Outside a project, normally stack --nix setup shouldn't try to download a GHC, this is indeed abnormal behaviour. But for the rest, I wasn't able to reproduce the bug(s). E.g running

$ stack new test-project --resolver lts-3.0 && cd test-project
$ stack --nix setup
$ stack --nix install happy

works fine on my laptop.

[YPares]

@davidar Can you try your out-of-project case against stack's HEAD? Ie after building it, outside of the project, run:

$ path/to/stack/local/copy/.stack-work/install/x86_64-linux/lts-4.0/7.10.3/bin/stack --nix --resolver lts-3.20 setup

(setting the resolver is necessary, as Nixpkgs still don't know resolver more recent than lts-3.20.
On my computer that outputs:

[...]
stack will use the GHC on your PATH

which is the expected behaviour (stack using Nix-provided GHC). I think it's a side effect of the recently merged PR #1646.

And likewise, installing happy with

$ path/to/stack/local/copy/.stack-work/install/x86_64-linux/lts-4.0/7.10.3/bin/stack --nix --resolver lts-3.20 install happy

works as expected too.

[davidar]

You should be able to override the location of any temporary files using the TMPDIR environment variable.

Ah, that seems to resolve the (within-project) problem. Is it Stack that's trying to build things within /run/user, or Nix?

Will test the out-of-project behaviour in a minute.

[YPares]

@davidar @borsboom After checking, indeed nix-shell sets the $TMPDIR to /run/user/1001, unless it's already set in the parent environment.
So this explains the difference in the behaviour of Stack.

[YPares]

Issue NixOS/nix#395 in Nix might expose a reason as to why Nix sets $TMPDIR (to void it as soon as possible, so that builds are isolated, which it cannot do if TMPDIR is the /tmp every program writes in)
@borsboom It looks like a conflict of use of env var between Stack & Nix, as both may have slightly different views regarding what the use of a tmp directory should be. Is it acceptable to ask the Stack users (in that case) to manually set $TMPDIR to something else?