Stack is not using nix cached packages for cabal dependencies

[ericsagnes]

Stack version: Version 1.0.0 x86_64
NixOS version: 16.03.git.5f6af50M

Steps to reproduce

1/ Generate a new project using a resolver with nix binary cache available

$ stack new foo --resolver lts-4.1
$ cd foo

2/ enable nix in stack.yaml

nix:
  enable: true

3/ setup stack

$ stack setup
stack will use the GHC on your PATH
For more information on paths, see 'stack path' and 'stack exec env'
To use this GHC and packages outside of a project, consider using:
stack ghc, stack ghci, stack runghc, or stack exec

$ stack path
...
bin-path: /home/eric/.stack/snapshots/x86_64-linux-nix/lts-4.1/7.10.3/bin:/nix/store/20ydc8gy186yrjk06x8cbsqs4mly7xaq-ghc-7.10.3/bin:/nix/store/1z72dcicw4r87vmcnwbmgdw424nrz58c-patchelf-0.8/bin:/nix/store/n27xxmjjmcvwz5pss08qxz9isdh9mgbd-paxctl-0.9/bin:/nix/store/f3q55vxs9i3zxasnhlf5il0byrg456d2-gcc-wrapper-4.9.3/bin:/nix/store/1bwrkqji9jsq0ypip3cpgz5qcn2rad55-coreutils-8.24/bin:/nix/store/05z0y2c87p4z0gcn9wlyxskg5kcyjxmy-findutils-4.4.2/bin:/nix/store/l489hcmi1la2qj5px08g1bszijm9z1hm-diffutils-3.3/bin:/nix/store/s1qfi4hrxvs8bqs148haisgcxzvf2cjy-gnused-4.2.2/bin:/nix/store/vk8b9m1pnlv3vf9l4wnak9grxp5j3c55-gnugrep-2.22/bin:/nix/store/bsiqv3v9dcijgd275fglxxi93a1mfm9w-gawk-4.1.3/bin:/nix/store/60zf1xjmhwbf4g42724wrrj6ydy45cch-gnutar-1.28/bin:/nix/store/nag704c7wqqyv3n86i8zs3b66sgpq02q-gzip-1.6/bin:/nix/store/xx0lhx08rxin0w20j0y7ib4lkvl72vw4-bzip2-1.0.6/bin:/nix/store/zym9b3asfqcyhllrz2r8lzh6ir62ll1s-gnumake-4.1/bin:/nix/store/fx1kaw6faw8vmpwbhmlhl4agxgb110yw-bash-4.3-p42/bin:/nix/store/mqd6mwww0zi14zjnh6pcvb4wr03z8q9g-patch-2.7.5/bin:/nix/store/gnvv792j5jb7xzzjpp602ihs2y5ffjbv-xz-5.2.2/bin:/nix/store/rpdv6pi9z5dq7mcfh3rsavx0aixvcgl1-gcc-4.9.3/bin:/nix/store/3wjska842abcj7k36jnk5233i2x83c66-binutils-2.23.1/bin:/nix/store/jlmb88f5hgigbmc3c74ynxgn3frlzxkr-glibc-2.21/bin:/nix/store/1bwrkqji9jsq0ypip3cpgz5qcn2rad55-coreutils-8.24/bin
...

Stack is using ghc from the nix store, all good.

4/ Adding a dependency to foo.cabal, eg. lens

library
  hs-source-dirs:      src
  exposed-modules:     Lib
  build-depends:       base >= 4.7 && < 5
                      ,lens
  default-language:    Haskell2010

5/ building with stack build

$ stack build

Expected

Using the nix binary cache/nix store as haskell.packages.lts-4_1.lens is in the binary cache (S flag) and in the local nix store (P flag):

$ nix-env -f "<nixpkgs>" -qaPs -A haskell.packages.lts-4_1.lens
-PS  haskell.packages.lts-4_1.lens  lens-4.13
$ nix-shell -p haskell.packages.lts-4_1.lens
[nix-shell:~]$

Actual

Stack build lens:

base-orphans-0.4.5: copying precompiled package
mtl-2.2.1: copying precompiled package
parallel-3.2.1.0: copying precompiled package
prelude-extras-0.4.0.2: copying precompiled package
primitive-0.6.1.0: copying precompiled package
tagged-0.8.2: copying precompiled package
stm-2.4.4.1: copying precompiled package
transformers-compat-0.4.0.4: copying precompiled package
text-1.2.2.0: configure
StateVar-1.1.0.2: copying precompiled package
distributive-0.4.4: copying precompiled package
exceptions-0.8.0.2: copying precompiled package
text-1.2.2.0: build
vector-0.11.0.0: configure
reflection-2.1.1.1: download
vector-0.11.0.0: build
reflection-2.1.1.1: configure
reflection-2.1.1.1: build
reflection-2.1.1.1: copy/register
...

---

Am I missing some settings or using nix packages cache for dependencies is not yet implemented?

[YPares]

Using Nix packages as a cache for Haskell dependencies is indeed not implemented.
I realize that the intent of Nix integration wasn't maybe clear enough, so I made an addendum to the docs in commit 7a212a5

Note that there is a Stack option that might allow one to do that (see PR #990), by reusing the GHC package DB in the nix store.
However, I've only briefly experimented with it but I got errors due to build numbers not matching.
I suppose Stack expects that the packages present in the package db you give to --extra-packages-dbs have been built by Stack itself. If I'm right, that means using Nix Haskell packages as a cache is not really doable right now.

[ericsagnes]

Thanks for clarifying the documentation and the explanations.
I hope this will be implemented at some point as not having to compile packages is a very handy feature.

[mboes]

The motto for Stack's Nix integration is: let Nix manage system dependencies, let Stack manage Haskell dependencies. It makes a lot of sense to keep things that way, because that means Stack can exploit language specific knowledge that Nix cannot.

Having Nix manage all dependencies is a different project entirely I think, but you might be interested in taking that on. :)

[puffnfresh]

@mboes I think Nix has a lot of Haskell specific knowledge. What does Stack exploit which Nix does not?

@YPares is this issue closed because Stack shouldn't use nixpkgs' Stackage expressions? Is there a benefit to letting Stack resolve the dependencies?

[nh2]

A while later, but still grabbing a chance of answering unanswered questions:

What does Stack exploit which Nix does not?

@puffnfresh Stack can resume incremental builds, and re-use existing build dirs of completed builds when builds have to be done with changed inputs or changed build options, to make builds more incremental/faster, while Nix would do a full clean rebuild for such cases. Stack can also resolve package versions that are not in a Stackage release, and get and install their out-of-stackage dependencies, which nixpkgs can't do (it naturally only supports the packages and versions you declare, or generate). From that perspective, stack is more flexible. But stack cannot guarantee the up-to-date-ness and reproducibility that Nix gives (e.g. stack cannot in all cases determine when globally system libraries change and recompile accordingly; Nix can do this).

Is there a benefit to letting Stack resolve the dependencies?

Yes, it's the "Stack can also resolve package versions that are not in a Stackage release" part I mentioned above.