From 514abbe9b59f7bd1ef1b5e8358dc352e06b0c26e Mon Sep 17 00:00:00 2001 From: David Baynard Date: Tue, 8 Jan 2019 14:01:27 +0000 Subject: [PATCH 1/5] Document how stack uses the Cabal library in FAQ Based on conversation with @snoyberg --- ChangeLog.md | 1 + doc/faq.md | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+) diff --git a/ChangeLog.md b/ChangeLog.md index e1deb97ef1..7ec99ff521 100644 --- a/ChangeLog.md +++ b/ChangeLog.md @@ -14,6 +14,7 @@ Other enhancements: * Travis no longer builds using `sudo: false` as that behaviour is due to be deprecated. * Add `--cabal-files` flag to `stack ide targets` command. * Add `--stdout` flag to all `stack ide` subcommands. +- Document the way stack interacts with the Cabal library. Bug fixes: diff --git a/doc/faq.md b/doc/faq.md index fbfe9febca..8a254463f2 100644 --- a/doc/faq.md +++ b/doc/faq.md @@ -57,6 +57,65 @@ directory. None of this should affect any existing Haskell tools at all. there may be more options here for new projects in the future (see issue [253](https://github.com/commercialhaskell/stack/issues/253)) +### Cabal-the-library + +The relationship between stack and Cabal-the-library is complicated. + +1. Stack itself builds against a version of the Cabal library, which it uses for parsing Cabal files. + +2. Stack wraps functionality from a version of the Cabal library in order to build packages. + + i. For packages with `build-type: Simple`, Stack uses a setup executable compiled with a version of the Cabal library to perform builds. All packages using the same compiler and Cabal version are built with the same executable. These executables are cached in the `setup-exe-cache` configuration directory. + + ii. For packages with `build-type: Custom`, Stack compiles `Setup.hs` against a version of the Cabal library and uses that setup executable to perform builds. The executable is stored TODO + +3. Packages themselves may depend on the Cabal library. + +There are three (nearly) corresponding versions of the Cabal library which are relevant to a build. + +1. The version used to compile stack, which is not necessarily present on stack users’ machines. + + Stack will always be compiled using the most recent stable version of Cabal-the-library. + +2. The [boot version used by GHC](https://ghc.haskell.org/trac/ghc/wiki/Commentary/Libraries/VersionHistory), which is globally available to stack. + + This version is used to compile the `build-type: Simple` setup executable. + Build artefacts are placed in corresponding `.stack-work/dist/Cabal-xxxxx` directory. + +3. The snapshot version (which may be overridden using `extra-deps`). + + This is the version on which a package may depend. + + However, it is also used to compile any `build-type: Custom` setup executables. + +In addition, the Cabal version required by `custom-build` will be installed, if necessary. + +Note that these versions may be the same. + +There are a number of consequences of this design. + +1. Snapshot packages only depend on the GHC compiler version, not the Cabal library version (with the exception of `build-type: Custom` packages). + +2. The most recent stack can always read the most recent Cabal files. + +3. However, stack may not be able to build packages defined using those files. + + This occurs when + + i. The package uses `build-type: Simple` and the Cabal file format requires a more recent version of Cabal than the global (boot) version for the compiler. + + e.g. Stack lts-11.22 uses GHC 8.2.2, corresponding to Cabal 2.0.1.0. + A library using `build-type: Simple` and SPDX license identifiers (introduced in Cabal 2.2) will not build, though stack will process the Cabal file correctly. + + Cabal-simple_mPHDZzAJ_2.0.1.0_ghc-8.2.2: ./file.cabal:7: Parse of field 'license' failed. + +4. Stack only occasionally needs to build Cabal-the-library. + This is a resource intensive build, so avoiding it improves performance. + +5. Stack uses `ghc-pkg` to identify the Cabal version it should use for the setup executable for a build. + The build output is stored in a directory `.stack-work/dist/Cabal-xxxxx` named for the Cabal version used to compile the setup executable. + Together, this means it is only possible to know for sure which Cabal version ought to be used if the corresponding compiler is installed. + ## I need to use a different version of a package than what is provided by the LTS Haskell snapshot I'm using, what should I do? You can make tweaks to a snapshot by modifying the `extra-deps` configuration value in your `stack.yaml` file, e.g.: From 2766305c475416da41e803e26ea781bb4c80ab1e Mon Sep 17 00:00:00 2001 From: David Baynard Date: Sun, 20 Jan 2019 21:09:21 +0000 Subject: [PATCH 2/5] Address standalone reviewer comments The remaining comments will be addressed through a refactor of the text. --- doc/faq.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/doc/faq.md b/doc/faq.md index 8a254463f2..1c0b1d73f4 100644 --- a/doc/faq.md +++ b/doc/faq.md @@ -63,7 +63,7 @@ The relationship between stack and Cabal-the-library is complicated. 1. Stack itself builds against a version of the Cabal library, which it uses for parsing Cabal files. -2. Stack wraps functionality from a version of the Cabal library in order to build packages. +2. Stack builds a `Setup.hs` file against a version of the Cabal library, in order to build packages. i. For packages with `build-type: Simple`, Stack uses a setup executable compiled with a version of the Cabal library to perform builds. All packages using the same compiler and Cabal version are built with the same executable. These executables are cached in the `setup-exe-cache` configuration directory. @@ -75,7 +75,7 @@ There are three (nearly) corresponding versions of the Cabal library which are r 1. The version used to compile stack, which is not necessarily present on stack users’ machines. - Stack will always be compiled using the most recent stable version of Cabal-the-library. + Stack releases will be compiled using the most recent version of Cabal-the-library where possible, in order to support the most recent versions of the Cabal file format. 2. The [boot version used by GHC](https://ghc.haskell.org/trac/ghc/wiki/Commentary/Libraries/VersionHistory), which is globally available to stack. @@ -96,7 +96,7 @@ There are a number of consequences of this design. 1. Snapshot packages only depend on the GHC compiler version, not the Cabal library version (with the exception of `build-type: Custom` packages). -2. The most recent stack can always read the most recent Cabal files. +2. The most recent stack can usually read the most recent Cabal files. 3. However, stack may not be able to build packages defined using those files. @@ -116,6 +116,8 @@ There are a number of consequences of this design. The build output is stored in a directory `.stack-work/dist/Cabal-xxxxx` named for the Cabal version used to compile the setup executable. Together, this means it is only possible to know for sure which Cabal version ought to be used if the corresponding compiler is installed. + This behaviour will change with `stack` 2.0. + ## I need to use a different version of a package than what is provided by the LTS Haskell snapshot I'm using, what should I do? You can make tweaks to a snapshot by modifying the `extra-deps` configuration value in your `stack.yaml` file, e.g.: From f31e9dd962aefa6c11854496ff578244618a9623 Mon Sep 17 00:00:00 2001 From: David Baynard Date: Sun, 20 Jan 2019 21:28:14 +0000 Subject: [PATCH 3/5] Refactor documentation addition corresponding to the Cabal library --- doc/faq.md | 41 ++++++++++++++++++++++------------------- 1 file changed, 22 insertions(+), 19 deletions(-) diff --git a/doc/faq.md b/doc/faq.md index 1c0b1d73f4..90d36243fc 100644 --- a/doc/faq.md +++ b/doc/faq.md @@ -61,36 +61,39 @@ directory. None of this should affect any existing Haskell tools at all. The relationship between stack and Cabal-the-library is complicated. -1. Stack itself builds against a version of the Cabal library, which it uses for parsing Cabal files. +#### Parsing Cabal files -2. Stack builds a `Setup.hs` file against a version of the Cabal library, in order to build packages. +Stack itself builds against a version of the Cabal library, which it uses for parsing Cabal files. - i. For packages with `build-type: Simple`, Stack uses a setup executable compiled with a version of the Cabal library to perform builds. All packages using the same compiler and Cabal version are built with the same executable. These executables are cached in the `setup-exe-cache` configuration directory. +This version is not necessarily present on stack users’ machines. - ii. For packages with `build-type: Custom`, Stack compiles `Setup.hs` against a version of the Cabal library and uses that setup executable to perform builds. The executable is stored TODO +This version determines which version of the Cabal file format stack is able to parse. +Where possible, releases of stack will be compiled using the most recent version of +Cabal-the-library, in order to support the most recent versions of the Cabal file format. -3. Packages themselves may depend on the Cabal library. +#### Building packages -There are three (nearly) corresponding versions of the Cabal library which are relevant to a build. +Stack builds a `Setup.hs` file against a version of the Cabal library, in order to build packages. -1. The version used to compile stack, which is not necessarily present on stack users’ machines. +The [boot version used by GHC](https://ghc.haskell.org/trac/ghc/wiki/Commentary/Libraries/VersionHistory), +which is globally available to stack, is used to compile the `build-type: Simple` setup executable. +All packages using the same compiler and Cabal version are built with the same executable. +These executables are cached in the `setup-exe-cache` configuration directory. - Stack releases will be compiled using the most recent version of Cabal-the-library where possible, in order to support the most recent versions of the Cabal file format. +Build artefacts are placed in the corresponding `.stack-work/dist/Cabal-xxxxx` directory. -2. The [boot version used by GHC](https://ghc.haskell.org/trac/ghc/wiki/Commentary/Libraries/VersionHistory), which is globally available to stack. +For packages with `build-type: Custom`, Stack compiles `Setup.hs` against the version of the Cabal +library present in the snapshot (which may be overridden using `extra-deps`), and uses that +setup executable to perform builds. This treats Cabal as any other dependency package. +The executable is stored TODO - This version is used to compile the `build-type: Simple` setup executable. - Build artefacts are placed in corresponding `.stack-work/dist/Cabal-xxxxx` directory. +#### Importing Cabal as a library -3. The snapshot version (which may be overridden using `extra-deps`). +Packages may themselves depend on the Cabal library. - This is the version on which a package may depend. +As any other dependency, they will use the snapshot version (which may be overridden using `extra-deps`). - However, it is also used to compile any `build-type: Custom` setup executables. - -In addition, the Cabal version required by `custom-build` will be installed, if necessary. - -Note that these versions may be the same. +#### ⁂ There are a number of consequences of this design. @@ -116,7 +119,7 @@ There are a number of consequences of this design. The build output is stored in a directory `.stack-work/dist/Cabal-xxxxx` named for the Cabal version used to compile the setup executable. Together, this means it is only possible to know for sure which Cabal version ought to be used if the corresponding compiler is installed. - This behaviour will change with `stack` 2.0. + This behaviour will change with stack 2.0. ## I need to use a different version of a package than what is provided by the LTS Haskell snapshot I'm using, what should I do? From 9b4f596502d3fcb24b55fdcce5ae3efef0856a59 Mon Sep 17 00:00:00 2001 From: David Baynard Date: Sun, 10 Feb 2019 17:58:51 +0000 Subject: [PATCH 4/5] Describe the behaviour of stack with build-type: Custom --- doc/faq.md | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/doc/faq.md b/doc/faq.md index 90d36243fc..cfe235bb14 100644 --- a/doc/faq.md +++ b/doc/faq.md @@ -75,7 +75,7 @@ Cabal-the-library, in order to support the most recent versions of the Cabal fil Stack builds a `Setup.hs` file against a version of the Cabal library, in order to build packages. -The [boot version used by GHC](https://ghc.haskell.org/trac/ghc/wiki/Commentary/Libraries/VersionHistory), +The [boot version of Cabal used by GHC](https://ghc.haskell.org/trac/ghc/wiki/Commentary/Libraries/VersionHistory), which is globally available to stack, is used to compile the `build-type: Simple` setup executable. All packages using the same compiler and Cabal version are built with the same executable. These executables are cached in the `setup-exe-cache` configuration directory. @@ -85,7 +85,17 @@ Build artefacts are placed in the corresponding `.stack-work/dist/Cabal-xxxxx` d For packages with `build-type: Custom`, Stack compiles `Setup.hs` against the version of the Cabal library present in the snapshot (which may be overridden using `extra-deps`), and uses that setup executable to perform builds. This treats Cabal as any other dependency package. -The executable is stored TODO +The process is as follows: + +1. Stack uses the boot version of Cabal to build the required version of Cabal, which is treated as though built with `build-type: Simple`. + This will take a short while — typically a few minutes. + +2. Stack uses this version of Cabal to build the setup executable from the `Setup.hs` file. + The resulting executable is placed in the `.stack-work/dist/Cabal-xxxxx` directory corresponding to the boot version of Cabal — even though it was built with the snapshot version. + +3. This setup executable builds the package. + Again, build artefacts are placed in the `.stack-work/dist/Cabal-xxxxx` directory — even though it was built with the snapshot version. + A resulting executable is copied to the `.stack-work/install/$compiler-variant/$snapshot/$compiler-version/bin` directory. #### Importing Cabal as a library @@ -93,7 +103,7 @@ Packages may themselves depend on the Cabal library. As any other dependency, they will use the snapshot version (which may be overridden using `extra-deps`). -#### ⁂ +#### What this means There are a number of consequences of this design. @@ -116,7 +126,7 @@ There are a number of consequences of this design. This is a resource intensive build, so avoiding it improves performance. 5. Stack uses `ghc-pkg` to identify the Cabal version it should use for the setup executable for a build. - The build output is stored in a directory `.stack-work/dist/Cabal-xxxxx` named for the Cabal version used to compile the setup executable. + The build output is stored in a directory `.stack-work/dist/Cabal-xxxxx` named for the boot version of Cabal corresponding to the GHC version.. Together, this means it is only possible to know for sure which Cabal version ought to be used if the corresponding compiler is installed. This behaviour will change with stack 2.0. From b18158cbd774072b92876ac1545195f5cfd646bf Mon Sep 17 00:00:00 2001 From: David Baynard Date: Mon, 11 Feb 2019 16:10:11 +0000 Subject: [PATCH 5/5] Move documentation on Cabal from FAQ to Architecture document --- doc/architecture.md | 74 +++++++++++++++++++++++++++++++++++++++++- doc/faq.md | 79 +++------------------------------------------ 2 files changed, 78 insertions(+), 75 deletions(-) diff --git a/doc/architecture.md b/doc/architecture.md index 2182d2e224..069f633c72 100644 --- a/doc/architecture.md +++ b/doc/architecture.md @@ -110,7 +110,79 @@ authoritative on this and should be consulted. The basic idea though is: Once we have the plan, execution is a relatively simple process of calling `runghc Setup.hs` in the correct order with the correct parameters. See -Stack.Build.Execute for more information. +`Stack.Build.Execute` for the implementation. + +The `Setup.hs` file uses the Cabal library to build packages. + +#### Parsing Cabal files + +Stack itself builds against a version of the Cabal library, which it uses for parsing Cabal files. + +This version is not necessarily present on stack users’ machines. + +This version determines which version of the Cabal file format stack is able to parse. +Where possible, releases of stack will be compiled using the most recent version of +Cabal-the-library, in order to support the most recent versions of the Cabal file format. + +#### Building packages + +The version of the Cabal library used to build packages Stack builds a `Setup.hs` file against a version of the Cabal library, in order to build packages. + +The [boot version of Cabal used by GHC](https://ghc.haskell.org/trac/ghc/wiki/Commentary/Libraries/VersionHistory), +which is globally available to stack, is used to compile the `build-type: Simple` setup executable. +All packages using the same compiler and Cabal version are built with the same executable. +These executables are cached in the `setup-exe-cache` configuration directory. + +Build artefacts are placed in the corresponding `.stack-work/dist/Cabal-xxxxx` directory. + +For packages with `build-type: Custom`, Stack compiles `Setup.hs` against the version of the Cabal +library present in the snapshot (which may be overridden using `extra-deps`), and uses that +setup executable to perform builds. This treats Cabal as any other dependency package. +The process is as follows: + +1. Stack uses the boot version of Cabal to build the required version of Cabal, which is treated as though built with `build-type: Simple`. + This will take a short while — typically a few minutes. + +2. Stack uses this version of Cabal to build the setup executable from the `Setup.hs` file. + The resulting executable is placed in the `.stack-work/dist/Cabal-xxxxx` directory corresponding to the boot version of Cabal — even though it was built with the snapshot version. + +3. This setup executable builds the package. + Again, build artefacts are placed in the `.stack-work/dist/Cabal-xxxxx` directory — even though it was built with the snapshot version. + A resulting executable is copied to the `.stack-work/install/$compiler-variant/$snapshot/$compiler-version/bin` directory. + +#### Importing Cabal as a library + +Packages may themselves depend on the Cabal library. + +As any other dependency, they will use the snapshot version (which may be overridden using `extra-deps`). + +#### What this means + +There are a number of consequences of this design. + +1. Snapshot packages only depend on the GHC compiler version, not the Cabal library version (with the exception of `build-type: Custom` packages). + +2. The most recent stack can usually read the most recent Cabal files. + +3. However, stack may not be able to build packages defined using those files. + + This occurs when + + i. The package uses `build-type: Simple` and the Cabal file format requires a more recent version of Cabal than the global (boot) version for the compiler. + + e.g. Stack lts-11.22 uses GHC 8.2.2, corresponding to Cabal 2.0.1.0. + A library using `build-type: Simple` and SPDX license identifiers (introduced in Cabal 2.2) will not build, though stack will process the Cabal file correctly. + + Cabal-simple_mPHDZzAJ_2.0.1.0_ghc-8.2.2: ./file.cabal:7: Parse of field 'license' failed. + +4. Stack only occasionally needs to build Cabal-the-library. + This is a resource intensive build, so avoiding it improves performance. + +5. Stack uses `ghc-pkg` to identify the Cabal version it should use for the setup executable for a build. + The build output is stored in a directory `.stack-work/dist/Cabal-xxxxx` named for the boot version of Cabal corresponding to the GHC version.. + Together, this means it is only possible to know for sure which Cabal version ought to be used if the corresponding compiler is installed. + + This behaviour will change with stack 2.0. ## Configuration diff --git a/doc/faq.md b/doc/faq.md index cfe235bb14..574a0954f5 100644 --- a/doc/faq.md +++ b/doc/faq.md @@ -44,7 +44,9 @@ directory. None of this should affect any existing Haskell tools at all. ## What is the relationship between stack and cabal? -* Cabal-the-library is used by stack to build your Haskell code. +* Cabal-the-library is used by stack to build your Haskell code. See the + [Architecture: Plan Execution](architecture.md#plan-execution) section for + more detail, including how the Cabal version is chosen. * cabal-install (the executable) is used by stack for its dependency solver functionality. * A .cabal file is provided for each package, and defines all package-level @@ -57,79 +59,8 @@ directory. None of this should affect any existing Haskell tools at all. there may be more options here for new projects in the future (see issue [253](https://github.com/commercialhaskell/stack/issues/253)) -### Cabal-the-library - -The relationship between stack and Cabal-the-library is complicated. - -#### Parsing Cabal files - -Stack itself builds against a version of the Cabal library, which it uses for parsing Cabal files. - -This version is not necessarily present on stack users’ machines. - -This version determines which version of the Cabal file format stack is able to parse. -Where possible, releases of stack will be compiled using the most recent version of -Cabal-the-library, in order to support the most recent versions of the Cabal file format. - -#### Building packages - -Stack builds a `Setup.hs` file against a version of the Cabal library, in order to build packages. - -The [boot version of Cabal used by GHC](https://ghc.haskell.org/trac/ghc/wiki/Commentary/Libraries/VersionHistory), -which is globally available to stack, is used to compile the `build-type: Simple` setup executable. -All packages using the same compiler and Cabal version are built with the same executable. -These executables are cached in the `setup-exe-cache` configuration directory. - -Build artefacts are placed in the corresponding `.stack-work/dist/Cabal-xxxxx` directory. - -For packages with `build-type: Custom`, Stack compiles `Setup.hs` against the version of the Cabal -library present in the snapshot (which may be overridden using `extra-deps`), and uses that -setup executable to perform builds. This treats Cabal as any other dependency package. -The process is as follows: - -1. Stack uses the boot version of Cabal to build the required version of Cabal, which is treated as though built with `build-type: Simple`. - This will take a short while — typically a few minutes. - -2. Stack uses this version of Cabal to build the setup executable from the `Setup.hs` file. - The resulting executable is placed in the `.stack-work/dist/Cabal-xxxxx` directory corresponding to the boot version of Cabal — even though it was built with the snapshot version. - -3. This setup executable builds the package. - Again, build artefacts are placed in the `.stack-work/dist/Cabal-xxxxx` directory — even though it was built with the snapshot version. - A resulting executable is copied to the `.stack-work/install/$compiler-variant/$snapshot/$compiler-version/bin` directory. - -#### Importing Cabal as a library - -Packages may themselves depend on the Cabal library. - -As any other dependency, they will use the snapshot version (which may be overridden using `extra-deps`). - -#### What this means - -There are a number of consequences of this design. - -1. Snapshot packages only depend on the GHC compiler version, not the Cabal library version (with the exception of `build-type: Custom` packages). - -2. The most recent stack can usually read the most recent Cabal files. - -3. However, stack may not be able to build packages defined using those files. - - This occurs when - - i. The package uses `build-type: Simple` and the Cabal file format requires a more recent version of Cabal than the global (boot) version for the compiler. - - e.g. Stack lts-11.22 uses GHC 8.2.2, corresponding to Cabal 2.0.1.0. - A library using `build-type: Simple` and SPDX license identifiers (introduced in Cabal 2.2) will not build, though stack will process the Cabal file correctly. - - Cabal-simple_mPHDZzAJ_2.0.1.0_ghc-8.2.2: ./file.cabal:7: Parse of field 'license' failed. - -4. Stack only occasionally needs to build Cabal-the-library. - This is a resource intensive build, so avoiding it improves performance. - -5. Stack uses `ghc-pkg` to identify the Cabal version it should use for the setup executable for a build. - The build output is stored in a directory `.stack-work/dist/Cabal-xxxxx` named for the boot version of Cabal corresponding to the GHC version.. - Together, this means it is only possible to know for sure which Cabal version ought to be used if the corresponding compiler is installed. - - This behaviour will change with stack 2.0. +For detail on the differences between a `stack.yaml` and Cabal package file, see +[stack.yaml vs cabal package file](stack_yaml_vs_cabal_package_file.md). ## I need to use a different version of a package than what is provided by the LTS Haskell snapshot I'm using, what should I do?