From 43abb24d8ef78d428046a0feca35177539d50243 Mon Sep 17 00:00:00 2001 From: Caspar van Leeuwen Date: Mon, 7 Oct 2024 13:16:25 +0200 Subject: [PATCH 1/9] Using EESSI extend to build on top of EESSI --- docs/using_eessi/building_on_eessi.md | 114 ++++++++++++++++++-------- 1 file changed, 80 insertions(+), 34 deletions(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 92d599bb1e..c2aa95b1bd 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -6,67 +6,113 @@ Building on top of EESSI with EasyBuild is relatively straightforward. One cruci ### Starting the EESSI software environment Start your environment as described [here](../using_eessi/setting_up_environment.md) -### Configure EasyBuild -To configure EasyBuild, first, check out the [EESSI software-layer repository](https://github.com/EESSI/software-layer.git). We advise you to check out the branch corresponding to the version of EESSI you would like to use. +### Using the EESSI-extend module +The `EESSI-extend` module facilitates building on top of EESSI using EasyBuild. It does a few key things: -If you are unsure which version you are using, you can run +1. It configures EasyBuild to match how the rest of the EESSI software is build +2. It configures EasyBuild to use a certain installation path (e.g. in your homedir), taking into account the hardware architecture you are building on +3. It adds the relevant subdirectory from your installation path to your `MODULEPATH`, to make sure your newly installed modules are available +4. It loads the EasyBuild module + +The `EESSI-extend` module recognizes a few environment variables. To print an up-to-date list, check the module itself ``` -echo ${EESSI_VERSION} +module help EESSI-extend/2023.06-easybuild ``` -to check it. -To build on top of e.g. version `2023.06` of the EESSI software stack, we check it out, and go into that directory: +The key environment variables it will respect are: + - EESSI_USER_INSTALL can be set to a location to install modules for use by the user only. The location must already exist on the filesystem. Set this environment variable if you're an end-user who wants to install additional software just for yourself. + - EESSI_PROJECT_INSTALL can be set to a location to install modules for use by a project. The location must already exist on the filesystem and you should ensure that the location has the correct Linux group and the SGID permission is set on that directory (`chmod g+s $EESSI_PROJECT_INSTALL`) so that all members of the group have permission to read and write installations. Set this environment variable if you're an end-user who's part of a project, and wants to create additional installations that should be available to all project members. + - EESSI_SITE_INSTALL is either defined or not and cannot be used with another environment variable. A site installation is done in a defined location and any installations there are (by default) world readable. Set this environmnet variable if you are a site hosting a system that has EESSI available, and you want to build on top to make additional modules available to all of your users. + - EESSI_CVMFS_INSTALL is either defined or not and cannot be used with another environment variable. A CVMFS installation targets a defined location which will be ingested into CVMFS and is only useful for CVMFS administrators. Set this environment variable if you manage your own CVMFS repository where the software has to be built on top of EESSI. + +Note that if none of the environment variables above are defined, an EESSI_USER_INSTALL is assumed with a value of $HOME/EESSI. + +Here, we assume you are just an end-user and load the `EESSI-extend` module with the default installation prefix: ``` -git clone https://github.com/EESSI/software-layer/ --branch 2023.06 -cd software-layer -``` -Then, you have to pick a working directory (that you have write access to) where EasyBuild can do the build, and an install directory (with sufficient storage space), where EasyBuild can install it. In this example, we create a temporary directory in `/tmp/` as our working directory, and use `$HOME/.local/easybuild` as our installpath: -``` -export WORKDIR=$(mktemp --directory --tmpdir=/tmp -t eessi-build.XXXXXXXXXX) -source configure_easybuild -export EASYBUILD_INSTALLPATH="${HOME}/.local/easybuild" -``` -Next, you load the EasyBuild module that you want to use, e.g. -``` -module load EasyBuild/4.8.2 +module load EESSI-extend/2023.06-easybuild ``` -Finally, you can check the current configuration for EasyBuild using + +Now, if we check the EasyBuild configuration + + ``` eb --show-config +allow-loaded-modules (E) = EasyBuild, EESSI-extend +buildpath (E) = /tmp//easybuild/build +containerpath (E) = /tmp//easybuild/containers +debug (E) = True +experimental (E) = True +filter-deps (E) = Autoconf, Automake, Autotools, binutils, bzip2, DBus, flex, gettext, gperf, help2man, intltool, libreadline, libtool, M4, makeinfo, ncurses, util-linux, XZ, zlib +filter-env-vars (E) = LD_LIBRARY_PATH +hooks (E) = /cvmfs/software.eessi.io/versions/2023.06/init/easybuild/eb_hooks.py +ignore-osdeps (E) = True +installpath (E) = /home//eessi/versions/2023.06/software/linux/x86_64/amd/zen2 +module-extensions (E) = True +packagepath (E) = /tmp//easybuild/packages +prefix (E) = /tmp//easybuild +read-only-installdir (E) = True +repositorypath (E) = /tmp//easybuild/ebfiles_repo +robot-paths (D) = /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/EasyBuild/4.9.4/easybuild/easyconfigs +rpath (E) = True +sourcepath (E) = /tmp//easybuild/sources +sticky-bit (E) = True +sysroot (E) = /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64 +trace (E) = True +umask (E) = 077 +zip-logs (E) = bzip2 +``` + +Apart from the `installpath`, this is exactly how EasyBuild is configured when software is built for EESSI itself. + +!!! note + Be aware that `EESSI-extend` will optimize the installation for your current hardware architecture, and the `installpath` also contains this architecture in it's directory structure (just like regular EESSI installations do). This means you should run the installation on the node type on which you also want to use the software. If you want the installation to be present for multiple node types, you can simply run it once on each type of node. + +And, if we check our `MODULEPATH`, we see that the `installpath` that EasyBuild will use here is prepended +``` +$ echo $MODULEPATH +/home//eessi/versions/2023.06/software/linux/x86_64/amd/zen2/modules/all:... ``` -!!! Note - We use EasyBuild's default behaviour in optimizing for the host architecture. Since the EESSI initialization script also loads the EESSI stack that is optimized for your host architecture, this matches nicely. However, if you work on a cluster with heterogeneous node types, you have to realize you can only use these builds on the same architecture as where you build them. You can use different `EASYBUILD_INSTALLPATH`s if you want to build for different host architectures. For example, when you are on a system that has a mix of `AMD zen3` and `AMD zen4` nodes, you might want to use `EASYBUILD_INSTALLPATH=$HOME/.local/easybuild/zen3` when building on a `zen3` node, `EASYBUILD_INSTALLPATH=$HOME/.local/easybuild/zen4` when building on a `zen4` node. Then, in the step beloww, instead of the `module use` command listed there, you can use `module use $HOME/.local/easybuild/zen3/modules/all` when you want to run on a `zen3` node and `module use $HOME/.local/easybuild/zen4/modules/all` when you want to run on a `zen4` node. - ### Building -Now, you are ready to build. For example, at the time of writing, `netCDF-4.9.0-gompi-2022a.eb` was not in the EESSI environment yet, so you can build it yourself: +Now, you are ready to build. For example, suppose you want to install `netcdf4-python-1.6.5-foss-2023b.eb` (which is not present at the time of writing), you run: + ``` -eb netCDF-4.9.0-gompi-2022a.eb +eb netcdf4-python-1.6.5-foss-2023b.eb ``` !!! Note - If this netCDF module is available by the time you are trying, you can force a local rebuild by adding the `--rebuild` argument in order to experiment with building locally, or pick a different EasyConfig to build. + If this netCDF for python module is available by the time you are trying, you can force a local rebuild by adding the `--rebuild` argument in order to experiment with building locally, or pick a different EasyConfig to build. ### Using the newly built module -First, you'll need to add the subdirectory of the `EASYBUILD_INSTALLPATH` that contains the modules to the `MODULEPATH`. You can do that using: +Our new module is readily available on the `MODULEPATH`: ``` -module use ${EASYBUILD_INSTALLPATH}/modules/all -``` +$ module av netcdf4-python/1.6.5-foss-2023b -you may want to do this as part of your `.bashrc`. +---- /home//eessi/versions/2023.06/software/linux/x86_64/amd/zen2/modules/all ---- + netcdf4-python/1.6.5-foss-2023b +``` -!!! Note - Be careful adding to the `MODULEPATH` in your `.bashrc` if you are on a cluster with heterogeneous architectures. You don't want to pick up on a module that was not compiled for the correct architectures accidentally. +Note however that it is the `EESSI-extend` module that added this to our `MODULEPATH`. If we unload that module, the user-installation is no longer on the `MODULEPATH` +``` +$ module av netcdf4-python/1.6.5-foss-2023b +No module(s) or extension(s) found! +``` -Since your module is built on top of the EESSI environment, that needs to be loaded first (as described [here](../using_eessi/setting_up_environment.md)), if you haven't already done so. +This means you'll _always_ need to load the `EESSI-extend` module if you want to use these modules. For example, if you want to load this module in a job, you should have -Finally, you should be able to load our newly build module: ``` -module load netCDF/4.9.0-gompi-2022a +# Loading EESSI-extend makes the netcdf4-python module available on the MODULEPATH +module load EESSI-extend/2023.06-easybuild +module load netcdf4-python/1.6.5-foss-2023b ``` +in your job script. + +!!! Note + The `EESSI-extend` module does _not_ add `EESSI_SITE_INSTALL` to the `MODULEPATH`. As a system administrator, you will have to make sure that's on the `MODULEPATH` of your users through other means if you want your users to be able to use module installed by `EESSI-extend`. + ## Manually building software op top of EESSI Building software on top of EESSI would require your linker to use the same system-dependencies as the software in EESSI does. In other words: it requires you to link against libraries from the compatibility layer, instead of from your host OS. From 82dc0d14765505f4afbace1132ddc15a0c3115c3 Mon Sep 17 00:00:00 2001 From: Caspar van Leeuwen <33718780+casparvl@users.noreply.github.com> Date: Mon, 7 Oct 2024 14:17:51 +0200 Subject: [PATCH 2/9] Properly quite stuff. Fix grammar error. Co-authored-by: ocaisa --- docs/using_eessi/building_on_eessi.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index c2aa95b1bd..d4212671cb 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -9,7 +9,7 @@ Start your environment as described [here](../using_eessi/setting_up_environment ### Using the EESSI-extend module The `EESSI-extend` module facilitates building on top of EESSI using EasyBuild. It does a few key things: -1. It configures EasyBuild to match how the rest of the EESSI software is build +1. It configures EasyBuild to match how the rest of the EESSI software is built 2. It configures EasyBuild to use a certain installation path (e.g. in your homedir), taking into account the hardware architecture you are building on 3. It adds the relevant subdirectory from your installation path to your `MODULEPATH`, to make sure your newly installed modules are available 4. It loads the EasyBuild module @@ -20,12 +20,12 @@ module help EESSI-extend/2023.06-easybuild ``` The key environment variables it will respect are: - - EESSI_USER_INSTALL can be set to a location to install modules for use by the user only. The location must already exist on the filesystem. Set this environment variable if you're an end-user who wants to install additional software just for yourself. - - EESSI_PROJECT_INSTALL can be set to a location to install modules for use by a project. The location must already exist on the filesystem and you should ensure that the location has the correct Linux group and the SGID permission is set on that directory (`chmod g+s $EESSI_PROJECT_INSTALL`) so that all members of the group have permission to read and write installations. Set this environment variable if you're an end-user who's part of a project, and wants to create additional installations that should be available to all project members. - - EESSI_SITE_INSTALL is either defined or not and cannot be used with another environment variable. A site installation is done in a defined location and any installations there are (by default) world readable. Set this environmnet variable if you are a site hosting a system that has EESSI available, and you want to build on top to make additional modules available to all of your users. + - `EESSI_USER_INSTALL` can be set to a location to install modules for use by the user only. The location must already exist on the filesystem. Set this environment variable if you're an end-user who wants to install additional software just for yourself. + - `EESSI_PROJECT_INSTALL` can be set to a location to install modules for use by a project. The location must already exist on the filesystem and you should ensure that the location has the correct Linux group and the SGID permission is set on that directory (`chmod g+s $EESSI_PROJECT_INSTALL`) so that all members of the group have permission to read and write installations. Set this environment variable if you're an end-user who's part of a project, and wants to create additional installations that should be available to all project members. + - `EESSI_SITE_INSTALL` is either defined or not and cannot be used with another environment variable. A site installation is done in a defined location and any installations there are (by default) world readable. Set this environment variable if you are a site hosting a system that has EESSI available, and you want to build on top to make additional modules available to all of your users. - EESSI_CVMFS_INSTALL is either defined or not and cannot be used with another environment variable. A CVMFS installation targets a defined location which will be ingested into CVMFS and is only useful for CVMFS administrators. Set this environment variable if you manage your own CVMFS repository where the software has to be built on top of EESSI. -Note that if none of the environment variables above are defined, an EESSI_USER_INSTALL is assumed with a value of $HOME/EESSI. +Note that if none of the environment variables above are defined, an `EESSI_USER_INSTALL` is assumed with a value of `$HOME/EESSI`. Here, we assume you are just an end-user and load the `EESSI-extend` module with the default installation prefix: From 6ea835e32c3967c50e12ea7d66c988e473028074 Mon Sep 17 00:00:00 2001 From: Caspar van Leeuwen Date: Mon, 7 Oct 2024 16:24:30 +0200 Subject: [PATCH 3/9] Now correctly reflect what these environment variables do. And in which scenarios the EESSI-extend module should be loaded in order to have additionally installed modules available (i.e. when it was a project or user installation) --- docs/using_eessi/building_on_eessi.md | 40 ++++++++++++--------------- 1 file changed, 18 insertions(+), 22 deletions(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index c2aa95b1bd..70758fd001 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -19,15 +19,14 @@ The `EESSI-extend` module recognizes a few environment variables. To print an up module help EESSI-extend/2023.06-easybuild ``` -The key environment variables it will respect are: - - EESSI_USER_INSTALL can be set to a location to install modules for use by the user only. The location must already exist on the filesystem. Set this environment variable if you're an end-user who wants to install additional software just for yourself. - - EESSI_PROJECT_INSTALL can be set to a location to install modules for use by a project. The location must already exist on the filesystem and you should ensure that the location has the correct Linux group and the SGID permission is set on that directory (`chmod g+s $EESSI_PROJECT_INSTALL`) so that all members of the group have permission to read and write installations. Set this environment variable if you're an end-user who's part of a project, and wants to create additional installations that should be available to all project members. - - EESSI_SITE_INSTALL is either defined or not and cannot be used with another environment variable. A site installation is done in a defined location and any installations there are (by default) world readable. Set this environmnet variable if you are a site hosting a system that has EESSI available, and you want to build on top to make additional modules available to all of your users. - - EESSI_CVMFS_INSTALL is either defined or not and cannot be used with another environment variable. A CVMFS installation targets a defined location which will be ingested into CVMFS and is only useful for CVMFS administrators. Set this environment variable if you manage your own CVMFS repository where the software has to be built on top of EESSI. +The installation prefix is determined by `EESSI-extend` through the following logic: +1. If `EESSI_CVMFS_INSTALL` is set, software inst installed in `EESSI_SOFTWARE_PATH`. This variable shouldn't be used by users and would only be used by CVMFS administrators of the EESSI repository. +2. If `EESSI_SITE_INSTALL` is set (and this is set by default when the EESSI environment is initialized) and the EESSI site installation prefix (`/cvmfs/software.eessi.io/host_injections/...`) is writeable, this prefix will be used. This is typically where sites hosting a system that has EESSI deployed would install additional software on top of EESSI and make it available to all their users. +3. If `EESSI_PROJECT_INSTALL` is set (and `EESSI_USER_INSTALL` is not set), this prefix will be used. You should use this if you want to install additional software on top of EESSI that should also be useable by your project partners on the same system. +4. If `EESSI_USER_INSTALL` is set, this prefix will be used. You should use this if yo uwant to install additional software on top of EESSI just for your own user. +If none of the above apply, the default is a user installation in `$HOME/EESSI` (i.e. effectively the same as setting `EESSI_USER_INSTALL=$HOME/EESSI`). -Note that if none of the environment variables above are defined, an EESSI_USER_INSTALL is assumed with a value of $HOME/EESSI. - -Here, we assume you are just an end-user and load the `EESSI-extend` module with the default installation prefix: +Here, we assume you are just an end-user, not having set any of the above environment variables, and loading the `EESSI-extend` module with the default installation prefix: ``` module load EESSI-extend/2023.06-easybuild @@ -86,32 +85,29 @@ eb netcdf4-python-1.6.5-foss-2023b.eb ### Using the newly built module -Our new module is readily available on the `MODULEPATH`: -``` -$ module av netcdf4-python/1.6.5-foss-2023b +If the installation was done in the site installation path (i.e. `EESSI_SITE_INSTALL` was set, and things were installed in `/cvmfs/software.eessi.io/host_injections/...`), the modules are available by default to anyone who has initialized the EESSI software environment. ----- /home//eessi/versions/2023.06/software/linux/x86_64/amd/zen2/modules/all ---- - netcdf4-python/1.6.5-foss-2023b -``` +If the installation through `EESSI-extend` was done in a `EESSI_PROJECT_INSTALL` or `EESSI_USER_INSTALL` location, one has to make sure to load the `EESSI-extend` module before loading the module of interest, since this adds those prefixes to the `MODULEPATH`. -Note however that it is the `EESSI-extend` module that added this to our `MODULEPATH`. If we unload that module, the user-installation is no longer on the `MODULEPATH` +If we don't have the `EESSI-extend` module loaded, it will not find any modules installed in the `EESSI_PROJECT_INSTALL` or `EESSI_USER_INSTALL` locations: ``` +$ module unload EESSI-extend $ module av netcdf4-python/1.6.5-foss-2023b No module(s) or extension(s) found! ``` -This means you'll _always_ need to load the `EESSI-extend` module if you want to use these modules. For example, if you want to load this module in a job, you should have +But, if we load `EESSI-extend` first: ``` -# Loading EESSI-extend makes the netcdf4-python module available on the MODULEPATH -module load EESSI-extend/2023.06-easybuild -module load netcdf4-python/1.6.5-foss-2023b +$ module load EESSI-extend/2023.06-easybuild +$ module av netcdf4-python/1.6.5-foss-2023b + +---- /home//eessi/versions/2023.06/software/linux/x86_64/amd/zen2/modules/all ---- + netcdf4-python/1.6.5-foss-2023b ``` -in your job script. -!!! Note - The `EESSI-extend` module does _not_ add `EESSI_SITE_INSTALL` to the `MODULEPATH`. As a system administrator, you will have to make sure that's on the `MODULEPATH` of your users through other means if you want your users to be able to use module installed by `EESSI-extend`. +This means you'll _always_ need to load the `EESSI-extend` module if you want to use these modules (also, and particularly when you want to use them in a job script). ## Manually building software op top of EESSI Building software on top of EESSI would require your linker to use the same system-dependencies as the software in EESSI does. In other words: it requires you to link against libraries from the compatibility layer, instead of from your host OS. From 92802aa65cf8dff219498a1059628270faa725de Mon Sep 17 00:00:00 2001 From: Caspar van Leeuwen Date: Mon, 7 Oct 2024 16:30:17 +0200 Subject: [PATCH 4/9] Fix grammar --- docs/using_eessi/building_on_eessi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 3e4ac49831..9f33a346c9 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -22,7 +22,7 @@ module help EESSI-extend/2023.06-easybuild The installation prefix is determined by `EESSI-extend` through the following logic: 1. If `EESSI_CVMFS_INSTALL` is set, software inst installed in `EESSI_SOFTWARE_PATH`. This variable shouldn't be used by users and would only be used by CVMFS administrators of the EESSI repository. 2. If `EESSI_SITE_INSTALL` is set (and this is set by default when the EESSI environment is initialized) and the EESSI site installation prefix (`/cvmfs/software.eessi.io/host_injections/...`) is writeable, this prefix will be used. This is typically where sites hosting a system that has EESSI deployed would install additional software on top of EESSI and make it available to all their users. -3. If `EESSI_PROJECT_INSTALL` is set (and `EESSI_USER_INSTALL` is not set), this prefix will be used. You should use this if you want to install additional software on top of EESSI that should also be useable by your project partners on the same system. +3. If `EESSI_PROJECT_INSTALL` is set (and `EESSI_USER_INSTALL` is not set), this prefix will be used. You should use this if you want to install additional software on top of EESSI that should also be usable by your project partners on the same system. 4. If `EESSI_USER_INSTALL` is set, this prefix will be used. You should use this if yo uwant to install additional software on top of EESSI just for your own user. If none of the above apply, the default is a user installation in `$HOME/EESSI` (i.e. effectively the same as setting `EESSI_USER_INSTALL=$HOME/EESSI`). From aaf126412da679a962e04bb1d59ebba8c42c556b Mon Sep 17 00:00:00 2001 From: Caspar van Leeuwen <33718780+casparvl@users.noreply.github.com> Date: Tue, 8 Oct 2024 15:05:09 +0200 Subject: [PATCH 5/9] Apply suggestions from code review Co-authored-by: ocaisa --- docs/using_eessi/building_on_eessi.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 9f33a346c9..d01109c44c 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -20,8 +20,8 @@ module help EESSI-extend/2023.06-easybuild ``` The installation prefix is determined by `EESSI-extend` through the following logic: -1. If `EESSI_CVMFS_INSTALL` is set, software inst installed in `EESSI_SOFTWARE_PATH`. This variable shouldn't be used by users and would only be used by CVMFS administrators of the EESSI repository. -2. If `EESSI_SITE_INSTALL` is set (and this is set by default when the EESSI environment is initialized) and the EESSI site installation prefix (`/cvmfs/software.eessi.io/host_injections/...`) is writeable, this prefix will be used. This is typically where sites hosting a system that has EESSI deployed would install additional software on top of EESSI and make it available to all their users. +1. If `EESSI_CVMFS_INSTALL` is set, software is installed in `EESSI_SOFTWARE_PATH`. This variable shouldn't be used by users and would only be used by CVMFS administrators of the EESSI repository. +2. If `EESSI_SITE_INSTALL` is set and the EESSI site installation prefix (`/cvmfs/software.eessi.io/host_injections/...`) is writeable, this prefix will be used. This is typically where sites hosting a system that has EESSI deployed would install additional software on top of EESSI and make it available to all their users. 3. If `EESSI_PROJECT_INSTALL` is set (and `EESSI_USER_INSTALL` is not set), this prefix will be used. You should use this if you want to install additional software on top of EESSI that should also be usable by your project partners on the same system. 4. If `EESSI_USER_INSTALL` is set, this prefix will be used. You should use this if yo uwant to install additional software on top of EESSI just for your own user. If none of the above apply, the default is a user installation in `$HOME/EESSI` (i.e. effectively the same as setting `EESSI_USER_INSTALL=$HOME/EESSI`). From 47343cac331db0b38c53b2a3e8246a39393dacb3 Mon Sep 17 00:00:00 2001 From: Caspar van Leeuwen Date: Tue, 8 Oct 2024 15:34:33 +0200 Subject: [PATCH 6/9] Take https://github.com/EESSI/software-layer/pull/778 into account, as that makes the explaination easier and more explicit --- docs/using_eessi/building_on_eessi.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index d01109c44c..a8cecbd245 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -20,10 +20,10 @@ module help EESSI-extend/2023.06-easybuild ``` The installation prefix is determined by `EESSI-extend` through the following logic: -1. If `EESSI_CVMFS_INSTALL` is set, software is installed in `EESSI_SOFTWARE_PATH`. This variable shouldn't be used by users and would only be used by CVMFS administrators of the EESSI repository. -2. If `EESSI_SITE_INSTALL` is set and the EESSI site installation prefix (`/cvmfs/software.eessi.io/host_injections/...`) is writeable, this prefix will be used. This is typically where sites hosting a system that has EESSI deployed would install additional software on top of EESSI and make it available to all their users. -3. If `EESSI_PROJECT_INSTALL` is set (and `EESSI_USER_INSTALL` is not set), this prefix will be used. You should use this if you want to install additional software on top of EESSI that should also be usable by your project partners on the same system. -4. If `EESSI_USER_INSTALL` is set, this prefix will be used. You should use this if yo uwant to install additional software on top of EESSI just for your own user. +1. If `$EESSI_CVMFS_INSTALL` is set, software is installed in `$EESSI_SOFTWARE_PATH`. This variable shouldn't be used by users and would only be used by CVMFS administrators of the EESSI repository. +2. If `$EESSI_SITE_INSTALL` is set and if the EESSI site installation prefix (`$EESSI_SITE_INSTALLPATH`) is writeable, this prefix will be used. This is typically where sites hosting a system that has EESSI deployed would install additional software on top of EESSI and make it available to all their users. +3. If `$EESSI_PROJECT_INSTALL` is set (and `$EESSI_USER_INSTALL` is not set), this prefix will be used. You should use this if you want to install additional software on top of EESSI that should also be usable by your project partners on the same system. +4. If `$EESSI_USER_INSTALL` is set, this prefix will be used. You should use this if yo uwant to install additional software on top of EESSI just for your own user. If none of the above apply, the default is a user installation in `$HOME/EESSI` (i.e. effectively the same as setting `EESSI_USER_INSTALL=$HOME/EESSI`). Here, we assume you are just an end-user, not having set any of the above environment variables, and loading the `EESSI-extend` module with the default installation prefix: From 7065e8e41af48f09f9bd618792601cc53bcafa90 Mon Sep 17 00:00:00 2001 From: Caspar van Leeuwen Date: Wed, 9 Oct 2024 11:25:19 +0200 Subject: [PATCH 7/9] The module does actually not check if it's writeable, the end-use will just have to make sure --- docs/using_eessi/building_on_eessi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index a8cecbd245..1385467bf5 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -21,7 +21,7 @@ module help EESSI-extend/2023.06-easybuild The installation prefix is determined by `EESSI-extend` through the following logic: 1. If `$EESSI_CVMFS_INSTALL` is set, software is installed in `$EESSI_SOFTWARE_PATH`. This variable shouldn't be used by users and would only be used by CVMFS administrators of the EESSI repository. -2. If `$EESSI_SITE_INSTALL` is set and if the EESSI site installation prefix (`$EESSI_SITE_INSTALLPATH`) is writeable, this prefix will be used. This is typically where sites hosting a system that has EESSI deployed would install additional software on top of EESSI and make it available to all their users. +2. If `$EESSI_SITE_INSTALL` is set, the EESSI site installation prefix (`$EESSI_SITE_INSTALLPATH`) will be used. This is typically where sites hosting a system that has EESSI deployed would install additional software on top of EESSI and make it available to all their users. 3. If `$EESSI_PROJECT_INSTALL` is set (and `$EESSI_USER_INSTALL` is not set), this prefix will be used. You should use this if you want to install additional software on top of EESSI that should also be usable by your project partners on the same system. 4. If `$EESSI_USER_INSTALL` is set, this prefix will be used. You should use this if yo uwant to install additional software on top of EESSI just for your own user. If none of the above apply, the default is a user installation in `$HOME/EESSI` (i.e. effectively the same as setting `EESSI_USER_INSTALL=$HOME/EESSI`). From 46da532e9d4b0d62cc7c5bad624b745d36345d9a Mon Sep 17 00:00:00 2001 From: Caspar van Leeuwen <33718780+casparvl@users.noreply.github.com> Date: Mon, 14 Oct 2024 09:54:29 +0200 Subject: [PATCH 8/9] Apply suggestions from code review Co-authored-by: ocaisa --- docs/using_eessi/building_on_eessi.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 1385467bf5..29d217e05e 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -21,9 +21,9 @@ module help EESSI-extend/2023.06-easybuild The installation prefix is determined by `EESSI-extend` through the following logic: 1. If `$EESSI_CVMFS_INSTALL` is set, software is installed in `$EESSI_SOFTWARE_PATH`. This variable shouldn't be used by users and would only be used by CVMFS administrators of the EESSI repository. -2. If `$EESSI_SITE_INSTALL` is set, the EESSI site installation prefix (`$EESSI_SITE_INSTALLPATH`) will be used. This is typically where sites hosting a system that has EESSI deployed would install additional software on top of EESSI and make it available to all their users. +2. If `$EESSI_SITE_INSTALL` is set, the EESSI site installation prefix (`$EESSI_SITE_SOFTWARE_PATH`) will be used. This is typically where sites hosting a system that has EESSI deployed would install additional software on top of EESSI and make it available to all their users. 3. If `$EESSI_PROJECT_INSTALL` is set (and `$EESSI_USER_INSTALL` is not set), this prefix will be used. You should use this if you want to install additional software on top of EESSI that should also be usable by your project partners on the same system. -4. If `$EESSI_USER_INSTALL` is set, this prefix will be used. You should use this if yo uwant to install additional software on top of EESSI just for your own user. +4. If `$EESSI_USER_INSTALL` is set, this prefix will be used. You should use this if you want to install additional software on top of EESSI just for your own user. If none of the above apply, the default is a user installation in `$HOME/EESSI` (i.e. effectively the same as setting `EESSI_USER_INSTALL=$HOME/EESSI`). Here, we assume you are just an end-user, not having set any of the above environment variables, and loading the `EESSI-extend` module with the default installation prefix: From 59a3ca67fcde9b85df687985f76f1322a4118236 Mon Sep 17 00:00:00 2001 From: Caspar van Leeuwen Date: Mon, 14 Oct 2024 10:57:39 +0200 Subject: [PATCH 9/9] Add example for Project and User based installations --- docs/using_eessi/building_on_eessi.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/using_eessi/building_on_eessi.md b/docs/using_eessi/building_on_eessi.md index 29d217e05e..3e3cd43fd8 100644 --- a/docs/using_eessi/building_on_eessi.md +++ b/docs/using_eessi/building_on_eessi.md @@ -22,8 +22,8 @@ module help EESSI-extend/2023.06-easybuild The installation prefix is determined by `EESSI-extend` through the following logic: 1. If `$EESSI_CVMFS_INSTALL` is set, software is installed in `$EESSI_SOFTWARE_PATH`. This variable shouldn't be used by users and would only be used by CVMFS administrators of the EESSI repository. 2. If `$EESSI_SITE_INSTALL` is set, the EESSI site installation prefix (`$EESSI_SITE_SOFTWARE_PATH`) will be used. This is typically where sites hosting a system that has EESSI deployed would install additional software on top of EESSI and make it available to all their users. -3. If `$EESSI_PROJECT_INSTALL` is set (and `$EESSI_USER_INSTALL` is not set), this prefix will be used. You should use this if you want to install additional software on top of EESSI that should also be usable by your project partners on the same system. -4. If `$EESSI_USER_INSTALL` is set, this prefix will be used. You should use this if you want to install additional software on top of EESSI just for your own user. +3. If `$EESSI_PROJECT_INSTALL` is set (and `$EESSI_USER_INSTALL` is not set), this prefix will be used. You should use this if you want to install additional software on top of EESSI that should also be usable by your project partners on the same system. For example, if you have a project space at `/project/my_project` that all your project partners can access, you could set `export EESSI_PROJECT_INSTALL=/project/my_project/eessi`. Make sure that this directory has the SGID permission set (`chmod g+s $EESSI_PROJECT_INSTALL`). This way, all the additional installations done with `EESSI-extend` will be put in that prefix, and will get the correct UNIX file permissions so that all your project partners can access it. +4. If `$EESSI_USER_INSTALL` is set, this prefix will be used. You should use this if you want to install additional software on top of EESSI just for your own user. For example, you could set `export EESSI_USER_INSTALL=$HOME/my/eessi/extend/prefix`, and `EESSI-extend` will install all software in this prefix. Unix file permissions will be set such that these installations will be readable only to the user. If none of the above apply, the default is a user installation in `$HOME/EESSI` (i.e. effectively the same as setting `EESSI_USER_INSTALL=$HOME/EESSI`). Here, we assume you are just an end-user, not having set any of the above environment variables, and loading the `EESSI-extend` module with the default installation prefix: