NorESSI · trz42 · Mar 7, 2023 · Mar 7, 2023 · Mar 7, 2023 · Mar 7, 2023
diff --git a/.github/workflows/build-and-publish-bootstrap-container.yml b/.github/workflows/build-and-publish-bootstrap-container.yml
@@ -4,14 +4,14 @@ on:
     branches:
       - main
     paths:
-      - Dockerfile.bootstrap-prefix-centos8
+      - Dockerfile.bootstrap-prefix-debian11
       - bootstrap-prefix.sh
 
   pull_request:
     branches:
       - main
     paths:
-      - Dockerfile.bootstrap-prefix-centos8
+      - Dockerfile.bootstrap-prefix-debian11
       - bootstrap-prefix.sh
 
 # Declare default permissions as read only.
@@ -48,7 +48,7 @@ jobs:
       - name: Build and push to GitHub Packages
         uses: docker/build-push-action@c56af957549030174b10d6867f20e78cfd7debc5 #v3.2.0
         with:
-          tags: ghcr.io/${{ env.REPOSITORY_OWNER }}/bootstrap-prefix:centos8
-          file: Dockerfile.bootstrap-prefix-centos8
+          tags: ghcr.io/${{ env.REPOSITORY_OWNER }}/bootstrap-prefix:debian11
+          file: Dockerfile.bootstrap-prefix-debian11
           platforms: linux/amd64, linux/arm64, linux/ppc64le
           push: ${{ github.event_name != 'pull_request' }}
diff --git a/Dockerfile.bootstrap-prefix-centos8 b/Dockerfile.bootstrap-prefix-centos8
diff --git a/Dockerfile.bootstrap-prefix-debian11 b/Dockerfile.bootstrap-prefix-debian11
@@ -0,0 +1,17 @@
+FROM debian:11-slim
+
+COPY bootstrap-prefix.sh /usr/local/bin/bootstrap-prefix.sh
+
+RUN apt-get update
+RUN apt-get install -y gcc g++ make diffutils libgmp-dev perl wget
+RUN apt-get install -y git python3-pip python3-cryptography
+RUN pip3 install --upgrade pip
+RUN pip3 install ansible
+RUN chmod 755 /usr/local/bin/bootstrap-prefix.sh
+
+ENV LC_ALL=C.UTF-8
+ENV PATH=/usr/local/bin:$PATH
+
+RUN groupadd -g 1000 portage && useradd -ms /bin/bash -u 1000 -g portage portage
+
+ENTRYPOINT ["/usr/local/bin/bootstrap-prefix.sh"]
diff --git a/README.md b/README.md
@@ -8,124 +8,15 @@ An alternative would be the [NixOS](https://nixos.org/).
 
 ## Installation and Configuration
 
-### Prerequisites
-
-The bootstrap process will need a clean environment with C and C++ compilers (the system version of gcc and g++ will do) as well as the `make` command. It also is very sensitive to 
-the environment, so setup a user with unset `CFFLAGS`, `CFLAGS`, `LDFLAGS`, `PKG_CONFIG_PATH` and the always harmful `LD_LIBRARY_PATH` variables.
-
-EESSI provides a Docker container for this, which can be run via Singularity.
-
-### Bootstrapping Gentoo Prefix
-Gentoo Prefix provides a bootstrap script to build the prefix, see [Gentoo Prefix Bootstrap](https://wiki.gentoo.org/wiki/Project:Prefix/Bootstrap).
-We forked [this version](https://gitweb.gentoo.org/repo/proj/prefix.git/tree/scripts/bootstrap-prefix.sh?id=e77fd01734f21ec2e9c985c28ba4eb30c1b2bc9d)
-and made some modifications. See issue [#8](https://github.com/EESSI/compatibility-layer/issues/8) for more details. 
-
-You can run our version of the bootstrap script (see `bootstrap-prefix.sh`) inside the Singularity container by executing:
-```
-singularity run docker://ghcr.io/eessi/bootstrap-prefix:centos8
-```
-
-If you want to run your own version of the bootstrap script, use:
-```
-singularity exec docker://ghcr.io/eessi/bootstrap-prefix:centos8 ./bootstrap-prefix.sh
-```
-Our version of the script allows you to pick a custom snapshot for the Portage tree. This can be done by setting `CUSTOM_SNAPSHOT_URL` to
-a URL that points to a directory, and setting `CUSTOM_SNAPSHOT_URL` to the name of a snapshot file (must be a bzip2 archive). For instance:
-```
-env CUSTOM_SNAPSHOT_URL="http://cvmfs-s0.eessi-hpc.org/snapshots" CUSTOM_SNAPSHOT_VERSION="20201209" singularity run docker://ghcr.io/eessi/bootstrap-prefix:centos8
-```
-If you want to limit the supported/installed Python version(s), you can set the environment variable `PYTHON_TARGETS` before starting the bootstrap script. By only including a Python 3 version, you can prevent Python 2 from being installed, e.g.:
-```
-env PYTHON_TARGETS="python3_8" CUSTOM_SNAPSHOT_URL="http://cvmfs-s0.eessi-hpc.org/snapshots" CUSTOM_SNAPSHOT_VERSION="20201126" singularity run docker://ghcr.io/eessi/bootstrap-prefix:centos8
-```
-
-After starting the bootstrap have a long coffee...
-
-Once the bootstrap is completed, run the script to replace some paths with symlinks into the host OS:
-
-```
-scripts/prefix-symlink-host-paths.sh
-```
-
-In order to generate the right locales for your installation, see [this item](https://wiki.gentoo.org/wiki/Project:Prefix/FAQ#Add_an_en_US.UTF-8_locale)
-of the Gentoo Prefix FAQ.
-
-### Adding the EESSI overlay and packages
-Additional packages are added in the EESSI overlay, which is based on ComputeCanada.
-You can add them manually or in an automated way by using Ansible, being Ansible the preferred way. Below you can find the two options explained.
-
-#### Ansible playbook (Option 1)
-The installation of the EESSI-specific parts can be automatically executed by running the Ansible playbook `install.yml` inside the folder `ansible/playbooks`. 
-This playbook will install the [EESSI Gentoo overlay](https://github.com/EESSI/gentoo-overlay) and a set of packages, including `Lmod` and `archspec`. See the `README` in the `ansible/playbooks` folder for more details.
-
-#### Manually (Option 2)
-First, set `EPREFIX` to the path containing your Gentoo Prefix installation, and start the prefix:
-```
-export EPREFIX=/path/to/your/prefix
-${EPREFIX}/startprefix
-```
-Ensure that the configuration directory for repositories exists:
-```
-mkdir -p ${EPREFIX}/etc/portage/repos.conf
-```
-If you used `${PYTHON_TARGETS}` during the bootstrap, be sure to set it to the same value now, e.g.:
-```
-export PYTHON_TARGETS="python3_7"
-```
-
-Next, configure and sync the overlay:
-```
-emerge eselect-repository
-eselect repository add eessi git https://github.com/EESSI/gentoo-overlay.git
-emerge --sync eessi
-```
-
-After synchronizing the overlay, add the EESSI package set(s) that you would like to install, e.g. for set `2020.08`:
-```
-mkdir ${EPREFIX}/etc/portage/sets/
-ln -s  ${EPREFIX}/var/db/repos/eessi/etc/portage/sets/2020.08 ${EPREFIX}/etc/portage/sets/
-```
-
-Finally, install the package set(s) defined at `${EPREFIX}/etc/portage/sets/`, e.g.:
-```
-emerge @2020.08
-```
-
-### Updating the Prefix
-#### Packages
-Updating packages can be as easy as
-```
-emerge --sync
-emerge
-```
-If you run into problems, usually a newer ebuild is not suited to build in a prefix environment.
-Try to mask latest versions:
-
-Create a mask file if not existing and mask newer versions from thin provisioning tools greater or equal to 0.7.6:
-```
-echo ">=sys-block/thin-provisioning-tools-0.7.6" >> ${EPREFIX}/etc/portage/package.mask
-```
-
-#### Portage
-Updating Portage requires the kernel source which corresponds to your running kernel on the host. Emerge will detect it in `/usr/src/linux`.
-
-Check your running kernel version with:
-```
-cat /proc/version
-Linux version 4.20.0-1.el7.elrepo.x86_64 (mockbuild@Build64R7) 
-```
-
-On a Centos 7 host kernel sources are installed in `/usr/src/kernels`. Link `/usr/src/linux` to the appropiate kernel source after installation. Example for an `elrepo` kernel:
-```
-rpm -ivh kernel-ml-devel-4.20.0-1.el7.elrepo.x86_64.rpm
-cd /usr/src ; ln -s kernels/4.20.0-1.el7.elrepo.x86_64 linux
-```
-
- When ready update Portage from the Prefix environment:
-```
-startprefix
-emerge --oneshot sys-apps/portage
-```
+The installation of the compatibility layer is implemented with an Ansible playbook and Ansible role,
+which you can find in the `ansible` subdirectory. It will do the Gentoo Prefix bootstrap (which usually takes several hours to complete!),
+add a lot of EESSI configurations and customizations, install a bunch of packages that we require for EESSI, and finally it will
+run a ReFrame test suite to check the installation.
+
+To make the installation even easier, we provide a script `install_compatibility_layer.sh` that can be used on basically any host that has Apptainer installed,
+without requiring special privileges.
+The script will execute the Ansible playbook inside an Apptainer build container, ensuring that all dependencies (including Ansible itself) are available.
+In order to be able to write to `/cvmfs`, the container will bind mount a directory from the host as `/cvmfs` inside the container.
 
 # License
 

diff --git a/ansible/playbooks/README.md b/ansible/playbooks/README.md
@@ -11,10 +11,10 @@ all functionality for installing the EESSI compatibility layer. It performs the
  - add a given overlay to the installation;
  - use the Portage configuration files from that overlay, if applicable, by making symlinks to them;
  - install a given list of package sets;
- - install a given list of additional packages.
+ - install a given list of additional packages;
+ - test the installation using ReFrame.
 
 The playbook `install.yml` will execute this role on a given server. 
-Note that if you want the role to install Gentoo Prefix, this particular task currently only supports Linux distributions based on RHEL 8 on the installation host.
 
 ## Configuration
 
@@ -29,9 +29,6 @@ Before running the playbook, make sure the following settings are correct, and o
 ### CVMFS settings
 | Variable | Description |
 | --- | --- |
-| cvmfs_start_transaction | Whether a CVMFS transaction should be start at the start |
-| cvmfs_publish_transaction | Whether a CVMFS transaction should be published at the end |
-| cvmfs_abort_transaction_on_failures | Whether a CVMFS transaction should be aborted on failures |
 | cvmfs_repository | Name of your CVMFS repository (used for the transaction) |
 
 ### Prefix and packages
@@ -45,16 +42,17 @@ Before running the playbook, make sure the following settings are correct, and o
 | prefix_default_gcc | GCC compiler version to use as default compiler in Gentoo Prefix installation |
 | prefix_user_defined_trusted_dirs | List of paths to the user defined trusted dirs for glibc |
 | prefix_mask_packages | Contents of a [package.mask file](https://wiki.gentoo.org/wiki//etc/portage/package.mask) that should be used during the bootstrap |
+| prefix_unmask_packages | Contents of a [package.unmask file](https://wiki.gentoo.org/wiki//etc/portage/package.unmask) that should be used during the bootstrap |
 | prefix_bootstrap_use_flags | Contents of [package.use file](https://wiki.gentoo.org/wiki//etc/portage/package.use) to put in place after bootstrap stage 3 |
 | prefix_use_builtin_bootstrap | Use the container's built-in bootstrap script? |
 | prefix_custom_bootstrap_script | Dictionary with the `local` source and `remote` destination of the bootstrap script |
-| prefix_singularity_command | Singularity command for launching the container with the bootstrap script |
-| prefix_source | Singularity container path used for the Prefix installtion |
 | prefix_source_options | Arguments to be passed to the Prefix bootstrap script |
 | prefix_install | Prefix installation command |
 | prefix_locales | List of locales to be generated |
 | package_sets | List of package sets to be installed |
 | prefix_packages | List of additional packages to be installed |
+| prefix_remove_packages | List of packages to be removed after the bootstrap |
+| reframe_venv | Path where a virtual environment will be created for the ReFrame installation |
 | symlinks_to_host | List of paths that should get a symlink to the corresponding host path |
 
 ### Logging

diff --git a/ansible/playbooks/install.yml b/ansible/playbooks/install.yml
@@ -1,7 +1,7 @@
 # Playbook for installing the EESSI compatibility layer.
 ---
 - name: Install compatibility layer
-  hosts: cvmfsstratum0servers
+  hosts: localhost
   environment:
     PATH:
       "{{ gentoo_prefix_path }}/usr/sbin\
@@ -14,4 +14,3 @@
     LC_ALL: C
   roles:
     - compatibility_layer
-  become: true
diff --git a/ansible/playbooks/roles/compatibility_layer/defaults/main.yml b/ansible/playbooks/roles/compatibility_layer/defaults/main.yml
@@ -1,16 +1,13 @@
 # Defaults file for the compatibility layer role.
 ---
-eessi_version: "2021.12"
+eessi_version: "2023.03"
 
 custom_overlays:
   - name: eessi
     source: git
     url: https://github.com/EESSI/gentoo-overlay.git
     eclass-overrides: true
 
-cvmfs_start_transaction: false
-cvmfs_publish_transaction: false
-cvmfs_abort_transaction_on_failures: false
 cvmfs_repository: pilot.eessi-hpc.org
 
 gentoo_prefix_path: /cvmfs/{{ cvmfs_repository }}/versions/{{ eessi_version }}/compat/{{ eessi_host_os }}/{{ eessi_host_arch }}
@@ -19,34 +16,27 @@ gentoo_prefix_path: /cvmfs/{{ cvmfs_repository }}/versions/{{ eessi_version }}/c
 gentoo_git_repo: https://github.com/gentoo/gentoo.git
 # Select a specific commit in the gentoo_git_repo that should be used for the bootstrap,
 # e.g. by checking: https://github.com/gentoo/gentoo/commits/master
-gentoo_git_commit: 7eaa2512d1e6ddb44e3b41bbddf6c74723f234ce
+gentoo_git_commit: 4ca74e7abe4f2b14e686267b517c59d43bb580b4
 prefix_required_space: 15 GB
-prefix_default_gcc: 9.4.0
+prefix_default_gcc: 9.5.0
 prefix_user_defined_trusted_dirs:
   - "/cvmfs/{{ cvmfs_repository }}/host_injections/{{ eessi_version }}/compat/{{ eessi_host_os }}/{{ eessi_host_arch }}/lib"
 prefix_mask_packages: |
-  # avoid glibc 2.34, as it's causing issues with the bootstrap, and it's not compatible with CUDA 11.
-  # see https://github.com/EESSI/compatibility-layer/issues/137 + https://bugs.gentoo.org/824482
-  >=sys-libs/glibc-2.34
   # stick to GCC 9.x; using a too recent compiler in the compat layer complicates stuff in the software layer,
   # see for example https://github.com/EESSI/software-layer/issues/151
   >=sys-devel/gcc-10
-  # avoid libgcrypt 1.9.4 due to compiler errros on ppc64le,
-  # see https://github.com/EESSI/compatibility-layer/issues/134 + https://bugs.gentoo.org/825722
-  =dev-libs/libgcrypt-1.9.4
+prefix_unmask_packages: |
+  # unmask older GCC to make it installable
+  =sys-devel/gcc-9*
 prefix_bootstrap_use_flags: |
   # make sure that gold linker is installed with binutils
   sys-devel/binutils gold
 prefix_use_builtin_bootstrap: false
 prefix_custom_bootstrap_script:
   local: "{{ playbook_dir }}/../../bootstrap-prefix.sh"
   remote: /tmp/bootstrap-prefix.sh
-prefix_singularity_command: "singularity exec -B {{ gentoo_prefix_path }}:{{ gentoo_prefix_path }}"
-prefix_source: "docker://ghcr.io/eessi/bootstrap-prefix:centos8"
 prefix_source_options: "{{ gentoo_prefix_path }} noninteractive"
 prefix_install: >-
-    SINGULARITYENV_USE_CPU_CORES={{ ansible_processor_vcpus }}
-    {{ prefix_singularity_command }} {{ prefix_source }}
     {{ prefix_use_builtin_bootstrap | ternary('/usr/local/bin/bootstrap-prefix.sh', prefix_custom_bootstrap_script.remote) }}
     {{ prefix_source_options }}
 
@@ -64,6 +54,12 @@ package_sets:
 
 prefix_packages:
 
+prefix_remove_packages:
+  - dev-lang/go
+  - dev-lang/go-bootstrap
+
+reframe_venv_dir: /tmp/reframe_venv
+
 # List of locations that should get a symlink $EPREFIX/$LOCATION -> $LOCATION.
 # This ensures that things like user/group ids are correct/looked up in the right way in the Prefix environment.
 symlinks_to_host:

diff --git a/ansible/playbooks/roles/compatibility_layer/tasks/add_overlay.yml b/ansible/playbooks/roles/compatibility_layer/tasks/add_overlay.yml
@@ -1,26 +1,13 @@
 # Add a custom overlay to the Gentoo Prefix installation.
 ---
-- name: Add portage user if missing
-  become: true
-  ansible.builtin.lineinfile:
-    path: /etc/passwd
-    regexp: portage
-    line: portage:x:250:250:portage:/var/tmp/portage:/bin/false
-
-- name: Add portage group if missing
-  become: true
-  ansible.builtin.lineinfile:
-    path: /etc/group
-    regexp: portage
-    line: portage::250:portage
-
 - name: Install equery command (dependency for the portage module)
   ansible.builtin.command:
     cmd: emerge gentoolkit
     creates: "{{ gentoo_prefix_path }}/usr/bin/equery"
 
 - name: Install eselect-repository
   community.general.portage:
+    verbose: true
     package: eselect-repository
     state: present
 

diff --git a/ansible/playbooks/roles/compatibility_layer/tasks/cleanup.yml b/ansible/playbooks/roles/compatibility_layer/tasks/cleanup.yml
@@ -0,0 +1,7 @@
+# Clean up
+---
+- name: Remove redundant packages
+  community.general.portage:
+    package: "{{ item }}"
+    state: absent
+  with_items: "{{ prefix_remove_packages }}"
diff --git a/ansible/playbooks/roles/compatibility_layer/tasks/install_packages.yml b/ansible/playbooks/roles/compatibility_layer/tasks/install_packages.yml
@@ -5,7 +5,6 @@
     package: "@{{ item }}"
     state: present
   with_items: "{{ package_sets }}"
-  become: false
   tags:
     - set
 
@@ -14,18 +13,3 @@
     package: "{{ item }}"
     state: present
   with_items: "{{ prefix_packages }}"
-  become: false
-
-- name: "Get the username running the deployment (not root)"
-  ansible.builtin.command: whoami
-  changed_when: false
-  become: false
-  register: username_on_host
-
-- name: "Fix permissions after installing as portage/root"
-  ansible.builtin.file:
-    owner: "{{ username_on_host.stdout }}"
-    group: "{{ username_on_host.stdout }}"
-    path: "{{ gentoo_prefix_path }}"
-    recurse: true
-  become: false