Skip to content

determine easystack files to process via PR patch file #351

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
trz42 merged 2 commits into from
Oct 4, 2023
Merged

determine easystack files to process via PR patch file #351

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
776e3cd
determine easystack files to process via PR patch file
boegel Oct 3, 2023
4657384
use 'git diff' to determine which easystack files were changed
boegel Oct 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 25 additions & 22 deletions EESSI-pilot-install-software.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ function copy_build_log() {
echo "- working directory: ${PWD}" >> ${build_log_path}
echo "- Slurm job ID: ${SLURM_OUT}" >> ${build_log_path}
echo "- EasyBuild version: ${eb_version}" >> ${build_log_path}
echo "- easystack file: ${es}" >> ${build_log_path}
echo "- easystack file: ${easystack_file}" >> ${build_log_path}

echo "EasyBuild log file ${build_log} copied to ${build_log_path} (with context appended)"
fi
Expand Down Expand Up @@ -186,36 +186,39 @@ else
echo_green ">> MODULEPATH set up: ${MODULEPATH}"
fi

for eb_version in '4.7.2' '4.8.0' '4.8.1'; do
# use 'git diff' to determine which easystack files were changed
for easystack_file in $(git diff --name-only | grep '^eessi.*yml$'); do

echo -e "Processing easystack file ${easystack_file}...\n\n"

# determine version of EasyBuild module to load based on EasyBuild version included in name of easystack file
eb_version=$(echo ${easystack_file} | sed 's/.*eb-\([0-9.]*\).*/\1/g')
Copy link
Copy Markdown
Collaborator

@trz42 trz42 Oct 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternative way might be

eb_version=$(echo ${easystack_file} | cut -f 4 -d '-')

Copy link
Copy Markdown
Contributor Author

@boegel boegel Oct 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a bit less strict, at least with eb- we have some guarantee that which follows is an EasyBuild version...

So I prefer sticking to the sed.

Eventually we probably want to specify the EasyBuild version to use in the easystack file itself (but then EasyBuild needs to allow for that, I think).

Copy link
Copy Markdown
Collaborator

@trz42 trz42 Oct 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack.


# load EasyBuild module (will be installed if it's not available yet)
source ${TOPDIR}/load_easybuild_module.sh ${eb_version}

echo_green "All set, let's start installing some software with EasyBuild v${eb_version} in ${EASYBUILD_INSTALLPATH}..."

for es in $(ls eessi-${EESSI_PILOT_VERSION}-eb-${eb_version}-*.yml); do

if [ -f ${es} ]; then
echo_green "Feeding easystack file ${es} to EasyBuild..."

${EB} --easystack ${TOPDIR}/${es} --robot
ec=$?
if [ -f ${easystack_file} ]; then
echo_green "Feeding easystack file ${easystack_file} to EasyBuild..."

# copy EasyBuild log file if EasyBuild exited with an error
if [ ${ec} -ne 0 ]; then
eb_last_log=$(unset EB_VERBOSE; eb --last-log)
# copy to current working directory
cp -a ${eb_last_log} .
echo "Last EasyBuild log file copied from ${eb_last_log} to ${PWD}"
# copy to build logs dir (with context added)
copy_build_log "${eb_last_log}" "${build_logs_dir}"
fi
${EB} --easystack ${TOPDIR}/${easystack_file} --robot
ec=$?

$TOPDIR/check_missing_installations.sh ${TOPDIR}/${es}
else
fatal_error "Easystack file ${es} not found!"
# copy EasyBuild log file if EasyBuild exited with an error
if [ ${ec} -ne 0 ]; then
eb_last_log=$(unset EB_VERBOSE; eb --last-log)
# copy to current working directory
cp -a ${eb_last_log} .
echo "Last EasyBuild log file copied from ${eb_last_log} to ${PWD}"
# copy to build logs dir (with context added)
copy_build_log "${eb_last_log}" "${build_logs_dir}"
fi
done

$TOPDIR/check_missing_installations.sh ${TOPDIR}/${easystack_file}
else
fatal_error "Easystack file ${easystack_file} not found!"
fi

done

Expand Down