Skip to content

Nov 2023 spack merge#371

Merged
climbfuji merged 1678 commits intoJCSDA:jcsda_emc_spack_stackfrom
AlexanderRichert-NOAA:nov2023_spackmerge
Dec 7, 2023
Merged

Nov 2023 spack merge#371
climbfuji merged 1678 commits intoJCSDA:jcsda_emc_spack_stackfrom
AlexanderRichert-NOAA:nov2023_spackmerge

Conversation

@AlexanderRichert-NOAA
Copy link
Copy Markdown
Collaborator

@AlexanderRichert-NOAA AlexanderRichert-NOAA commented Nov 21, 2023

Description

Main spack merge. See JCSDA/spack-stack#884 for description, testing plan, etc.

This also fixes #369.

Checklist

  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • I have run the unit tests before creating the PR

haampie and others added 30 commits October 25, 2023 17:35
* ci: don't register detectable compilers

Cause they go out of sync...

* remove intel compiler, it can be detected too

* Do not run spack compiler find since compilers are registered in concretize job already

* trilinos: work around +stokhos +cuda +superlu-dist bug due to EMPTY macro
* [cuda] Add NVHPC_CUDA_HOME.

* Add CUDA_HOME and NVHC_CUDA_HOME to cuda's dependent build env.

---------

Co-authored-by: Dominic Hofer <dominic.hofer@meteoswiss.ch>
* paraview: rebase the adios2 patch for 5.12-to-be

* paraview: disable fastfloat and token for 5.12-to-be

* paraview: require older protobuf for 5.12 as well

* paraview: require C++11-supporting protobuf for `master` too
Renames exclude_implicits to hide_implicits

When hide_implicits option is enabled, generate modulefile of
implicitly installed software and hide them. Even if implicit, those
modulefiles may be referred as dependency in other modulefiles thus they
should be generated to make module properly load dependent module.

A new hidden property is added to BaseConfiguration class.

To hide modulefiles, modulercs are generated along modulefiles. Such rc
files contain specific module command to indicate a module should be
hidden (for instance when using "module avail").

A modulerc property is added to TclFileLayout and LmodFileLayout classes
to get fully qualified path name of the modulerc associated to a given
modulefile.

Modulerc files will be located in each module directory, next to the
version modulefiles. This scheme is supported by both module tool
implementations.

modulerc_header and hide_cmd_format attributes are added to
TclModulefileWriter and LmodModulefileWriter. They help to know how to
generate a modulerc file with hidden commands for each module tool.

Tcl modulerc file requires an header. As we use a command introduced on
Modules 4.7 (module-hide --hidden-loaded), a version requirement is
added to header string.

For lmod, modules that open up a hierarchy are never hidden, even if
they are implicitly installed.

Modulerc is created, updated or removed when associated modulefile is
written or removed. If an implicit modulefile becomes explicit, hidden
command in modulerc for this modulefile is removed. If modulerc becomes
empty, this file is removed. Modulerc file is not rewritten when no
content change is detected.

Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
* PythonPackage: nested config_settings

* flake8
Update Tcl modulefile template to simplify generated `append-path`,
`prepend-path` and `remove-path` commands and improve their readability.

If path element delimiter is colon character, do not set the `--delim`
option as it is the default delimiter value.
* Add conflict with C++ standard > 17 and cuda < 12

* Removing map_cxxstd since boost supports C++20 flag
* spack checksum pkg@1.2, use as version filter

Currently pkg@1.2 splits on @ and looks for 1.2 specifically, with this
PR pkg@1.2 is a filter so any matching 1.2, 1.2.1, ..., 1.2.10 version
is displayed.

* fix tests

* fix style
* Update spack package for exago:1.6.0

* update style

* Weird spack style env bug fixed

* Update spack package for exago:1.6.0

* update style

* Weird spack style env bug fixed

* changes to allow release 1.6.0

* fix depends, and versioning

* rm cmake variable

* add s

* style fix

---------

Co-authored-by: Ryan Danehy <dane678@deception04.pnl.gov>
Co-authored-by: Ryan Danehy <dane678@deception03.pnl.gov>
Co-authored-by: ryan.danehy@pnnl.gov <dane678@we45149.home>
* rtmpdump: New package

* curl: Fix librtmp variant

Add the previously missing dependency required for rtmp support.

The variant has been broken since its addition in PR spack#25166.

Fixes one of the two issues reported in spack#26887.
* Update DataTransferKit for 3.1.1 release

* Require Trilinos-14 for 3.1.0 and higher
* ldak: update to 5.2, add maintainer

* ldak: use compiler.openmp_flag
* itk: patch missing include for newer compilers

* itk: The package doesn't use MPI

* itk: package requires the high-level hdf5 api

* itk: patch url with ?full_index=1

* itk: point to 4041 commit in master

* itk: don't constrain hdf5 with ~mpi
Co-authored-by: Nicolas Richart <nrichart@users.noreply.github.com>
* Fix cmake_args for spglib v2.1.0+

* Add option to build fortran interface in package spglib

* fix style as sugested by ci/prechecks/style

* Enable fortran variant from v1.16.4 as suggested

Co-authored-by: Rocco Meli <r.meli@bluemail.ch>

---------

Co-authored-by: Rocco Meli <r.meli@bluemail.ch>
otherwise it detected pre-configured compilers in an potentially different way.
* gromacs: default to external blas & lapack

* drop vendored lapack/blas altogether
@AlexanderRichert-NOAA
Copy link
Copy Markdown
Collaborator Author

@climbfuji for what it's worth, I fixed a macOS CI failure with hdf-eos2 by adding a CFLAGS setting with "-Wno-error=implicit-function-declaration". Basically, hdf-eos2 uses a compiler wrapper from hdf, and the hdf recipe changed so that that flag is no longer set as a configure argument, so it doesn't end up in libhdf4.settings and therefore doesn't get propagated to hdf-eos2. I'm not sure that the setting in the hdf-eos2 flag handler is doing anything (since the flag handler is basically just setting $SPACK_CFLAGS or something like that for the spack compiler wrappers), so if you can confirm that it's not needed then one of us can remove it.

@climbfuji
Copy link
Copy Markdown
Collaborator

@climbfuji for what it's worth, I fixed a macOS CI failure with hdf-eos2 by adding a CFLAGS setting with "-Wno-error=implicit-function-declaration". Basically, hdf-eos2 uses a compiler wrapper from hdf, and the hdf recipe changed so that that flag is no longer set as a configure argument, so it doesn't end up in libhdf4.settings and therefore doesn't get propagated to hdf-eos2. I'm not sure that the setting in the hdf-eos2 flag handler is doing anything (since the flag handler is basically just setting $SPACK_CFLAGS or something like that for the spack compiler wrappers), so if you can confirm that it's not needed then one of us can remove it.

What happens if you remove it? Does it still work?

@AlexanderRichert-NOAA
Copy link
Copy Markdown
Collaborator Author

Yep, works without it. I don't think it was having any effect on account of using the compiler wrappers from hdf.

@climbfuji
Copy link
Copy Markdown
Collaborator

climbfuji commented Nov 30, 2023 via email

@AlexanderRichert-NOAA AlexanderRichert-NOAA marked this pull request as ready for review December 1, 2023 03:33
@symoore90 symoore90 added the INFRA JEDI Infrastructure label Dec 4, 2023
Copy link
Copy Markdown
Collaborator

@climbfuji climbfuji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we are good to go. We have two systems that are still installing (Discover and ParallelWorks EPIC), but everything else worked just fine, and if any changes are needed those will most likely be in the site configs. Ready to rock and roll!

@AlexanderRichert-NOAA
Copy link
Copy Markdown
Collaborator Author

Is the macOS unit test failure a fluke? I thought everything was fine apart from the code coverage 😕

@climbfuji
Copy link
Copy Markdown
Collaborator

Is the macOS unit test failure a fluke? I thought everything was fine apart from the code coverage 😕

I think it started mid way, something changed again in the macOS Github runner? May have to pull in yet another update from spack develop. I think we should ignore that for this PR and fix it afterwards, given the size of the PR and the amount of testing that went into it.

@AlexanderRichert-NOAA
Copy link
Copy Markdown
Collaborator Author

Yeah, that sounds good

@climbfuji
Copy link
Copy Markdown
Collaborator

Discover just finished, too.

Copy link
Copy Markdown
Collaborator

@RatkoVasic-NOAA RatkoVasic-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on Hera, Jet and Gaea-c5. Approved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

INFRA JEDI Infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI style tests broken