Nov 2023 spack merge #371

AlexanderRichert-NOAA · 2023-11-21T21:59:45Z

Description

Main spack merge. See JCSDA/spack-stack#884 for description, testing plan, etc.

This also fixes #369.

Checklist

I have performed a self-review of my own code
~~I have made corresponding changes to the documentation~~
I have run the unit tests before creating the PR

* ci: don't register detectable compilers Cause they go out of sync... * remove intel compiler, it can be detected too * Do not run spack compiler find since compilers are registered in concretize job already * trilinos: work around +stokhos +cuda +superlu-dist bug due to EMPTY macro

…k#40697)

* [cuda] Add NVHPC_CUDA_HOME. * Add CUDA_HOME and NVHC_CUDA_HOME to cuda's dependent build env. --------- Co-authored-by: Dominic Hofer <[email protected]>

* paraview: rebase the adios2 patch for 5.12-to-be * paraview: disable fastfloat and token for 5.12-to-be * paraview: require older protobuf for 5.12 as well * paraview: require C++11-supporting protobuf for `master` too

Renames exclude_implicits to hide_implicits When hide_implicits option is enabled, generate modulefile of implicitly installed software and hide them. Even if implicit, those modulefiles may be referred as dependency in other modulefiles thus they should be generated to make module properly load dependent module. A new hidden property is added to BaseConfiguration class. To hide modulefiles, modulercs are generated along modulefiles. Such rc files contain specific module command to indicate a module should be hidden (for instance when using "module avail"). A modulerc property is added to TclFileLayout and LmodFileLayout classes to get fully qualified path name of the modulerc associated to a given modulefile. Modulerc files will be located in each module directory, next to the version modulefiles. This scheme is supported by both module tool implementations. modulerc_header and hide_cmd_format attributes are added to TclModulefileWriter and LmodModulefileWriter. They help to know how to generate a modulerc file with hidden commands for each module tool. Tcl modulerc file requires an header. As we use a command introduced on Modules 4.7 (module-hide --hidden-loaded), a version requirement is added to header string. For lmod, modules that open up a hierarchy are never hidden, even if they are implicitly installed. Modulerc is created, updated or removed when associated modulefile is written or removed. If an implicit modulefile becomes explicit, hidden command in modulerc for this modulefile is removed. If modulerc becomes empty, this file is removed. Modulerc file is not rewritten when no content change is detected. Co-authored-by: Harmen Stoppels <[email protected]>

* PythonPackage: nested config_settings * flake8

Update Tcl modulefile template to simplify generated `append-path`, `prepend-path` and `remove-path` commands and improve their readability. If path element delimiter is colon character, do not set the `--delim` option as it is the default delimiter value.

* Add conflict with C++ standard > 17 and cuda < 12 * Removing map_cxxstd since boost supports C++20 flag

* spack checksum [email protected], use as version filter Currently [email protected] splits on @ and looks for 1.2 specifically, with this PR [email protected] is a filter so any matching 1.2, 1.2.1, ..., 1.2.10 version is displayed. * fix tests * fix style

* Update spack package for exago:1.6.0 * update style * Weird spack style env bug fixed * Update spack package for exago:1.6.0 * update style * Weird spack style env bug fixed * changes to allow release 1.6.0 * fix depends, and versioning * rm cmake variable * add s * style fix --------- Co-authored-by: Ryan Danehy <[email protected]> Co-authored-by: Ryan Danehy <[email protected]> Co-authored-by: [email protected] <[email protected]>

* rtmpdump: New package * curl: Fix librtmp variant Add the previously missing dependency required for rtmp support. The variant has been broken since its addition in PR spack#25166. Fixes one of the two issues reported in spack#26887.

* Update DataTransferKit for 3.1.1 release * Require Trilinos-14 for 3.1.0 and higher

* ldak: update to 5.2, add maintainer * ldak: use compiler.openmp_flag

* itk: patch missing include for newer compilers * itk: The package doesn't use MPI * itk: package requires the high-level hdf5 api * itk: patch url with ?full_index=1 * itk: point to 4041 commit in master * itk: don't constrain hdf5 with ~mpi

Co-authored-by: Nicolas Richart <[email protected]>

* Fix cmake_args for spglib v2.1.0+ * Add option to build fortran interface in package spglib * fix style as sugested by ci/prechecks/style * Enable fortran variant from v1.16.4 as suggested Co-authored-by: Rocco Meli <[email protected]> --------- Co-authored-by: Rocco Meli <[email protected]>

otherwise it detected pre-configured compilers in an potentially different way.

* gromacs: default to external blas & lapack * drop vendored lapack/blas altogether

AlexanderRichert-NOAA · 2023-11-30T06:21:48Z

@climbfuji for what it's worth, I fixed a macOS CI failure with hdf-eos2 by adding a CFLAGS setting with "-Wno-error=implicit-function-declaration". Basically, hdf-eos2 uses a compiler wrapper from hdf, and the hdf recipe changed so that that flag is no longer set as a configure argument, so it doesn't end up in libhdf4.settings and therefore doesn't get propagated to hdf-eos2. I'm not sure that the setting in the hdf-eos2 flag handler is doing anything (since the flag handler is basically just setting $SPACK_CFLAGS or something like that for the spack compiler wrappers), so if you can confirm that it's not needed then one of us can remove it.

climbfuji · 2023-11-30T15:30:22Z

@climbfuji for what it's worth, I fixed a macOS CI failure with hdf-eos2 by adding a CFLAGS setting with "-Wno-error=implicit-function-declaration". Basically, hdf-eos2 uses a compiler wrapper from hdf, and the hdf recipe changed so that that flag is no longer set as a configure argument, so it doesn't end up in libhdf4.settings and therefore doesn't get propagated to hdf-eos2. I'm not sure that the setting in the hdf-eos2 flag handler is doing anything (since the flag handler is basically just setting $SPACK_CFLAGS or something like that for the spack compiler wrappers), so if you can confirm that it's not needed then one of us can remove it.

What happens if you remove it? Does it still work?

AlexanderRichert-NOAA · 2023-11-30T20:00:51Z

Yep, works without it. I don't think it was having any effect on account of using the compiler wrappers from hdf.

climbfuji · 2023-11-30T20:01:43Z

Great, thanks for testing that!

…

On Nov 30, 2023, at 1:01 PM, Alex Richert ***@***.***> wrote: Yep, works without it. I don't think it was having any effect on account of using the compiler wrappers from hdf. — Reply to this email directly, view it on GitHub <#371 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB5C2RJ3KNG5STJZ3GEYESLYHDQX5AVCNFSM6AAAAAA7VIOKYSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZUGQ3DANRQHE>. You are receiving this because you were mentioned.

…into HEAD

…ert-NOAA/spack into nov2023_spackmerge

climbfuji

I think we are good to go. We have two systems that are still installing (Discover and ParallelWorks EPIC), but everything else worked just fine, and if any changes are needed those will most likely be in the site configs. Ready to rock and roll!

AlexanderRichert-NOAA · 2023-12-07T02:48:21Z

Is the macOS unit test failure a fluke? I thought everything was fine apart from the code coverage 😕

climbfuji · 2023-12-07T02:49:54Z

Is the macOS unit test failure a fluke? I thought everything was fine apart from the code coverage 😕

I think it started mid way, something changed again in the macOS Github runner? May have to pull in yet another update from spack develop. I think we should ignore that for this PR and fix it afterwards, given the size of the PR and the amount of testing that went into it.

AlexanderRichert-NOAA · 2023-12-07T02:51:09Z

Yeah, that sounds good

climbfuji · 2023-12-07T03:25:26Z

Discover just finished, too.

RatkoVasic-NOAA

Tested on Hera, Jet and Gaea-c5. Approved.

haampie and others added 30 commits October 25, 2023 17:35

ci: darwin aarch64 use apple-clang-15 tag (spack#40706)

0c30418

build(deps): bump pytest from 7.4.2 to 7.4.3 in /lib/spack/docs (spac…

6bd2dd0

…k#40697)

PyTorch: patch breakpad dependency (spack#40648)

932d7a6

Added Highway versions up to 1.0.7 (spack#40691)

2a245fd

cuda: add NVHPC_CUDA_HOME. (spack#40507)

d9edc92

* [cuda] Add NVHPC_CUDA_HOME. * Add CUDA_HOME and NVHC_CUDA_HOME to cuda's dependent build env. --------- Co-authored-by: Dominic Hofer <[email protected]>

initial commit to fix mivisionx build for 5.6 (spack#40579)

3e47f3f

fdb: add releases v5.11.23 and v5.11.17 (spack#40571)

3c590ad

Paraview 5.12 prep (spack#40527)

8e19576

* paraview: rebase the adios2 patch for 5.12-to-be * paraview: disable fastfloat and token for 5.12-to-be * paraview: require older protobuf for 5.12 as well * paraview: require C++11-supporting protobuf for `master` too

git versions: fix commit shas [automated] (spack#40703)

b4cf3d9

libluv: require CMake 3 and CMP0042 (spack#40716)

bf88ed4

spack checksum: show long flags in usage output (spack#40407)

1c8073c

PythonPackage: nested config_settings (spack#40693)

f57c250

* PythonPackage: nested config_settings * flake8

Add conflict between cxxstd > 17 and cuda < 12 in pika (spack#40717)

53d5011

* Add conflict with C++ standard > 17 and cuda < 12 * Removing map_cxxstd since boost supports C++20 flag

unmaintained a* packages: update to use f-strings (spack#40467)

cbf9dd0

curl: Fix librtmp variant (spack#40713)

81172f9

* rtmpdump: New package * curl: Fix librtmp variant Add the previously missing dependency required for rtmp support. The variant has been broken since its addition in PR spack#25166. Fixes one of the two issues reported in spack#26887.

PythonPackage: allow archive_files to be overridden (spack#40694)

b4b25de

dataTransferKit: add v3.1.1, v3.1.0 (spack#40556)

d64f312

* Update DataTransferKit for 3.1.1 release * Require Trilinos-14 for 3.1.0 and higher

ldak: add v5.2 & add maintainer (spack#40710)

c3f5ee5

* ldak: update to 5.2, add maintainer * ldak: use compiler.openmp_flag

itk: misc fixes (spack#39832)

546695f

* itk: patch missing include for newer compilers * itk: The package doesn't use MPI * itk: package requires the high-level hdf5 api * itk: patch url with ?full_index=1 * itk: point to 4041 commit in master * itk: don't constrain hdf5 with ~mpi

plasma: add version 23.8.2 (spack#40728)

8959d65

akantu: use f-strings (spack#40466)

feda52f

Co-authored-by: Nicolas Richart <[email protected]>

gromacs: fix version branch in intel fftw (spack#40489)

37f48af

ci: spack compiler find should list extra config scopes (spack#40727)

51275df

otherwise it detected pre-configured compilers in an potentially different way.

gromacs: default to external blas & lapack (spack#40490)

9b8fb41

* gromacs: default to external blas & lapack * drop vendored lapack/blas altogether

AlexanderRichert-NOAA added 4 commits November 29, 2023 15:21

allow py-cryptography to use rust-bootstrap

6669ab0

style fix prod-util

c09cd7d

add rust-bootstrap option for py-setuptools-rust

8d9cb63

fix implicit declaration issue for apple-clang in hdf-eos2

7b9def0

remove flag_handler from hdf-eos2

e016228

Merge branch 'jcsda_emc_spack_stack' into nov2023_spackmerge

98aaec6

AlexanderRichert-NOAA marked this pull request as ready for review December 1, 2023 03:33

climbfuji added 2 commits December 4, 2023 06:57

Merge branch 'jcsda_emc_spack_stack' of https://github.com/jcsda/spack …

e2c7af8

…into HEAD

Merge branch 'nov2023_spackmerge' of https://github.com/AlexanderRich…

1b8756c

…ert-NOAA/spack into nov2023_spackmerge

symoore90 added the INFRA JEDI Infrastructure label Dec 4, 2023

fix python dep for met

12e7527

AlexanderRichert-NOAA mentioned this pull request Dec 5, 2023

Add util to check executables/shared libs for missing shared libs JCSDA/spack-stack#890

Merged

3 tasks

climbfuji requested review from RatkoVasic-NOAA, ulmononian, natalie-perlin and climbfuji December 7, 2023 02:40

climbfuji approved these changes Dec 7, 2023

View reviewed changes

climbfuji mentioned this pull request Dec 7, 2023

Add cylc-flow, cylc-rose, cylc-uiserver from spack develop; update setup-meta-modules extension for Python #377

Merged

3 tasks

RatkoVasic-NOAA approved these changes Dec 7, 2023

View reviewed changes

ulmononian approved these changes Dec 7, 2023

View reviewed changes

climbfuji merged commit 8683cc4 into JCSDA:jcsda_emc_spack_stack Dec 7, 2023
13 of 15 checks passed

srherbener mentioned this pull request Dec 7, 2023

py-scipy does not build on Mac using [email protected] JCSDA/spack-stack#882

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Nov 2023 spack merge #371

Nov 2023 spack merge #371

AlexanderRichert-NOAA commented Nov 21, 2023 •

edited by climbfuji

Loading

AlexanderRichert-NOAA commented Nov 30, 2023

climbfuji commented Nov 30, 2023

AlexanderRichert-NOAA commented Nov 30, 2023

climbfuji commented Nov 30, 2023 via email

climbfuji left a comment

AlexanderRichert-NOAA commented Dec 7, 2023

climbfuji commented Dec 7, 2023

AlexanderRichert-NOAA commented Dec 7, 2023

climbfuji commented Dec 7, 2023

RatkoVasic-NOAA left a comment

Nov 2023 spack merge #371

Nov 2023 spack merge #371

Conversation

AlexanderRichert-NOAA commented Nov 21, 2023 • edited by climbfuji Loading

Description

Checklist

AlexanderRichert-NOAA commented Nov 30, 2023

climbfuji commented Nov 30, 2023

AlexanderRichert-NOAA commented Nov 30, 2023

climbfuji commented Nov 30, 2023 via email

climbfuji left a comment

Choose a reason for hiding this comment

AlexanderRichert-NOAA commented Dec 7, 2023

climbfuji commented Dec 7, 2023

AlexanderRichert-NOAA commented Dec 7, 2023

climbfuji commented Dec 7, 2023

RatkoVasic-NOAA left a comment

Choose a reason for hiding this comment

AlexanderRichert-NOAA commented Nov 21, 2023 •

edited by climbfuji

Loading