Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

document contribution policy for adding software to EESSI #108

Merged
merged 11 commits into from
Nov 9, 2023
Merged

document contribution policy for adding software to EESSI #108

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
544d928
document contribution policy for adding software to EESSI
boegel Jul 5, 2023
3e25c08
tweak initial contribution policy based on provided feedback
boegel Oct 24, 2023
d559a58
Merge branch 'main' into contrib_policy
boegel Oct 24, 2023
00bb6df
link to test-suite section in contribution policy
boegel Oct 24, 2023
86f99c3
rework v0.1.0 of contribution policy based on provided feedback
boegel Nov 9, 2023
4d00501
"It informs contributors"
boegel Nov 9, 2023
3d2600a
Merge branch 'main' into contrib_policy
boegel Nov 9, 2023
991f1e6
Merge branch 'contrib_policy' of github.com:boegel/docs into contrib_…
boegel Nov 9, 2023
5b4240f
fix links to other docs pages
boegel Nov 9, 2023
c580bfd
fix more links to other docs pages, consistently use .md in links to …
boegel Nov 9, 2023
cef6f85
fix typo
boegel Nov 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 111 additions & 0 deletions docs/contribution_policy.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
# Contribution policy

*(version v0.1.0 - updated 24 Oct 2023)*

!!! note

This policy is subject to change, please check back regularly.

## Purpose

The purpose of this contribution policy is to provide guidelines for [adding software](software_layer/adding_software.md) to EESSI.

It informs about what requirements must be met in order for software to be eligible for inclusion
boegel marked this conversation as resolved.
Show resolved Hide resolved
in the EESSI software layer.

## Requirements

The following requirements must be taken into account when adding software to EESSI.

---

### a) Open source software { #open_source_software }

Only **open source software** can be added to the EESSI repository.

Make sure that you are aware of relevant software license, and that redistribution is allowed.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small typo?

Suggested change
Make sure that you are aware of relevant software license, and that redistribution is allowed.
Make sure that you are aware of relevant software licenses, and that redistribution is allowed.

or even?

Suggested change
Make sure that you are aware of relevant software license, and that redistribution is allowed.
Make sure that the software uses an open source license, and that redistribution is allowed.

Copy link
Member

@ocaisa ocaisa Oct 26, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit-picking but the original formulation is better, we do allow binary distribution as long as the redistribution is permitted by the licence (CUDA for example falls into this category, we distribute the runtime)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ocaisa Then this shouldn't refer to "open source" software at all?
That is going to make the wording here a lot less clear though...


For more information about a specific license, see the [SPDX license list](https://spdx.org/licenses/).

!!! note

We intend to automatically verify that this requirement is met,
by requiring that the [SPDX license identifier](https://spdx.dev/ids/) is provided for all software.


---

### b) Built by the bot { #built_by_bot }

All software included in the EESSI repository *must* be built autonomously by [our bot :robot:](bot.md),
see also the [semi-automatic software installation procedure](software_layer/adding_software.md).


---

### c) Supported by EasyBuild { #supported_by_easybuild }

Currently, we require that all software being added to EESSI is supported by the *EasyBuild release* being used
to perform the installation.

That is, the easyconfig files used for the installation *must be included in the EasyBuild release*.

We do allow the use of [`--from-pr`](https://docs.easybuild.io/integration-with-github/#github_from_pr) and
[`--include-easyblocks-from-pr`](https://docs.easybuild.io/integration-with-github/#github_include_easyblocks_from_pr)
to pull in changes required to make the installation work correctly in the EESSI build environment,
but only if that is strictly required.


---

### d) Compiler toolchain { #compiler_toolchain }

A [compiler toolchain](https://docs.easybuild.io/terminology/#toolchains) that is still supported by the latest
EasyBuild must be used for building the software.

More information on supported toolchains in EasyBuild is available [here](https://docs.easybuild.io/policies/toolchains).


---

### e) CPU targets { #cpu_targets }

The software *should* work on all [CPU targets supported by EESSI](software_layer/cpu_targets.md).

Exceptions to this requirement are allowed if technical problems that can not be resolved with reasonable effort
prevent the installation of the software for specific CPU targets.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use architecture instead of targets?

Suggested change
### e) CPU targets { #cpu_targets }
The software *should* work on all [CPU targets supported by EESSI](software_layer/cpu_targets.md).
Exceptions to this requirement are allowed if technical problems that can not be resolved with reasonable effort
prevent the installation of the software for specific CPU targets.
### e) CPU architectures { #cpu_targets }
The software *should* work on all [CPU architectures supported by EESSI](software_layer/cpu_targets.md).
Exceptions to this requirement are allowed if technical problems that can not be resolved with reasonable effort
prevent the installation of the software for specific CPU architectures.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@trz42 We use "targets" in https://www.eessi.io/docs/software_layer/cpu_targets, so sticking to targets is more consistent?

I guess we could use "target CPU architectures"? (it should really be microarchitectures, though)



---

### f) Versions & toolchains { #versions_toolchains }

Recent software versions and toolchains *should* be preferred,
although the installation of older software versions and toolchains is allowed if sufficiently motivated.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not keen on this, we open the door to a lot of pain by looking backwards. In general, I do believe we should decide what toolchains to support on a specific compat and stick to it, and probably take the opportunity once a year to update the compat layer

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That feels like swinging over to the other side a bit too much (by "locking" a toolchain version to a particular EESSI version, and hence compat layer).

Keeping compat layers updated is gradually going to become quite a bit of effort, and so we may want to install an existing toolchain version in a newer compat layer at some point, since that's likely to be a lot less painful than updating a previous compat layer version.

I guess we could stick to "a toolchain version is only installed on top of a single compat layer version" initially, and then see if that makes sense going forward, and then adjust accordingly if needed.

But should this be part of the contribution policy?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm on the fence on this one. On the one hand, for a good user / contributor experience, discouraging of forbidding the deployment of old toolchains will probably lead to fewer issues. On the other hand: if someone needs an older toolchain, and if it works, I see little reason not to accept a PR like that.

I had a chat yesterday with someone from a university HPC support staff who asked 'if my user asks for something from a 2019b toolchain, because he needs the particular software version in that toolchain, can I deploy that through EESSI?'. Even though he fully agreed we should give pushback to users requesting this stuff, in the end he's also realistic and just wants to help the researcher. If it were a local software stack, he would have definitely given it a try. If our policy 'forbids' that, he would be pushed to a local solution. I'm a bit concerned if it wouldn't turn HPC support staff away from using EESSI if we are too restrictive on these things.

I guess we could stick to "a toolchain version is only installed on top of a single compat layer version" initially, and then see if that makes sense going forward, and then adjust accordingly if needed.

What do you mean here? If foss-2022a went into 2023.06, it cannot also go into 2023.12? I wouldn't do that. I think Alan's suggestion was more like: we can decide for 2023.06 to support toolchains 2021a onwards, for 2023.12 we do 2021b onwards, etc. So then there is overlap (2022a would still be on top of both versions of the compat layer), but it prevents deployment of very old toolchains on new compat layers (which probably won't work anyway).

The more I think about it, the more I am not in favour of limiting this initially in the policy. I think people will find it's hard to get older compilers to build, and older MPIs to function properly. In practice, I think they might still be blocked from contributing such toolchains, simply because it's impossible to get them to work (on new compat layers, at least). I wouldn't forbid that, just let the technical limitation be the blocker, not our 'rule'. And, I agree we could still mention this (i.e. old stuff is discouraged, likely to cause issues, and the community probably won't be very motivated to help you out) clearly in the policy.

Another advantage of not limiting it initially is that it allows us to build experience, which we can later use to make an informed decision to forbid it after all (if needed).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@casparvl We discussed this aspect during the bot sync meeting earlier today, and I've made some changes to the policy in 86f99c3 based on that.

There are now separate entries for recent toolchains and recent software versions, where the former mentions that contributors should either use already installed toolchains, or submit a motivated support request to get an additional toolchain installed, which puts the ball in our camp, and leaves the door open for contributors.

There was a consensus during the meeting that this makes sense for the initial version of the policy, and we can revise that later should the need arise.
In practice, it's very likely anyway that is will be EESSI admins who will add toolchains.

We also briefly discussed that we should only install the latest N toolchains in the latest version of EESSI at any time, which allows for minimal overlap across EESSI versions (but that's not part of the policy, that's more of an internal thing we should try and stick to).
In practice this will probably mostly be governed by the compatibility of the glibc version in the compat layer and the GCC version in the toolchains...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In EESSI/software-layer#371 I've included the ability for both the site and the user to extend EESSI as they see fit. So even if we don't support it (most likely because of the architecture situation) doesn't mean that they can't do that locally if it "just works".



---

### g) Testing { #testing }

We should be able to test the software installations via the [EESSI test suite](../test-suite).

Ideally one or more tests are available that verify that the software is functionally correct,
and performs well.

It should be possible to run a minimal *smoke test*, for example using EasyBuild's `--sanity-check-only` feature.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question is, should we ask contributors to help us define a meaningful test?

Such tests could be added to the EESSI test suite. "Ideally" sounds like a very loose requirement ... and missed opportunity to me.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making this a requirement for someone who is really just an end-user is a high bar. I agree for that marquee applications we should have such tests, but it's not realistic to expect that for all cases. In general, it's probably a much more pro-active effort on our part where we'd need to ensure we are reaching out to the actual developers (or very experienced users)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This indeed necessarily needs to be a rather loose requirement, or we will be way too restrictive, and won't get any contributions because of that.

It's definitely unreasonable to require that a test must be available in the EESSI test suite, since adding something there is far from trivial (requires knowing Python, ReFrame, how to write a portable test, etc.).
We can probably provide some documentation with guidelines on how to do that eventually, but then it will still be quite an elaborate task.

We should phrase this such that it's clear that we prefer being able to test the software, in one way or another, without imposing a huge amount of effort on contributors, at least initially.
Maybe we can mention options like providing a test case, referring to an example or documentation of a basic run, etc.?


!!! note

The [EESSI test suite](https://github.com/EESSI/test-suite) is still in active development,
and currently only has a minimal set of tests available.

When the test suite is more mature, this requirement will be enforced more strictly.

---

## Changelog

### v0.1.0 (24 Oct 2023)

- initial contribution policy
1 change: 1 addition & 0 deletions docs/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ More details about the project are available in the different subsections:
* [Filesystem layer](filesystem_layer.md)
* [Compatibility layer](compatibility_layer.md)
* [Software layer](software_layer.md)
* [Contribution policy](contribution_policy.md)
* [Pilot repository](pilot.md)
* [Getting access to EESSI](getting_access/is_eessi_accessible.md)
* [Using EESSI](using_eessi/setting_up_environment.md)
Expand Down
4 changes: 4 additions & 0 deletions docs/software_layer/adding_software.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ To add software to EESSI, you should go through the semi-automatic software inst
* 3) Instructing the [bot :robot:](../bot.md) to deploy the built software for ingestion into the EESSI repository;
* 4) Merging the pull request once CI indicates that the software has been ingested. :white_check_mark:

!!! warning

Make sure you are also aware of our [contribution policy](../contribution_policy.md) when adding software to EESSI.

### Preparation

Before you can make a pull request to the [software-layer](https://github.com/EESSI/software-layer),
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ nav:
- software_layer/cpu_targets.md
- software_layer/build_nodes.md
- software_layer/adding_software.md
- Contribution policy: contribution_policy.md
- Test suite:
- Overview: test-suite/index.md
- Installation & configuration: test-suite/installation-configuration.md
Expand Down