-
Notifications
You must be signed in to change notification settings - Fork 119
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] Move CI infrastructure to GitHub Actions #3321
Draft
JCGoran
wants to merge
31
commits into
master
Choose a base branch
from
jelic/move_infra_to_gha
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 30 commits
Commits
Show all changes
31 commits
Select commit
Hold shift + click to select a range
952d3d9
Move to `manylinux_2_28` base image
JCGoran 0b01fd8
Test manylinux_2_28 image
JCGoran 38aa6c4
Modify Dockerfile and build script
JCGoran c7ae29e
Remove BBP-specific testing
JCGoran 2571c65
Set min version of GCC everywhere
JCGoran d29bf6f
Change tag of Docker image
JCGoran de3fcc8
Cleanup Docker image to reduce size
JCGoran 44ef470
MPI headers need some tweaking
JCGoran 4ffe46c
Fix checking of compiler
JCGoran fae53da
Update docs and CI vars
JCGoran e95b63f
Add CI for building wheels on MacOS on arm64
JCGoran 1e0293f
Add merging of artifacts to wheel CI job
JCGoran 4bc8e31
Add `pattern: wheels-*` to merge job in CI
JCGoran 1a8e344
Append `matrix.os` to artifact name
JCGoran c8b9277
Update .github/workflows/wheels.yml
JCGoran f461d85
Add other configs to CI
JCGoran cbeb0ce
Remove azure and circleCI
JCGoran 14f925d
Fix CI issue
JCGoran bb8ac58
Do not use git shenanigans when building wheels
JCGoran de7d057
Try normal Python on MacOS
JCGoran dcfb8a1
Minor fixes
JCGoran 3481ccb
Typo
JCGoran 3acfbe5
Maybe quotes?
JCGoran 9f41e67
Maybe dots are the problem
JCGoran 9303ff6
Separate build and test steps
JCGoran 30c69a2
Forgot steps
JCGoran d1ace17
Revert back to testing after build
JCGoran 3e1817c
Fix missing property
JCGoran 75a30b9
Remove container
JCGoran 5ada0ee
Merge into one big job
JCGoran f14461f
Merge branch 'master' into jelic/move_infra_to_gha
JCGoran File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,168 @@ | ||
name: Build NEURON Python wheels | ||
|
||
on: | ||
push: | ||
branches: | ||
- master | ||
pull_request: | ||
branches: | ||
- master | ||
|
||
workflow_dispatch: | ||
inputs: | ||
rel_release: | ||
description: Release branch/commit | ||
default: 'release/x.y' | ||
required: true | ||
rel_version: | ||
description: Release version (tag name) | ||
default: 'x.y.z' | ||
required: true | ||
|
||
|
||
jobs: | ||
build-test: | ||
name: Build and test Python ${{ matrix.python-version }} wheel on ${{ matrix.os }} | ||
runs-on: ${{ matrix.os }} | ||
timeout-minutes: 60 | ||
strategy: | ||
matrix: | ||
os: ['macos-13', 'macos-14', 'ubuntu-22.04', 'ubuntu-22.04-arm'] | ||
python-version: ['3.9', '3.13'] | ||
include: | ||
- python-version: '3.9' | ||
python-org-version: '3.9.13' | ||
python-installer-name: 'macos11.pkg' | ||
|
||
- python-version: '3.13' | ||
python-org-version: '3.13.0' | ||
python-installer-name: 'macos11.pkg' | ||
|
||
- os: 'ubuntu-22.04' | ||
base_image: 'docker.io/neuronsimulator/neuron_wheel:manylinux_2_28_x86_64' | ||
|
||
- os: 'ubuntu-22.04-arm' | ||
base_image: 'docker.io/neuronsimulator/neuron_wheel:manylinux_2_28_aarch64' | ||
|
||
steps: | ||
- name: Check out code | ||
if: github.event_name != 'workflow_dispatch' | ||
uses: actions/checkout@v4 | ||
with: | ||
submodules: recursive | ||
|
||
- name: Check out code for release | ||
if: github.event_name == 'workflow_dispatch' | ||
uses: actions/checkout@v4 | ||
with: | ||
ref: ${{ github.inputs.rel_release }} | ||
submodules: recursive | ||
|
||
- name: Install Python from python.org | ||
if: runner.os == 'macOS' | ||
run: | | ||
installer="python-${{ matrix.python-org-version }}-${{ matrix.python-installer-name }}" | ||
url="https://www.python.org/ftp/python/${{ matrix.python-org-version }}/${installer}" | ||
curl $url -o $installer | ||
sudo installer -pkg $installer -target / | ||
|
||
- name: Install System Dependencies | ||
if: runner.os == 'macOS' | ||
run: | | ||
brew install --cask xquartz | ||
brew install flex bison cmake mpich | ||
brew unlink mpich && brew install openmpi | ||
cmake --version | ||
# Uninstall libomp for compatibility with issue #817 | ||
brew uninstall --ignore-dependencies libomp || echo "libomp doesn't exist" | ||
echo "$(brew --prefix)/opt/cmake/bin:$(brew --prefix)/opt/flex/bin:$(brew --prefix)/opt/bison/bin" >> $GITHUB_PATH | ||
|
||
- name: Install readline | ||
if: runner.os == 'macOS' | ||
run: | | ||
sudo mkdir -p /opt/nrnwheel/$(uname -m) | ||
sudo bash packaging/python/build_static_readline_osx.bash | ||
|
||
- name: Set env for release | ||
if: github.event_name == 'workflow_dispatch' | ||
run: | | ||
echo "NRN_NIGHTLY_UPLOAD=false" >> $GITHUB_ENV | ||
echo NRN_RELEASE_UPLOAD=${{ github.event.inputs.upload }} >> $GITHUB_ENV | ||
echo "NEURON_NIGHTLY_TAG=" >> $GITHUB_ENV | ||
echo SETUPTOOLS_SCM_PRETEND_VERSION=${{ github.event.inputs.rel_version }} >> $GITHUB_ENV | ||
|
||
- name: Build wheel | ||
if: runner.os == 'macOS' | ||
run: | | ||
packaging/python/build_wheels.bash osx ${{ matrix.python-version }} coreneuron | ||
|
||
- name: Build Manylinux wheel | ||
if: runner.os != 'macOS' | ||
run: | | ||
docker run --rm \ | ||
-w /root/nrn \ | ||
-v $(pwd):/root/nrn \ | ||
-e NEURON_NIGHTLY_TAG \ | ||
-e NRN_NIGHTLY_UPLOAD \ | ||
-e NRN_RELEASE_UPLOAD \ | ||
-e SETUPTOOLS_SCM_PRETEND_VERSION \ | ||
-e NRN_BUILD_FOR_UPLOAD=1 \ | ||
${{ matrix.base_image }} \ | ||
packaging/python/build_wheels.bash linux ${{ matrix.python-version }} coreneuron | ||
|
||
- name: Upload wheel files | ||
uses: actions/upload-artifact@v4 | ||
with: | ||
name: wheels-${{ matrix.python-version }}-${{ matrix.os }} | ||
path: wheelhouse/*.whl | ||
|
||
- name: Setup Python ${{ matrix.python-version }} for testing | ||
uses: actions/setup-python@v5 | ||
with: | ||
python-version: ${{ matrix.python-version }} | ||
|
||
- name: Install test dependencies | ||
if: runner.os != 'macOS' | ||
run: | | ||
sudo apt update | ||
sudo apt install -y mpich openmpi-bin libopenmpi-dev libmpich-dev | ||
|
||
- name: Test wheel with ${{ matrix.python-version }} | ||
run: | | ||
minor_version="$(python${{ matrix.python-version }} -c 'import sys;print(sys.version_info.minor)')" | ||
packaging/python/test_wheels.sh $(which python${{ matrix.python-version }}) wheelhouse/*cp3${minor_version}*.whl | ||
|
||
merge: | ||
name: Merge artifacts | ||
runs-on: ubuntu-latest | ||
needs: [build-test] | ||
steps: | ||
- name: Merge Artifacts | ||
uses: actions/upload-artifact/merge@v4 | ||
with: | ||
delete-merged: true | ||
name: wheels | ||
pattern: wheels-* | ||
|
||
upload: | ||
name: Upload wheels | ||
runs-on: ubuntu-latest | ||
needs: merge | ||
steps: | ||
- name: Upload wheels | ||
run: echo "TODO" | ||
|
||
final: | ||
name: Final CI | ||
needs: [merge] | ||
if: ${{ always() }} | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Check if CI matrix is successful | ||
if: >- | ||
${{ | ||
contains(needs.*.result, 'failure') | ||
|| contains(needs.*.result, 'cancelled') | ||
|| contains(needs.*.result, 'skipped') | ||
}} | ||
run: exit 1 |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@JCGoran : Just mentioning - we need to update the docs from https://nrn.readthedocs.io/en/8.2.6/install/python_wheels.html#publishing-the-wheels-on-pypi-via-azure.
Especially, having the ability to just build the release artifacts without upload has been quite helpful.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, there's also the matter of "Azure drop URLs" being mentioned in other repos, so we should make sure those are modified accordingly.