Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Move CI infrastructure to GitHub Actions #3321

Draft
wants to merge 31 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
952d3d9
Move to `manylinux_2_28` base image
JCGoran Jan 10, 2025
0b01fd8
Test manylinux_2_28 image
JCGoran Jan 10, 2025
38aa6c4
Modify Dockerfile and build script
JCGoran Jan 13, 2025
c7ae29e
Remove BBP-specific testing
JCGoran Jan 13, 2025
2571c65
Set min version of GCC everywhere
JCGoran Jan 13, 2025
d29bf6f
Change tag of Docker image
JCGoran Jan 13, 2025
de3fcc8
Cleanup Docker image to reduce size
JCGoran Jan 13, 2025
44ef470
MPI headers need some tweaking
JCGoran Jan 13, 2025
4ffe46c
Fix checking of compiler
JCGoran Jan 13, 2025
fae53da
Update docs and CI vars
JCGoran Jan 14, 2025
e95b63f
Add CI for building wheels on MacOS on arm64
JCGoran Nov 15, 2024
1e0293f
Add merging of artifacts to wheel CI job
JCGoran Nov 18, 2024
4bc8e31
Add `pattern: wheels-*` to merge job in CI
JCGoran Nov 18, 2024
1a8e344
Append `matrix.os` to artifact name
JCGoran Nov 19, 2024
c8b9277
Update .github/workflows/wheels.yml
JCGoran Dec 5, 2024
f461d85
Add other configs to CI
JCGoran Jan 28, 2025
cbeb0ce
Remove azure and circleCI
JCGoran Jan 28, 2025
14f925d
Fix CI issue
JCGoran Jan 28, 2025
bb8ac58
Do not use git shenanigans when building wheels
JCGoran Jan 28, 2025
de7d057
Try normal Python on MacOS
JCGoran Jan 28, 2025
dcfb8a1
Minor fixes
JCGoran Jan 28, 2025
3481ccb
Typo
JCGoran Jan 28, 2025
3acfbe5
Maybe quotes?
JCGoran Jan 28, 2025
9f41e67
Maybe dots are the problem
JCGoran Jan 28, 2025
9303ff6
Separate build and test steps
JCGoran Jan 28, 2025
30c69a2
Forgot steps
JCGoran Jan 28, 2025
d1ace17
Revert back to testing after build
JCGoran Jan 28, 2025
3e1817c
Fix missing property
JCGoran Jan 28, 2025
75a30b9
Remove container
JCGoran Jan 28, 2025
5ada0ee
Merge into one big job
JCGoran Jan 28, 2025
f14461f
Merge branch 'master' into jelic/move_infra_to_gha
JCGoran Jan 28, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 0 additions & 98 deletions .circleci/config.yml

This file was deleted.

168 changes: 168 additions & 0 deletions .github/workflows/wheels.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
name: Build NEURON Python wheels

on:
push:
branches:
- master
pull_request:
branches:
- master

workflow_dispatch:
inputs:
rel_release:
description: Release branch/commit
default: 'release/x.y'
required: true
rel_version:
description: Release version (tag name)
default: 'x.y.z'
required: true


jobs:
build-test:
name: Build and test Python ${{ matrix.python-version }} wheel on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
timeout-minutes: 60
strategy:
matrix:
os: ['macos-13', 'macos-14', 'ubuntu-22.04', 'ubuntu-22.04-arm']
python-version: ['3.9', '3.13']
include:
- python-version: '3.9'
python-org-version: '3.9.13'
python-installer-name: 'macos11.pkg'

- python-version: '3.13'
python-org-version: '3.13.0'
python-installer-name: 'macos11.pkg'

- os: 'ubuntu-22.04'
base_image: 'docker.io/neuronsimulator/neuron_wheel:manylinux_2_28_x86_64'

- os: 'ubuntu-22.04-arm'
base_image: 'docker.io/neuronsimulator/neuron_wheel:manylinux_2_28_aarch64'

steps:
- name: Check out code
if: github.event_name != 'workflow_dispatch'
uses: actions/checkout@v4
with:
submodules: recursive

- name: Check out code for release
if: github.event_name == 'workflow_dispatch'
uses: actions/checkout@v4
with:
ref: ${{ github.inputs.rel_release }}
submodules: recursive

- name: Install Python from python.org
if: runner.os == 'macOS'
run: |
installer="python-${{ matrix.python-org-version }}-${{ matrix.python-installer-name }}"
url="https://www.python.org/ftp/python/${{ matrix.python-org-version }}/${installer}"
curl $url -o $installer
sudo installer -pkg $installer -target /

- name: Install System Dependencies
if: runner.os == 'macOS'
run: |
brew install --cask xquartz
brew install flex bison cmake mpich
brew unlink mpich && brew install openmpi
cmake --version
# Uninstall libomp for compatibility with issue #817
brew uninstall --ignore-dependencies libomp || echo "libomp doesn't exist"
echo "$(brew --prefix)/opt/cmake/bin:$(brew --prefix)/opt/flex/bin:$(brew --prefix)/opt/bison/bin" >> $GITHUB_PATH

- name: Install readline
if: runner.os == 'macOS'
run: |
sudo mkdir -p /opt/nrnwheel/$(uname -m)
sudo bash packaging/python/build_static_readline_osx.bash

- name: Set env for release
if: github.event_name == 'workflow_dispatch'
run: |
echo "NRN_NIGHTLY_UPLOAD=false" >> $GITHUB_ENV
echo NRN_RELEASE_UPLOAD=${{ github.event.inputs.upload }} >> $GITHUB_ENV
echo "NEURON_NIGHTLY_TAG=" >> $GITHUB_ENV
echo SETUPTOOLS_SCM_PRETEND_VERSION=${{ github.event.inputs.rel_version }} >> $GITHUB_ENV

- name: Build wheel
if: runner.os == 'macOS'
run: |
packaging/python/build_wheels.bash osx ${{ matrix.python-version }} coreneuron

- name: Build Manylinux wheel
if: runner.os != 'macOS'
run: |
docker run --rm \
-w /root/nrn \
-v $(pwd):/root/nrn \
-e NEURON_NIGHTLY_TAG \
-e NRN_NIGHTLY_UPLOAD \
-e NRN_RELEASE_UPLOAD \
-e SETUPTOOLS_SCM_PRETEND_VERSION \
-e NRN_BUILD_FOR_UPLOAD=1 \
${{ matrix.base_image }} \
packaging/python/build_wheels.bash linux ${{ matrix.python-version }} coreneuron

- name: Upload wheel files
uses: actions/upload-artifact@v4
with:
name: wheels-${{ matrix.python-version }}-${{ matrix.os }}
path: wheelhouse/*.whl

- name: Setup Python ${{ matrix.python-version }} for testing
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install test dependencies
if: runner.os != 'macOS'
run: |
sudo apt update
sudo apt install -y mpich openmpi-bin libopenmpi-dev libmpich-dev

- name: Test wheel with ${{ matrix.python-version }}
run: |
minor_version="$(python${{ matrix.python-version }} -c 'import sys;print(sys.version_info.minor)')"
packaging/python/test_wheels.sh $(which python${{ matrix.python-version }}) wheelhouse/*cp3${minor_version}*.whl

merge:
name: Merge artifacts
runs-on: ubuntu-latest
needs: [build-test]
steps:
- name: Merge Artifacts
uses: actions/upload-artifact/merge@v4
with:
delete-merged: true
name: wheels
pattern: wheels-*

upload:
name: Upload wheels
runs-on: ubuntu-latest
needs: merge
steps:
- name: Upload wheels
run: echo "TODO"
Comment on lines +151 to +153
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JCGoran : Just mentioning - we need to update the docs from https://nrn.readthedocs.io/en/8.2.6/install/python_wheels.html#publishing-the-wheels-on-pypi-via-azure.
Especially, having the ability to just build the release artifacts without upload has been quite helpful.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, there's also the matter of "Azure drop URLs" being mentioned in other repos, so we should make sure those are modified accordingly.


final:
name: Final CI
needs: [merge]
if: ${{ always() }}
runs-on: ubuntu-latest
steps:
- name: Check if CI matrix is successful
if: >-
${{
contains(needs.*.result, 'failure')
|| contains(needs.*.result, 'cancelled')
|| contains(needs.*.result, 'skipped')
}}
run: exit 1
Loading
Loading