Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Spack #3980

Open
wants to merge 197 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 191 commits
Commits
Show all changes
197 commits
Select commit Hold shift + click to select a range
ec00dfd
remove some workflows temporarily
paulgessinger Dec 11, 2024
1a7fd8c
test
paulgessinger Dec 11, 2024
81f61ac
use correct folder
paulgessinger Dec 11, 2024
5cf1d76
prog
paulgessinger Dec 11, 2024
bc75425
add missing file
paulgessinger Dec 11, 2024
f128513
script_dir
paulgessinger Dec 11, 2024
d394c3b
maybe sudo
paulgessinger Dec 11, 2024
84ab8aa
download lock file
paulgessinger Dec 11, 2024
75867aa
run the install
paulgessinger Dec 11, 2024
9daaddd
don't activate
paulgessinger Dec 11, 2024
d308cf6
named
paulgessinger Dec 11, 2024
4d044f3
add the build cache
paulgessinger Dec 11, 2024
963c13c
add some other jobs
paulgessinger Dec 11, 2024
7414ac9
correct xcode version
paulgessinger Dec 11, 2024
e1aa09e
lock to xcode 16.1.0
paulgessinger Dec 11, 2024
6514dd9
another error
paulgessinger Dec 11, 2024
5b9ac21
print folder, create env
paulgessinger Dec 12, 2024
04e302d
add view creation
paulgessinger Dec 12, 2024
527c06f
get g4 data
paulgessinger Dec 12, 2024
a961147
different root detection mechanism
paulgessinger Dec 12, 2024
2826fe6
other env mechanism
paulgessinger Dec 12, 2024
005638c
get python going
paulgessinger Dec 12, 2024
0b691bb
paths
paulgessinger Dec 12, 2024
2f9ecba
debug
paulgessinger Dec 12, 2024
8d3ce2c
make python stuff later
paulgessinger Dec 12, 2024
46a3f86
more debug
paulgessinger Dec 12, 2024
c90864b
different way of setting PATH
paulgessinger Dec 12, 2024
02f7f89
enable build
paulgessinger Dec 12, 2024
edc202e
set CMake prefix path
paulgessinger Dec 12, 2024
54f3ec9
additional variables
paulgessinger Dec 12, 2024
f6bafc6
use more CPUs to install
paulgessinger Dec 12, 2024
f56a378
add timing
paulgessinger Dec 12, 2024
2e59eb2
try better parallel
paulgessinger Dec 13, 2024
d374232
number
paulgessinger Dec 13, 2024
3c27432
make parallel script executable
paulgessinger Dec 13, 2024
75bb108
higher count on install from buildcache
paulgessinger Dec 13, 2024
a2180e7
separate concretize
paulgessinger Dec 13, 2024
7c8fa52
install combo
paulgessinger Dec 13, 2024
faeea38
correct tag
paulgessinger Dec 13, 2024
2f82d79
don't overlap the install
paulgessinger Dec 13, 2024
8ddd0e1
i don't understand the spec
paulgessinger Dec 13, 2024
288fde5
move all to a script
paulgessinger Dec 13, 2024
1e7b4fe
run
paulgessinger Dec 13, 2024
4e9e43c
all
paulgessinger Dec 13, 2024
c87a1dc
set it up if we just installed it
paulgessinger Dec 13, 2024
d402b93
correct tag
paulgessinger Dec 13, 2024
b83fe5a
diagnostics
paulgessinger Dec 13, 2024
b0cbe89
use the right python
paulgessinger Dec 13, 2024
8ca7c8d
diag
paulgessinger Dec 13, 2024
dd83cfb
output filter
paulgessinger Dec 13, 2024
13975e3
limit parallel
paulgessinger Dec 13, 2024
2b4fc82
update tag
paulgessinger Dec 13, 2024
f9eab1f
quotes
paulgessinger Dec 13, 2024
c9b8a26
long spec output
paulgessinger Dec 13, 2024
0b17997
see if the other builds work
paulgessinger Dec 13, 2024
8403c02
fix: Use toUnderlying to avoid cross-enum comparison
paulgessinger Dec 13, 2024
f23d4a4
Merge branch 'main' into spack-ci
paulgessinger Dec 13, 2024
ac6c351
Revert "fix: Use toUnderlying to avoid cross-enum comparison"
paulgessinger Dec 13, 2024
fc6f99b
update concepts with toUnderlying
paulgessinger Dec 13, 2024
8ae3da1
another compile fix
paulgessinger Dec 14, 2024
763798e
be smarter about selecting a compiler
paulgessinger Dec 15, 2024
ea4b66b
use the compiler in cmake
paulgessinger Dec 15, 2024
a9251b3
handle empty argument
paulgessinger Dec 15, 2024
e897c5b
detect g++
paulgessinger Dec 15, 2024
9b87e4b
b5
paulgessinger Dec 16, 2024
fa0e367
fix opengl detection
paulgessinger Dec 16, 2024
c85073c
fix glob
paulgessinger Dec 16, 2024
c392686
family
paulgessinger Dec 16, 2024
3503bb8
opengl
paulgessinger Dec 16, 2024
e7e95df
wrap script in action
paulgessinger Dec 16, 2024
e61e056
explicit token
paulgessinger Dec 16, 2024
d947b2f
action
paulgessinger Dec 16, 2024
ae1a227
shell
paulgessinger Dec 16, 2024
5204350
use runner os
paulgessinger Dec 16, 2024
8c9cdf9
update tag
paulgessinger Dec 19, 2024
b59c846
dedupe matrix cache keys
paulgessinger Dec 19, 2024
0920ebd
remove remnant
paulgessinger Dec 19, 2024
b9ffa07
see if we still need to supply the python interpreter
paulgessinger Dec 19, 2024
980f8ce
change path ordering
paulgessinger Dec 19, 2024
111b4dc
we need it in macos at least
paulgessinger Dec 19, 2024
f4d3e57
let's try to cache the spack checkout
paulgessinger Dec 19, 2024
cd631d7
yaml error
paulgessinger Dec 19, 2024
dde89b2
compress before installing, rebase
paulgessinger Dec 19, 2024
6c8b4b7
better compression
paulgessinger Dec 19, 2024
94d4318
repack?
paulgessinger Dec 19, 2024
ec20674
actually, set up with update
paulgessinger Dec 19, 2024
7c5ef98
shuffle
paulgessinger Dec 19, 2024
2acd2ab
need to do it after we did the environment
paulgessinger Dec 19, 2024
8b6b883
exclude tarball
paulgessinger Dec 19, 2024
4c0a7bb
drop the caching again
paulgessinger Dec 19, 2024
7e01b2b
path python macos
paulgessinger Dec 20, 2024
49f24af
tmate
paulgessinger Dec 20, 2024
760891f
other tmate
paulgessinger Dec 20, 2024
c18a483
point cmakeprefixpath at venv dir
paulgessinger Dec 20, 2024
9783a07
chain updates
paulgessinger Dec 20, 2024
13a3fe7
system site packages?
paulgessinger Dec 20, 2024
09fb061
install base python packages
paulgessinger Dec 20, 2024
e8b0a9d
tmae macos
paulgessinger Dec 20, 2024
baad99b
add ptyhon find framework directive
paulgessinger Dec 20, 2024
6beabe8
no tmate
paulgessinger Dec 20, 2024
b7250a8
location strategy
paulgessinger Dec 20, 2024
110ecc1
see if policy also works
paulgessinger Dec 20, 2024
841de09
more general CMAKE_FIND_FRAMEWORK
paulgessinger Dec 20, 2024
9122a38
collect all policy settings in one place
paulgessinger Dec 20, 2024
bbe9844
run downstream jobs
paulgessinger Dec 20, 2024
debddb0
update hashes due to G4 version change
paulgessinger Dec 20, 2024
f67fe6a
include more in the cache key
paulgessinger Dec 20, 2024
901b874
try spack caching again
paulgessinger Dec 20, 2024
ae4856f
syntax
paulgessinger Dec 20, 2024
3f19f02
tag fix
paulgessinger Dec 20, 2024
d2dfaff
switch repo, no patches
paulgessinger Dec 20, 2024
976552d
special branch
paulgessinger Dec 20, 2024
eae5a6a
single cache key
paulgessinger Dec 20, 2024
ed30c34
let's keep the .spack directory as well
paulgessinger Dec 20, 2024
6e03ab4
namwes, shuffle
paulgessinger Dec 20, 2024
cdbc1ae
binary path
paulgessinger Dec 20, 2024
25dfad2
no zstd install
paulgessinger Dec 20, 2024
30485e8
key issue?
paulgessinger Dec 20, 2024
3cb1f22
trigger
paulgessinger Dec 20, 2024
65617b6
restore key?
paulgessinger Dec 20, 2024
77f8be8
keys?
paulgessinger Dec 20, 2024
47c1297
debug examples failure
paulgessinger Jan 6, 2025
a127c69
try runner specific to reduce race condition likelihood
paulgessinger Jan 6, 2025
606ecb0
save with correct key
paulgessinger Jan 6, 2025
fb54b01
bump cache action
paulgessinger Jan 7, 2025
8ce198a
bump dependency tag
paulgessinger Jan 7, 2025
76f2c35
set compiler family on default job
paulgessinger Jan 7, 2025
9d26de7
debug root access
paulgessinger Jan 8, 2025
818d407
output
paulgessinger Jan 8, 2025
571978f
use relative path for root setup script
paulgessinger Jan 8, 2025
dd40048
fix compiler in macos
paulgessinger Jan 8, 2025
d13fd0e
fixes
paulgessinger Jan 8, 2025
9708692
disable extra builds
paulgessinger Jan 8, 2025
0aaa5b5
change cache key
paulgessinger Jan 8, 2025
aebf030
missing bin
paulgessinger Jan 8, 2025
2b7eebb
test root import
paulgessinger Jan 8, 2025
9f0d9e4
restore builds
paulgessinger Jan 8, 2025
00f3ede
hopefully fix cache
paulgessinger Jan 8, 2025
0d7ddff
disable GHA builds (to save resources)
paulgessinger Jan 8, 2025
6fccdb8
enable gitlab
paulgessinger Jan 8, 2025
5d5527d
update tag in gitlab
paulgessinger Jan 9, 2025
c93e082
fix ubuntu extra
paulgessinger Jan 9, 2025
92aa7c7
use cern mirror on gitlab ci
paulgessinger Jan 9, 2025
9006f67
disable unrelated jobs for now
paulgessinger Jan 9, 2025
a64c80b
reduce ccache size
paulgessinger Jan 9, 2025
ec2c95a
add caching feature to lock file getter
paulgessinger Jan 9, 2025
51f0c5b
let's try caching
paulgessinger Jan 9, 2025
7b20950
cache fix
paulgessinger Jan 9, 2025
d775a42
cache config
paulgessinger Jan 9, 2025
b1370d0
fix compiler to gcc
paulgessinger Jan 9, 2025
b83cc3e
change cwd for deps install
paulgessinger Jan 9, 2025
6f86c5f
add a global digest for the cache content
paulgessinger Jan 9, 2025
a33c62c
see if lockfile caching works
paulgessinger Jan 9, 2025
cabe566
path
paulgessinger Jan 9, 2025
c88aedc
update paths for caching
paulgessinger Jan 9, 2025
83900e0
prune spack directory before caching
paulgessinger Jan 9, 2025
cc65f71
more pruning
paulgessinger Jan 9, 2025
d498f9d
only add mirror if not already there
paulgessinger Jan 9, 2025
5e121a8
delete the git folder too
paulgessinger Jan 9, 2025
b592344
enable build
paulgessinger Jan 9, 2025
d311fcc
use ccache again
paulgessinger Jan 9, 2025
e4ef4a6
reenable some jobs
paulgessinger Jan 10, 2025
d0667a3
Merge branch 'main' into spack-ci
paulgessinger Jan 10, 2025
6b36f01
add spack cache, cleanup to other jobs
paulgessinger Jan 10, 2025
81e1191
fix syntax
paulgessinger Jan 10, 2025
ac99ab4
and another one
paulgessinger Jan 10, 2025
a002ddd
another one
paulgessinger Jan 10, 2025
f887058
lcg doesn't need spack
paulgessinger Jan 10, 2025
7f41271
fix for downstream execeution breaking
paulgessinger Jan 10, 2025
5600c5e
increase FPE stack trace depth
paulgessinger Jan 11, 2025
46dcbec
Merge branch 'main' into spack-ci
paulgessinger Jan 11, 2025
904336d
try to mask G4 FPE
paulgessinger Jan 12, 2025
6cb9639
restore ccache cache size
paulgessinger Jan 12, 2025
fd25cb8
wider mask range
paulgessinger Jan 12, 2025
61479f3
cleanup, re-add clang-tidy, explicit compilers
paulgessinger Jan 12, 2025
4a08350
add analysis.yml to debug
paulgessinger Jan 12, 2025
6406e3a
bump image for ubuntu clang
paulgessinger Jan 12, 2025
c14e67e
missing dependency tag
paulgessinger Jan 12, 2025
2a90a86
set to correct compiler in build
paulgessinger Jan 13, 2025
9ffcf92
limit pytest to failing test for debug
paulgessinger Jan 13, 2025
a52bcc0
ignore find failure
paulgessinger Jan 13, 2025
5cce4e8
mask two more FPEs
paulgessinger Jan 13, 2025
956db09
remove DEPENDENCY_URL, add spack cache to clang-tidy job
paulgessinger Jan 13, 2025
68f97e5
Merge branch 'main' into spack-ci
paulgessinger Jan 13, 2025
a1b2dd1
Revert "increase FPE stack trace depth"
paulgessinger Jan 13, 2025
3751f6e
add other github jobs back in
paulgessinger Jan 13, 2025
57e5bc5
try to make spack location independent of current directory
paulgessinger Jan 13, 2025
a37a5b9
github: give spack clone path
paulgessinger Jan 13, 2025
6fce372
path fix
paulgessinger Jan 13, 2025
1c59936
add tests and downstream build to main ubuntu job, run tests single t…
paulgessinger Jan 14, 2025
aab69b8
remove unneeded compiler setting
paulgessinger Jan 14, 2025
592c74c
pipe install log to file
paulgessinger Jan 14, 2025
84ecb53
fix
paulgessinger Jan 14, 2025
6a348f3
follow up
paulgessinger Jan 15, 2025
f3f74cd
Merge branch 'main' into spack-ci
paulgessinger Jan 15, 2025
7b9402c
Merge branch 'main' into spack-ci
paulgessinger Jan 16, 2025
fe3f516
bump images
paulgessinger Jan 17, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions .github/actions/dependencies/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
name: 'Dependencies'
description: 'Install dependencies from lockfile'

inputs:
compiler:
description: 'Compiler to use'
required: false
default: ''
GITHUB_TOKEN:
description: 'Secrets to pass to the action'
required: false
default: ''
spack_version:
description: 'Version of Spack to use'
required: false
default: 'develop-acts'

runs:
using: 'composite'
steps:
- name: Prerequisites
if: startsWith(runner.os, 'macos')
shell: bash
run: |
brew install ninja ccache

paulgessinger marked this conversation as resolved.
Show resolved Hide resolved
- name: Restore cache
id: cache-restore
uses: actions/cache/restore@v4
with:
path: spack_cache
key: spack-r2-${{ inputs.spack_version }}-${{ hashFiles('CI/dependencies/setup_spack.sh') }}

- name: Extract cached Spack
if: steps.cache-restore.outputs.cache-hit == 'true'
shell: bash
run: |
mv spack_cache/spack .
mv spack_cache/.spack ~

- name: Setup Spack from scratch
if: steps.cache-restore.outputs.cache-hit != 'true'
shell: bash
env:
SPACK_VERSION: ${{ inputs.spack_version }}
run: |
CI/dependencies/setup_spack.sh ${PWD}/spack
spack/bin/spack list > /dev/null
rm -rf spack/.git
mkdir spack_cache
cp -r spack spack_cache/
cp -r ~/.spack spack_cache/

- name: Save cache
if: steps.cache-restore.outputs.cache-hit != 'true'
uses: actions/cache/save@v4
with:
path: |
spack_cache
key: ${{ steps.cache-restore.outputs.cache-primary-key }}

- name: Install dependencies
shell: bash
env:
GITHUB_TOKEN: ${{ inputs.GITHUB_TOKEN }}
run: |
if [ -n "${{ inputs.compiler }}" ]; then
echo "With compiler"
CI/dependencies/setup.sh -c "${{ inputs.compiler }}"
else
echo "Without compiler"
CI/dependencies/setup.sh
fi
15 changes: 10 additions & 5 deletions .github/workflows/analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ env:
CCACHE_MAXSIZE: 1.25G
CCACHE_KEY_SUFFIX: r2
ACTS_LOG_FAILURE_THRESHOLD: WARNING
DEPENDENCY_URL: https://acts.web.cern.ch/ACTS/ci/ubuntu-24.04/deps.v5.tar.zst
DEPENDENCY_TAG: v7_b7

# NOTE this only builds core unittests to reduce the output size. if we
# found a way to have Github actions not fail regularly with this job
Expand All @@ -36,7 +36,10 @@ jobs:
- uses: actions/checkout@v4

- name: Install dependencies
run: CI/dependencies.sh
uses: ./.github/actions/dependencies
with:
compiler: g++
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Cache build
uses: actions/cache@v4
Expand All @@ -53,7 +56,6 @@ jobs:
--preset=github-ci
-DCMAKE_BUILD_TYPE=Debug
-DCMAKE_CXX_FLAGS="-Werror --coverage -g -gz -g1"
-DPython_EXECUTABLE=$(which python3)
-DACTS_BUILD_ODD=OFF
- name: Build
run: cmake --build build
Expand Down Expand Up @@ -92,12 +94,15 @@ jobs:
build_performance:
runs-on: ubuntu-latest
container: ghcr.io/acts-project/ubuntu2404:63
if: github.ref == 'refs/heads/main'
# if: github.ref == 'refs/heads/main'
paulgessinger marked this conversation as resolved.
Show resolved Hide resolved
steps:
- uses: actions/checkout@v4

- name: Install dependencies
run: CI/dependencies.sh
uses: ./.github/actions/dependencies
with:
compiler: g++
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Install dependencies
run: pip3 install git+https://github.com/paulgessinger/cmakeperf.git@2a409b5
Expand Down
Loading
Loading