Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: CDash dashboard support #5566

Open
wants to merge 5 commits into
base: development
Choose a base branch
from
Open

CI: CDash dashboard support #5566

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
8017fac
CDash dashboard support
EZoni Jan 15, 2025
eb7f107
Simplify
ax3l Jan 15, 2025
6adb907
Update Azure workflow file
EZoni Jan 16, 2025
49df5c8
Add site and build names
EZoni Jan 16, 2025
3a4867f
Run tests and submit to CDash as `Experimental`
EZoni Jan 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions .azure-pipelines.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,8 +155,15 @@ jobs:
- bash: |
# set options
set -o nounset errexit pipefail
# run tests (exclude pytest.AMReX when running Python tests)
ctest --test-dir build --output-on-failure -E AMReX
# determine if the build was triggered by a push to the development branch
if [[ "$(Build.SourceBranch)" == "refs/heads/development" ]]; then
Copy link
Member Author

@EZoni EZoni Jan 17, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this does not work (but we'll know only after merging the PR into development), we can also try the following (see https://learn.microsoft.com/en-us/azure/devops/pipelines/build/variables for more documentation):

Suggested change
if [[ "$(Build.SourceBranch)" == "refs/heads/development" ]]; then
if [[ "$(Build.Reason)" == "IndividualCI" ]]; then

# run tests (exclude pytest.AMReX when running Python tests)
# and submit results to CDash as Continuous
ctest --test-dir build --output-on-failure -E AMReX -D Continuous
else
# run tests (exclude pytest.AMReX when running Python tests)
ctest --test-dir build --output-on-failure -E AMReX
fi
Copy link
Member Author

@EZoni EZoni Jan 16, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what I would try first, based on the information reported in #5566 (comment).

Update
This is probably duplicating the configure and build step. Looking more into how to avoid duplication.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again on this, and on the issues reported in #5566 (comment), reading the documentation more carefully it seems to me that the various "steps" (e.g., Start, Update, Configure, Build, Test, MemoryCheck, Coverage, Submit) can be run independently only for the Nightly or Experimental modes, but not for the other modes such as Continuous. This could explain what we observed so far.

Screenshot from 2025-01-17 11-09-18

Copy link
Member Author

@EZoni EZoni Jan 17, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Continuous still seems like the right choice for main branch events, such as merging PRs into the main branch, from a theoretical point of view. But, as discussed, we could start with Experimental and fine tune things later on, once we have gained more experience with CDash overall.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# determine if the build was triggered by a push to the development branch
if [[ "$(Build.SourceBranch)" == "refs/heads/development" ]]; then
# run tests (exclude pytest.AMReX when running Python tests)
# and submit results to CDash as Continuous
ctest --test-dir build --output-on-failure -E AMReX -D Continuous
else
# run tests (exclude pytest.AMReX when running Python tests)
ctest --test-dir build --output-on-failure -E AMReX
fi
# determine if the build was triggered by a push to the development branch
if [[ "$(Build.SourceBranch)" == "refs/heads/development" ]]; then
# run tests (exclude pytest.AMReX when running Python tests)
# and submit results to CDash as Experimental
ctest --test-dir build --output-on-failure -E AMReX \
-D ExperimentalTest -D ExperimentalSubmit
else
# run tests (exclude pytest.AMReX when running Python tests)
ctest --test-dir build --output-on-failure -E AMReX
fi

displayName: 'Test'

- bash: |
Expand Down
3 changes: 2 additions & 1 deletion CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,8 @@ if(WarpX_FFT)
set(ABLASTR_FFT ON CACHE STRING "FFT-based solvers" FORCE)
endif()

# this defined the variable BUILD_TESTING which is ON by default
# Define the variable BUILD_TESTING (ON by default),
# include CDash dashboard testing module
include(CTest)


Expand Down
14 changes: 14 additions & 0 deletions CTestConfig.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
## This file should be placed in the root directory of your project.
## Then modify the CMakeLists.txt file in the root directory of your
## project to incorporate the testing dashboard.
##
## # The following are required to submit to the CDash dashboard:
## ENABLE_TESTING()
## INCLUDE(CTest)

set(CTEST_PROJECT_NAME WarpX)
set(CTEST_NIGHTLY_START_TIME 08:00:00 UTC)

set(CTEST_SUBMIT_URL https://my.cdash.org/submit.php?project=WarpX)

set(CTEST_DROP_SITE_CDASH TRUE)
Loading