Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test grouping to workflow inputs #1029

Merged
merged 1 commit into from
Jan 14, 2025
Merged

Add test grouping to workflow inputs #1029

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 22 additions & 3 deletions .github/workflows/build-and-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,16 @@ on:
description: 'Git SHA of commit in tenstorrent/tt-mlir or branch name'
required: false
type: string
test_group_cnt:
description: 'Test group count'
required: true
default: 2
type: number
test_group_ids:
description: 'Test group ids'
required: true
default: '[1,2]'
type: string
workflow_call:
inputs:
test_mark:
Expand All @@ -28,14 +38,23 @@ on:
description: 'Git SHA of commit in tenstorrent/tt-mlir or branch name'
required: false
type: string
test_group_cnt:
description: 'Test group count'
required: false
default: 2
type: number
test_group_ids:
description: 'Test group ids'
required: false
default: '[1,2]'
type: string

permissions:
packages: write
checks: write
pull-requests: write # only required if `comment: true` was enabled

jobs:

docker-build:
uses: ./.github/workflows/build-image.yml
secrets: inherit
Expand All @@ -48,7 +67,7 @@ jobs:
matrix:
build:
- runs-on: runner
test_group_id: [1,2]
test_group_id: ${{ fromJson(inputs.test_group_ids) }}

runs-on:
- in-service
Expand Down Expand Up @@ -152,7 +171,7 @@ jobs:
apt install -y libgl1 libglx-mesa0
set -o pipefail # Ensures that the exit code reflects the first command that fails
pip install pytest-split
pytest -m push --splits 2 \
pytest -m push --splits ${{ inputs.test_group_cnt }} \
--group ${{ matrix.test_group_id }} \
--splitting-algorithm least_duration \
-m "${{ inputs.test_mark }}" \
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/on-nightly-sweeps.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,5 @@ jobs:
secrets: inherit
with:
test_mark: 'nightly_sweeps'
test_group_cnt: 4
test_group_ids: '[1,2,3,4]'
2 changes: 2 additions & 0 deletions .github/workflows/on-nightly.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,5 @@ jobs:
secrets: inherit
with:
test_mark: 'nightly'
test_group_cnt: 4
test_group_ids: '[1,2,3,4]'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to specify IDs? Is it possible for this to be automatically picked?

Copy link
Collaborator Author

@vmilosevic vmilosevic Jan 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, unfortunately the functions you can use in gihub context are very limited (contains, startsWith, fromJSON, toJSON ..). You can't even get the length of an array.

A way this could be done is to add one more job that can do more complicated things in bash and output the list but I thought this complicates workflow unnecessarily and gone with simpler solution.

You also need to provide matrix values before job is scheduled, so you can't do any manipulation in same job, it needs to be a job that run before.

2 changes: 2 additions & 0 deletions .github/workflows/on-pr.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ jobs:
secrets: inherit
with:
test_mark: 'push'
test_group_cnt: 2
test_group_ids: '[1,2]'
perf-benchmark:
needs: docker-build
uses: ./.github/workflows/perf-benchmark.yml
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/on-push.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,5 @@ jobs:
secrets: inherit
with:
test_mark: 'push'
test_group_cnt: 2
test_group_ids: '[1,2]'
Loading