Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MGMT-19258: Split KubeAPI and non KubeAPI subsystem tests #7154

Merged
merged 1 commit into from
Jan 6, 2025

Conversation

paul-maidment
Copy link
Contributor

@paul-maidment paul-maidment commented Dec 31, 2024

We are dealing with an issue where some tests have not been executed since they were created, this is because they lack the [kube-api] tag in the test name. It's not ideal.

I have decided to solve this by placing the KubeAPI and non KubeAPI tests into different packages

github.com/openshift/assisted-service/subsystem for non KubeAPI tests
github.com/openshift/assisted-service/subsystem/kubeapi for KubeAPI tests

Now every test within the respective package will be executed, irrespective of how it has been tagged.

In order to get to this point, some refactoring of common code between tests was required. The common code has been placed in a package called utils_test.

Each of the test suites has been manually executed and confirmed to be working after these changes. KubeAPI tests now run without the need for a [kube-api] label.

Some kube-api tests that never executed before this fix were found to be failing after they started to run.

These tests have been disabled by marking them with PDescribe for now as they have been broken for some time and are non-trivial to fix.

These tests will be dealt with in MGMT-19596

List all the issues related to this PR

  • New Feature
  • Enhancement
  • Bug fix
  • Tests
  • Documentation
  • CI/CD

What environments does this code impact?

  • Automation (CI, tools, etc)
  • Cloud
  • Operator Managed Deployments
  • None

How was this code tested?

  • assisted-test-infra environment
  • dev-scripts environment
  • Reviewer's test appreciated
  • Waiting for CI to do a full test run
  • Manual (Elaborate on how it was tested)
  • No tests needed

Checklist

  • Title and description added to both, commit and PR.
  • Relevant issues have been associated (see CONTRIBUTING guide)
  • This change does not require a documentation update (docstring, docs, README, etc)
  • Does this change include unit-tests (note that code changes require unit-tests)

Reviewers Checklist

  • Are the title and description (in both PR and commit) meaningful and clear?
  • Is there a bug required (and linked) for this change?
  • Should this PR be backported?

Copy link

openshift-ci bot commented Dec 31, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci-robot
Copy link

openshift-ci-robot commented Dec 31, 2024

@paul-maidment: This pull request references MGMT-19258 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.19.0" version, but no target version was set.

In response to this:

We are dealing with an issue where some tests have not been executed since they were created, this is because they lack the [kube-api] tag in the test name. It's not ideal to need to specify this tag in each test as it is easily missed and the consequences of this are not good.

I have decided to solve this by placing the KubeAPI and non KubeAPI tests into different packages

github.com/openshift/assisted-service/subsystem for non KubeAPI tests 'github.com/openshift/assisted-service/subsystem/kubeapi' for KubeAPI tests

Now every test within the respective package will be executed, irrespective of how it has been tagged.

In order to get to this point, some refactoring of common code between tests was required. The common code has been placed in a package called utils_test Some of the existing utils code has been moved here also in order to support sharing of this code.

This is quite a big change in terms of volume, but the principles are simple. functions, consts and vars between tests have been moved to the new namespace and exported. References to package private members that have been moved to the new namespace have been updated to point to the correct place.

Some tests were found to be failing after they started to run, not unexpected. As these are non trivial to fix and somewhat unrelated to this fix for how we run the tests, these tests have been disabled by marking them with PDescribe for now. These tests will be dealt with in MGMT-19596

List all the issues related to this PR

  • New Feature
  • Enhancement
  • Bug fix
  • Tests
  • Documentation
  • CI/CD

What environments does this code impact?

  • Automation (CI, tools, etc)
  • Cloud
  • Operator Managed Deployments
  • None

How was this code tested?

  • assisted-test-infra environment
  • dev-scripts environment
  • Reviewer's test appreciated
  • Waiting for CI to do a full test run
  • Manual (Elaborate on how it was tested)
  • No tests needed

Checklist

  • Title and description added to both, commit and PR.
  • Relevant issues have been associated (see CONTRIBUTING guide)
  • This change does not require a documentation update (docstring, docs, README, etc)
  • Does this change include unit-tests (note that code changes require unit-tests)

Reviewers Checklist

  • Are the title and description (in both PR and commit) meaningful and clear?
  • Is there a bug required (and linked) for this change?
  • Should this PR be backported?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 31, 2024
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Dec 31, 2024
@openshift-ci-robot
Copy link

openshift-ci-robot commented Dec 31, 2024

@paul-maidment: This pull request references MGMT-19258 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.19.0" version, but no target version was set.

In response to this:

We are dealing with an issue where some tests have not been executed since they were created, this is because they lack the [kube-api] tag in the test name. It's not ideal to need to specify this tag in each test as it is easily missed and the consequences of this are not good.

I have decided to solve this by placing the KubeAPI and non KubeAPI tests into different packages

github.com/openshift/assisted-service/subsystem for non KubeAPI tests 'github.com/openshift/assisted-service/subsystem/kubeapi' for KubeAPI tests

Now every test within the respective package will be executed, irrespective of how it has been tagged.

In order to get to this point, some refactoring of common code between tests was required. The common code has been placed in a package called utils_test Some of the existing utils code has been moved here also in order to support sharing of this code.

This is quite a big change in terms of volume, but the principles are simple. functions, consts and vars between tests have been moved to the new namespace and exported. References to package private members that have been moved to the new namespace have been updated to point to the correct place.

Some tests were found to be failing after they started to run, not unexpected. As these are non trivial to fix and somewhat unrelated to this fix for how we run the tests, these tests have been disabled by marking them with PDescribe for now. These tests will be dealt with in MGMT-19596

List all the issues related to this PR

  • New Feature
  • Enhancement
  • Bug fix
  • Tests
  • Documentation
  • CI/CD

What environments does this code impact?

  • Automation (CI, tools, etc)
  • Cloud
  • Operator Managed Deployments
  • None

How was this code tested?

  • assisted-test-infra environment
  • dev-scripts environment
  • Reviewer's test appreciated
  • Waiting for CI to do a full test run
  • Manual (Elaborate on how it was tested)
  • No tests needed

Checklist

  • Title and description added to both, commit and PR.
  • Relevant issues have been associated (see CONTRIBUTING guide)
  • This change does not require a documentation update (docstring, docs, README, etc)
  • Does this change include unit-tests (note that code changes require unit-tests)

Reviewers Checklist

  • Are the title and description (in both PR and commit) meaningful and clear?
  • Is there a bug required (and linked) for this change?
  • Should this PR be backported?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Dec 31, 2024
@openshift-ci-robot
Copy link

openshift-ci-robot commented Dec 31, 2024

@paul-maidment: This pull request references MGMT-19258 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.19.0" version, but no target version was set.

In response to this:

We are dealing with an issue where some tests have not been executed since they were created, this is because they lack the [kube-api] tag in the test name. It's not ideal to need to specify this tag in each test as it is easily missed and the consequences of this are not good.

I have decided to solve this by placing the KubeAPI and non KubeAPI tests into different packages

github.com/openshift/assisted-service/subsystem for non KubeAPI tests github.com/openshift/assisted-service/subsystem/kubeapi for KubeAPI tests

Now every test within the respective package will be executed, irrespective of how it has been tagged.

In order to get to this point, some refactoring of common code between tests was required. The common code has been placed in a package called utils_test Some of the existing utils code has been moved here also in order to support sharing of this code.

This is quite a big change in terms of volume, but the principles are simple. functions, consts and vars between tests have been moved to the new namespace and exported. References to package private members that have been moved to the new namespace have been updated to point to the correct place.

Some tests were found to be failing after they started to run, not unexpected. As these are non trivial to fix and somewhat unrelated to this fix for how we run the tests, these tests have been disabled by marking them with PDescribe for now. These tests will be dealt with in MGMT-19596

List all the issues related to this PR

  • New Feature
  • Enhancement
  • Bug fix
  • Tests
  • Documentation
  • CI/CD

What environments does this code impact?

  • Automation (CI, tools, etc)
  • Cloud
  • Operator Managed Deployments
  • None

How was this code tested?

  • assisted-test-infra environment
  • dev-scripts environment
  • Reviewer's test appreciated
  • Waiting for CI to do a full test run
  • Manual (Elaborate on how it was tested)
  • No tests needed

Checklist

  • Title and description added to both, commit and PR.
  • Relevant issues have been associated (see CONTRIBUTING guide)
  • This change does not require a documentation update (docstring, docs, README, etc)
  • Does this change include unit-tests (note that code changes require unit-tests)

Reviewers Checklist

  • Are the title and description (in both PR and commit) meaningful and clear?
  • Is there a bug required (and linked) for this change?
  • Should this PR be backported?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Dec 31, 2024

@paul-maidment: This pull request references MGMT-19258 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.19.0" version, but no target version was set.

In response to this:

We are dealing with an issue where some tests have not been executed since they were created, this is because they lack the [kube-api] tag in the test name. It's not ideal to need to specify this tag in each test as it is easily missed and the consequences of this are not good.

I have decided to solve this by placing the KubeAPI and non KubeAPI tests into different packages

github.com/openshift/assisted-service/subsystem for non KubeAPI tests github.com/openshift/assisted-service/subsystem/kubeapi for KubeAPI tests

Now every test within the respective package will be executed, irrespective of how it has been tagged.

In order to get to this point, some refactoring of common code between tests was required. The common code has been placed in a package called utils_test Some of the existing utils code has been moved here also in order to support sharing of this code.

This is quite a big change in terms of volume, but the principles are simple. functions, consts and vars between tests have been moved to the new package and exported. References to package private members that have been moved to the new package have been updated to point to the correct place.

Some tests were found to be failing after they started to run, not unexpected. As these are non trivial to fix and somewhat unrelated to this fix for how we run the tests, these tests have been disabled by marking them with PDescribe for now. These tests will be dealt with in MGMT-19596

List all the issues related to this PR

  • New Feature
  • Enhancement
  • Bug fix
  • Tests
  • Documentation
  • CI/CD

What environments does this code impact?

  • Automation (CI, tools, etc)
  • Cloud
  • Operator Managed Deployments
  • None

How was this code tested?

  • assisted-test-infra environment
  • dev-scripts environment
  • Reviewer's test appreciated
  • Waiting for CI to do a full test run
  • Manual (Elaborate on how it was tested)
  • No tests needed

Checklist

  • Title and description added to both, commit and PR.
  • Relevant issues have been associated (see CONTRIBUTING guide)
  • This change does not require a documentation update (docstring, docs, README, etc)
  • Does this change include unit-tests (note that code changes require unit-tests)

Reviewers Checklist

  • Are the title and description (in both PR and commit) meaningful and clear?
  • Is there a bug required (and linked) for this change?
  • Should this PR be backported?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Dec 31, 2024

@paul-maidment: This pull request references MGMT-19258 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.19.0" version, but no target version was set.

In response to this:

We are dealing with an issue where some tests have not been executed since they were created, this is because they lack the [kube-api] tag in the test name. It's not ideal to need to specify this tag in each test as it is easily missed and the consequences of this are not good.

I have decided to solve this by placing the KubeAPI and non KubeAPI tests into different packages

github.com/openshift/assisted-service/subsystem for non KubeAPI tests github.com/openshift/assisted-service/subsystem/kubeapi for KubeAPI tests

Now every test within the respective package will be executed, irrespective of how it has been tagged.

In order to get to this point, some refactoring of common code between tests was required. The common code has been placed in a package called utils_test Some of the existing utils code has been moved here also in order to support sharing of this code.

This is quite a big change in terms of volume, but the principles are simple. functions, consts and vars between tests have been moved to the new package and exported. References to package private members that have been moved to the new package have been updated to point to the correct place.

Some kube-api tests that never executed before this fix were found to be failing after they started to run.

This is not unexpected.

As these are non trivial to fix and somewhat unrelated to the fix to how we run the tests, these tests have been disabled by marking them with PDescribe for now.

These tests will be dealt with in MGMT-19596

List all the issues related to this PR

  • New Feature
  • Enhancement
  • Bug fix
  • Tests
  • Documentation
  • CI/CD

What environments does this code impact?

  • Automation (CI, tools, etc)
  • Cloud
  • Operator Managed Deployments
  • None

How was this code tested?

  • assisted-test-infra environment
  • dev-scripts environment
  • Reviewer's test appreciated
  • Waiting for CI to do a full test run
  • Manual (Elaborate on how it was tested)
  • No tests needed

Checklist

  • Title and description added to both, commit and PR.
  • Relevant issues have been associated (see CONTRIBUTING guide)
  • This change does not require a documentation update (docstring, docs, README, etc)
  • Does this change include unit-tests (note that code changes require unit-tests)

Reviewers Checklist

  • Are the title and description (in both PR and commit) meaningful and clear?
  • Is there a bug required (and linked) for this change?
  • Should this PR be backported?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Dec 31, 2024

@paul-maidment: This pull request references MGMT-19258 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.19.0" version, but no target version was set.

In response to this:

We are dealing with an issue where some tests have not been executed since they were created, this is because they lack the [kube-api] tag in the test name. It's not ideal>

I have decided to solve this by placing the KubeAPI and non KubeAPI tests into different packages

github.com/openshift/assisted-service/subsystem for non KubeAPI tests github.com/openshift/assisted-service/subsystem/kubeapi for KubeAPI tests

Now every test within the respective package will be executed, irrespective of how it has been tagged.

In order to get to this point, some refactoring of common code between tests was required. The common code has been placed in a package called utils_test Some of the exist>

This is quite a big change in terms of volume, but the principles are simple. functions, consts and vars between tests have been moved to the new package and exported. Ref>

Each of the test suites has been manually executed and confirmed to be working after these changes. KubeAPI tests now run without the need for a [kube-api] label.

Some kube-api tests that never executed before this fix were found to be failing after they started to run.

This is not unexpected.

As these are non trivial to fix and somewhat unrelated to the fix to how we run the tests, these tests have been disabled by marking them with PDescribe for now.

These tests will be dealt with in MGMT-19596

List all the issues related to this PR

  • New Feature
  • Enhancement
  • Bug fix
  • Tests
  • Documentation
  • CI/CD

What environments does this code impact?

  • Automation (CI, tools, etc)
  • Cloud
  • Operator Managed Deployments
  • None

How was this code tested?

  • assisted-test-infra environment
  • dev-scripts environment
  • Reviewer's test appreciated
  • Waiting for CI to do a full test run
  • Manual (Elaborate on how it was tested)
  • No tests needed

Checklist

  • Title and description added to both, commit and PR.
  • Relevant issues have been associated (see CONTRIBUTING guide)
  • This change does not require a documentation update (docstring, docs, README, etc)
  • Does this change include unit-tests (note that code changes require unit-tests)

Reviewers Checklist

  • Are the title and description (in both PR and commit) meaningful and clear?
  • Is there a bug required (and linked) for this change?
  • Should this PR be backported?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@paul-maidment paul-maidment marked this pull request as ready for review December 31, 2024 19:17
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 31, 2024
@openshift-ci-robot
Copy link

openshift-ci-robot commented Dec 31, 2024

@paul-maidment: This pull request references MGMT-19258 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.19.0" version, but no target version was set.

In response to this:

We are dealing with an issue where some tests have not been executed since they were created, this is because they lack the [kube-api] tag in the test name. It's not ideal.

I have decided to solve this by placing the KubeAPI and non KubeAPI tests into different packages

github.com/openshift/assisted-service/subsystem for non KubeAPI tests github.com/openshift/assisted-service/subsystem/kubeapi for KubeAPI tests

Now every test within the respective package will be executed, irrespective of how it has been tagged.

In order to get to this point, some refactoring of common code between tests was required. The common code has been placed in a package called utils_test Some of the exist

This is quite a big change in terms of volume, but the principles are simple. functions, consts and vars between tests have been moved to the new package and exported.

Each of the test suites has been manually executed and confirmed to be working after these changes. KubeAPI tests now run without the need for a [kube-api] label.

Some kube-api tests that never executed before this fix were found to be failing after they started to run.

This is not unexpected.

As these are non trivial to fix and somewhat unrelated to the fix to how we run the tests, these tests have been disabled by marking them with PDescribe for now.

These tests will be dealt with in MGMT-19596

List all the issues related to this PR

  • New Feature
  • Enhancement
  • Bug fix
  • Tests
  • Documentation
  • CI/CD

What environments does this code impact?

  • Automation (CI, tools, etc)
  • Cloud
  • Operator Managed Deployments
  • None

How was this code tested?

  • assisted-test-infra environment
  • dev-scripts environment
  • Reviewer's test appreciated
  • Waiting for CI to do a full test run
  • Manual (Elaborate on how it was tested)
  • No tests needed

Checklist

  • Title and description added to both, commit and PR.
  • Relevant issues have been associated (see CONTRIBUTING guide)
  • This change does not require a documentation update (docstring, docs, README, etc)
  • Does this change include unit-tests (note that code changes require unit-tests)

Reviewers Checklist

  • Are the title and description (in both PR and commit) meaningful and clear?
  • Is there a bug required (and linked) for this change?
  • Should this PR be backported?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@paul-maidment paul-maidment marked this pull request as draft December 31, 2024 19:22
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 31, 2024
@paul-maidment paul-maidment marked this pull request as ready for review December 31, 2024 19:41
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 31, 2024
Copy link

codecov bot commented Dec 31, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 68.06%. Comparing base (6c35570) to head (eba0446).
Report is 6 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #7154      +/-   ##
==========================================
+ Coverage   67.60%   68.06%   +0.45%     
==========================================
  Files         296      296              
  Lines       40266    40619     +353     
==========================================
+ Hits        27223    27646     +423     
+ Misses      10590    10512      -78     
- Partials     2453     2461       +8     

see 5 files with indirect coverage changes

@paul-maidment paul-maidment marked this pull request as draft December 31, 2024 21:00
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 31, 2024
@openshift-ci openshift-ci bot added the api-review Categorizes an issue or PR as actively needing an API review. label Jan 1, 2025
@paul-maidment paul-maidment force-pushed the MGMT-19258 branch 3 times, most recently from 8930f91 to 1dce038 Compare January 1, 2025 12:27
@paul-maidment paul-maidment force-pushed the MGMT-19258 branch 2 times, most recently from 0ce3c54 to 2dddd54 Compare January 4, 2025 17:30
@paul-maidment paul-maidment marked this pull request as ready for review January 4, 2025 23:42
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 4, 2025
@openshift-ci openshift-ci bot requested review from gamli75 and omertuc January 4, 2025 23:43
@openshift-ci-robot
Copy link

openshift-ci-robot commented Jan 5, 2025

@paul-maidment: This pull request references MGMT-19258 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.19.0" version, but no target version was set.

In response to this:

We are dealing with an issue where some tests have not been executed since they were created, this is because they lack the [kube-api] tag in the test name. It's not ideal.

I have decided to solve this by placing the KubeAPI and non KubeAPI tests into different packages

github.com/openshift/assisted-service/subsystem for non KubeAPI tests
github.com/openshift/assisted-service/subsystem/kubeapi for KubeAPI tests

Now every test within the respective package will be executed, irrespective of how it has been tagged.

In order to get to this point, some refactoring of common code between tests was required. The common code has been placed in a package called utils_test.

Each of the test suites has been manually executed and confirmed to be working after these changes. KubeAPI tests now run without the need for a [kube-api] label.

Some kube-api tests that never executed before this fix were found to be failing after they started to run.

These tests have been disabled by marking them with PDescribe for now as they have been broken for some time and are non-trivial to fix.

These tests will be dealt with in MGMT-19596

List all the issues related to this PR

  • New Feature
  • Enhancement
  • Bug fix
  • Tests
  • Documentation
  • CI/CD

What environments does this code impact?

  • Automation (CI, tools, etc)
  • Cloud
  • Operator Managed Deployments
  • None

How was this code tested?

  • assisted-test-infra environment
  • dev-scripts environment
  • Reviewer's test appreciated
  • Waiting for CI to do a full test run
  • Manual (Elaborate on how it was tested)
  • No tests needed

Checklist

  • Title and description added to both, commit and PR.
  • Relevant issues have been associated (see CONTRIBUTING guide)
  • This change does not require a documentation update (docstring, docs, README, etc)
  • Does this change include unit-tests (note that code changes require unit-tests)

Reviewers Checklist

  • Are the title and description (in both PR and commit) meaningful and clear?
  • Is there a bug required (and linked) for this change?
  • Should this PR be backported?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

1 similar comment
@openshift-ci-robot
Copy link

openshift-ci-robot commented Jan 5, 2025

@paul-maidment: This pull request references MGMT-19258 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.19.0" version, but no target version was set.

In response to this:

We are dealing with an issue where some tests have not been executed since they were created, this is because they lack the [kube-api] tag in the test name. It's not ideal.

I have decided to solve this by placing the KubeAPI and non KubeAPI tests into different packages

github.com/openshift/assisted-service/subsystem for non KubeAPI tests
github.com/openshift/assisted-service/subsystem/kubeapi for KubeAPI tests

Now every test within the respective package will be executed, irrespective of how it has been tagged.

In order to get to this point, some refactoring of common code between tests was required. The common code has been placed in a package called utils_test.

Each of the test suites has been manually executed and confirmed to be working after these changes. KubeAPI tests now run without the need for a [kube-api] label.

Some kube-api tests that never executed before this fix were found to be failing after they started to run.

These tests have been disabled by marking them with PDescribe for now as they have been broken for some time and are non-trivial to fix.

These tests will be dealt with in MGMT-19596

List all the issues related to this PR

  • New Feature
  • Enhancement
  • Bug fix
  • Tests
  • Documentation
  • CI/CD

What environments does this code impact?

  • Automation (CI, tools, etc)
  • Cloud
  • Operator Managed Deployments
  • None

How was this code tested?

  • assisted-test-infra environment
  • dev-scripts environment
  • Reviewer's test appreciated
  • Waiting for CI to do a full test run
  • Manual (Elaborate on how it was tested)
  • No tests needed

Checklist

  • Title and description added to both, commit and PR.
  • Relevant issues have been associated (see CONTRIBUTING guide)
  • This change does not require a documentation update (docstring, docs, README, etc)
  • Does this change include unit-tests (note that code changes require unit-tests)

Reviewers Checklist

  • Are the title and description (in both PR and commit) meaningful and clear?
  • Is there a bug required (and linked) for this change?
  • Should this PR be backported?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

We are dealing with an issue where some tests have not been executed since they were created, this is because they lack the [kube-api] tag in the test name. It's not ideal.

I have decided to solve this by placing the KubeAPI and non KubeAPI tests into different packages

github.com/openshift/assisted-service/subsystem for non KubeAPI tests
github.com/openshift/assisted-service/subsystem/kubeapi for KubeAPI tests

Now every test within the respective package will be executed, irrespective of how it has been tagged.

In order to get to this point, some refactoring of common code between tests was required. The common code has been placed in a package called utils_test.

Each of the test suites has been manually executed and confirmed to be working after these changes. KubeAPI tests now run without the need for a [kube-api] label.

Some kube-api tests that never executed before this fix were found to be failing after they started to run.

These tests have been disabled by marking them with PDescribe for now as they have been broken for some time and are non-trivial to fix.

These tests will be dealt with in MGMT-19596
@paul-maidment
Copy link
Contributor Author

/test edge-e2e-ai-operator-disconnected-capi

@paul-maidment
Copy link
Contributor Author

/retest-required


test-kube-api:
$(MAKE) _run_subsystem_test AUTH_TYPE=local FOCUS="$(or ${FOCUS},kube-api)"
$(MAKE) _run_subsystem_test AUTH_TYPE=local TEST="$(or $(TEST),'github.com/openshift/assisted-service/subsystem/kubeapi')"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No way this is running some code that is not local right ? e.g remote github.com/openshift/assisted-service

Copy link
Contributor Author

@paul-maidment paul-maidment Jan 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's literally the name of the package

You can see here in go list...

paulmaidment@fedora ~/projects/workspaces/1/assisted-service/subsystem (MGMT-19258)$ go list ./...
github.com/openshift/assisted-service/subsystem
github.com/openshift/assisted-service/subsystem/kubeapi
github.com/openshift/assisted-service/subsystem/utils_test

It's the pattern that the package name is fully qualified with the module URI from go mod

module github.com/openshift/assisted-service

Always easy to verify in any case, just change one test and watch it fail. Which I have already seen.

Copy link
Contributor

@danmanor danmanor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice !

@danmanor
Copy link
Contributor

danmanor commented Jan 6, 2025

/override ci/prow/edge-e2e-ai-operator-disconnected-capi

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jan 6, 2025
Copy link

openshift-ci bot commented Jan 6, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: danmanor, paul-maidment

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [danmanor,paul-maidment]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link

openshift-ci bot commented Jan 6, 2025

@danmanor: Overrode contexts on behalf of danmanor: ci/prow/edge-e2e-ai-operator-disconnected-capi

In response to this:

/override ci/prow/edge-e2e-ai-operator-disconnected-capi

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD bfd8f26 and 2 for PR HEAD eba0446 in total

Copy link

openshift-ci bot commented Jan 6, 2025

@paul-maidment: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot openshift-merge-bot bot merged commit 1f1f449 into openshift:master Jan 6, 2025
18 checks passed
@openshift-bot
Copy link
Contributor

[ART PR BUILD NOTIFIER]

Distgit: ose-agent-installer-api-server
This PR has been included in build ose-agent-installer-api-server-container-v4.19.0-202501070436.p0.g1f1f449.assembly.stream.el9.
All builds following this will include this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api-review Categorizes an issue or PR as actively needing an API review. approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants