Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run Gradle integration tests with K2 analysis #3342

Closed
IgnatBeresnev opened this issue Nov 13, 2023 · 0 comments · Fixed by #3390
Closed

Run Gradle integration tests with K2 analysis #3342

IgnatBeresnev opened this issue Nov 13, 2023 · 0 comments · Fixed by #3390
Assignees
Labels
topic: K2 Issues / PRs that are related to the K2 migration. See #2888
Milestone

Comments

@IgnatBeresnev
Copy link
Member

It should be possible to run Gradle integration tests with the K2 analysis enabled.

Notes

  1. Under the hood, it should use the switch that was introduced in Add the ability to switch between K1 and K2 analysis for testing #3113 (i.e it should run Gradle test projects with that flag). This is to avoid situations where the automated tests pass, but manual testing fails - it should represent real-world scenarios.
  2. The task integrationTest run without any additional parameters or configuration should NOT run tests with the K2 analysis, it should use the stable K1 analysis only. K2 is still experimental and not everything is expected to work reliably, so its testing should be separate from the tests that are expected to be stable. Especially given that K1 integration tests are often flaky and take a lot of time (1h+) as it is.
  3. Not all integration tests are expected to pass with K2 enabled, so the tests should be configurable / selectable without having to configure or ignore it in code. In other words, tests should be selectable via an input parameter.
  4. It should be possible to run these tests from the command line and via TeamCity.

Proposal

JUnit's ability to run selected tests via command line could be utilized:

./gradlew :dokka-integration-tests:gradle:integrationTest 
    --tests org.jetbrains.dokka.it.gradle.kotlin.CoroutinesGradleIntegrationTest

Could also check if org.jetbrains.dokka.experimental.tryK2 was set to true when the integrationTest task was run, and pass it into the TestKit runner:

.withArguments(
listOfNotNull(
"-Pdokka_it_dokka_version=${System.getenv("DOKKA_VERSION")}",
"-Pdokka_it_kotlin_version=${buildVersions.kotlinVersion}",
buildVersions.androidGradlePluginVersion?.let { androidVersion ->
"-Pdokka_it_android_gradle_plugin_version=$androidVersion"
},
* arguments
)

So that the tests with K2 could be run as simple as:

./gradlew :dokka-integration-tests:gradle:integrationTest 
    --tests org.jetbrains.dokka.it.gradle.kotlin.CoroutinesGradleIntegrationTest 
    -Porg.jetbrains.dokka.experimental.tryK2=true

TeamCity

I've prepared a separate TeamCity build configuration (Dokka Integration Tests [K2]) for testing and for future use, it has the same settings as the main tests, but the run trigger is disabled, so you'll have to trigger it manually. Once this issue is fixed, the trigger could be enabled, otherwise we'll see build failures or waste agent time.

The list of tests to be run with K2 could for now be hardcoded inside Build step: Gradle. It might also be possible to turn each test into a checkbox, and run tests based on enabled checkboxes (this might be convenient for testing new versions or fixes). The checkboxes are not expected to be done as part of this issue as it might take some considerable time to implement.

@IgnatBeresnev IgnatBeresnev added the topic: K2 Issues / PRs that are related to the K2 migration. See #2888 label Nov 13, 2023
@IgnatBeresnev IgnatBeresnev linked a pull request Nov 13, 2023 that will close this issue
@vmishenev vmishenev linked a pull request Nov 30, 2023 that will close this issue
@IgnatBeresnev IgnatBeresnev added this to the Dokka 1.9.20 milestone Dec 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: K2 Issues / PRs that are related to the K2 migration. See #2888
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants