Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Test] Dev Spaces load tests based on kubect apply #22815

Merged
merged 2 commits into from
Feb 14, 2024
Merged

Conversation

SkorikSergey
Copy link
Contributor

@SkorikSergey SkorikSergey commented Feb 12, 2024

What does this PR do?

Currently, for load tests we use UI e2e approach - https://github.com/eclipse/che/tree/main/tests/e2e/specs

This is not the best way to do the load testing. Need to reconsider this approach and use
kubectl apply which would run simultaneously hundreds of workspaces. We should time:

  • number of failures
  • startup time (avg / median)

Screenshot/screencast of this PR

What issues does this PR fix or reference?

https://issues.redhat.com/browse/CRW-5633

How to test this PR?

  1. Log in to Openshift cluster with DevSpaces deployed from terminal
  2. Start load-test.sh script from test/e2e/performance/load-tests. Set number of started workspaces by -c parameter(like./load-test.sh -c 5).
  3. This script gets cpp sample devfile.yaml from DevSpaces devfile registry and starts workspaces.
  4. As results there are average time of workspace starting and number of failed workspaces.

Test logs

[user@fedora load-tests]$ ./load-test.sh -c 15
Context "user-devspaces/api-ocp413-crw-qe-com:6443/user" modified.
Clean up the environment
Download the devfile
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3811  100  3811    0     0   9274      0 --:--:-- --:--:-- --:--:--  9295
Start workspaces from sample devfile
2502
1309
devworkspacetemplate.workspace.devfile.io/che-code-cpp created
devworkspace.workspace.devfile.io/dw1 created
devworkspace.workspace.devfile.io/dw2 created
devworkspace.workspace.devfile.io/dw3 created
devworkspace.workspace.devfile.io/dw4 created
devworkspace.workspace.devfile.io/dw5 created
devworkspace.workspace.devfile.io/dw6 created
devworkspace.workspace.devfile.io/dw7 created
devworkspace.workspace.devfile.io/dw8 created
devworkspace.workspace.devfile.io/dw9 created
devworkspace.workspace.devfile.io/dw10 created
devworkspace.workspace.devfile.io/dw11 created
devworkspace.workspace.devfile.io/dw12 created
devworkspace.workspace.devfile.io/dw13 created
devworkspace.workspace.devfile.io/dw14 created
devworkspace.workspace.devfile.io/dw15 created
Wait for all workspaces are started and calculate average workspaces starting time
devworkspace.workspace.devfile.io/dw1 condition met
Devworkspace dw1 starting time: 13 seconds
devworkspace.workspace.devfile.io/dw2 condition met
Devworkspace dw2 starting time: 23 seconds
error: timed out waiting for the condition on devworkspaces/dw3
Timeout waiting for dw3 to become ready or an error occurred.
Error from server (NotFound): pods "workspace447d6cf327464a20" not found
error: timed out waiting for the condition on devworkspaces/dw4
Timeout waiting for dw4 to become ready or an error occurred.
Error from server (NotFound): pods "workspace30636f3ff61144ff" not found
devworkspace.workspace.devfile.io/dw5 condition met
Devworkspace dw5 starting time: 21 seconds
error: timed out waiting for the condition on devworkspaces/dw6
Timeout waiting for dw6 to become ready or an error occurred.
Error from server (NotFound): pods "workspace60b59ef1d3584e05" not found
devworkspace.workspace.devfile.io/dw7 condition met
Devworkspace dw7 starting time: 18 seconds
devworkspace.workspace.devfile.io/dw8 condition met
Devworkspace dw8 starting time: 16 seconds
devworkspace.workspace.devfile.io/dw9 condition met
Devworkspace dw9 starting time: 26 seconds
devworkspace.workspace.devfile.io/dw10 condition met
Devworkspace dw10 starting time: 13 seconds
devworkspace.workspace.devfile.io/dw11 condition met
Devworkspace dw11 starting time: 12 seconds
devworkspace.workspace.devfile.io/dw12 condition met
Devworkspace dw12 starting time: 11 seconds
devworkspace.workspace.devfile.io/dw13 condition met
Devworkspace dw13 starting time: 11 seconds
devworkspace.workspace.devfile.io/dw14 condition met
Devworkspace dw14 starting time: 10 seconds
devworkspace.workspace.devfile.io/dw15 condition met
Devworkspace dw15 starting time: 9 seconds
==================== Test results ====================
Average workspace starting time for 12 workspaces from 15 started: 15 seconds
3 workspaces failed. See logs for details.

image

PR Checklist

As the author of this Pull Request I made sure that:

Reviewers

Reviewers, please comment how you tested the PR when approving it.

@SkorikSergey SkorikSergey added status/in-progress This issue has been taken by an engineer and is under active development. kind/task Internal things, technical debt, and to-do tasks to be performed. sprint/current area/qe labels Feb 12, 2024
@SkorikSergey SkorikSergey self-assigned this Feb 12, 2024
@che-bot che-bot added the target/branch Indicates that a PR will be merged into a branch other than master. label Feb 12, 2024
@SkorikSergey SkorikSergey added status/code-review This issue has a pull request posted for it and is awaiting code review completion by the community. and removed status/in-progress This issue has been taken by an engineer and is under active development. labels Feb 13, 2024
@SkorikSergey SkorikSergey marked this pull request as ready for review February 14, 2024 10:49
@SkorikSergey SkorikSergey merged commit e76edad into main Feb 14, 2024
4 checks passed
@SkorikSergey SkorikSergey deleted the load-tests branch February 14, 2024 10:51
@che-bot che-bot removed the status/code-review This issue has a pull request posted for it and is awaiting code review completion by the community. label Feb 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/qe kind/task Internal things, technical debt, and to-do tasks to be performed. sprint/current target/branch Indicates that a PR will be merged into a branch other than master.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants