diff --git a/docs/20220923 - 250 pods per node.md b/docs/20220923 - 250 pods per node.md index 3537e740..85ef97d4 100644 --- a/docs/20220923 - 250 pods per node.md +++ b/docs/20220923 - 250 pods per node.md @@ -43,7 +43,7 @@ See [the rke2 installation script in this repo](../rke2/install_rke2.sh) for det ## Full configuration details -All infrastructure is defined via [Terraform](https://www.terraform.io/) files in the [20220923_250_pods_per_node](https://github.com/moio/scalability-tests/tree/20220923_250_pods_per_node) branch. +All infrastructure is defined via [Terraform](https://www.terraform.io/) files in the [20220923_250_pods_per_node](https://github.com/rancher/dartboard/tree/20220923_250_pods_per_node) branch. Note in particular [inputs.tf](../inputs.tf) for the main parameters. @@ -58,7 +58,7 @@ Note in particular [inputs.tf](../inputs.tf) for the main parameters. - get [Terraform](https://www.terraform.io/downloads) - check out this project ```shell -git clone https://github.com/moio/scalability-tests.git +git clone https://github.com/rancher/dartboard.git cd scalability-tests git checkout 20220923_250_pods_per_node ``` diff --git a/docs/20220928 - 250 pods per node with cluster operations.md b/docs/20220928 - 250 pods per node with cluster operations.md index bf38af40..df56711f 100644 --- a/docs/20220928 - 250 pods per node with cluster operations.md +++ b/docs/20220928 - 250 pods per node with cluster operations.md @@ -45,7 +45,7 @@ See [the rke2 installation script in this repo](../rke2/install_rke2.sh) for det ## Full configuration details -All infrastructure is defined via [Terraform](https://www.terraform.io/) files in the [20220928_250_pods_per_node_with_cluster_operations](https://github.com/moio/scalability-tests/tree/20220928_250_pods_per_node_with_cluster_operations) branch. +All infrastructure is defined via [Terraform](https://www.terraform.io/) files in the [20220928_250_pods_per_node_with_cluster_operations](https://github.com/rancher/dartboard/tree/20220928_250_pods_per_node_with_cluster_operations) branch. Note in particular [inputs.tf](../inputs.tf) for the main parameters. @@ -62,7 +62,7 @@ Note in particular [inputs.tf](../inputs.tf) for the main parameters. - get [Terraform](https://www.terraform.io/downloads) - check out this project ```shell -git clone https://github.com/moio/scalability-tests.git +git clone https://github.com/rancher/dartboard.git cd scalability-tests git checkout 20220928_250_pods_per_node_with_cluster_operations ``` diff --git a/docs/20221003 - 300 pods per node.md b/docs/20221003 - 300 pods per node.md index 06f04972..886f547f 100644 --- a/docs/20221003 - 300 pods per node.md +++ b/docs/20221003 - 300 pods per node.md @@ -79,8 +79,8 @@ See [the rke2 installation script in this repo](../rke2/install_rke2.sh) for det ## Full configuration details -All infrastructure is defined via [Terraform](https://www.terraform.io/) files in the [20221003_300_pods_per_node](https://github.com/moio/scalability-tests/tree/20221003_300_pods_per_node) branch. Note in particular [inputs.tf](../inputs.tf) for the main parameters. -All tests are driven by [Cypress](https://www.cypress.io/) files in the [cypress](https://github.com/moio/scalability-tests/tree/20221003_300_pods_per_node/cypress) directory. +All infrastructure is defined via [Terraform](https://www.terraform.io/) files in the [20221003_300_pods_per_node](https://github.com/rancher/dartboard/tree/20221003_300_pods_per_node) branch. Note in particular [inputs.tf](../inputs.tf) for the main parameters. +All tests are driven by [Cypress](https://www.cypress.io/) files in the [cypress](https://github.com/rancher/dartboard/tree/20221003_300_pods_per_node/cypress) directory. ## Reproduction Instructions @@ -102,7 +102,7 @@ All tests are driven by [Cypress](https://www.cypress.io/) files in the [cypress - clone this project: ```shell -git clone https://github.com/moio/scalability-tests.git +git clone https://github.com/rancher/dartboard.git cd scalability-tests git checkout 20221003_300_pods_per_node ``` diff --git a/docs/20221128 - API load benchmarks.md b/docs/20221128 - API load benchmarks.md index 8774c02a..cb73cb21 100644 --- a/docs/20221128 - API load benchmarks.md +++ b/docs/20221128 - API load benchmarks.md @@ -69,9 +69,9 @@ References: ## Full configuration details -All infrastructure is defined via [Terraform](https://www.terraform.io/) files in the [20221128_api_load_benchmarks](https://github.com/moio/scalability-tests/tree/20221128_api_load_benchmarks/terraform) branch. Note in particular [inputs.tf](https://github.com/moio/scalability-tests/blob/20221128_api_load_benchmarks/terraform/inputs.tf) for the main parameters. -Initial configuration is driven by [Cypress](https://www.cypress.io/) files in the [cypress/e2e](https://github.com/moio/scalability-tests/tree/20221128_api_load_benchmarks/cypress/cypress/e2e) directory. -Benchmark Python scripts are available in the [util](https://github.com/moio/scalability-tests/tree/20221128_api_load_benchmarks/util) directory. +All infrastructure is defined via [Terraform](https://www.terraform.io/) files in the [20221128_api_load_benchmarks](https://github.com/rancher/dartboard/tree/20221128_api_load_benchmarks/terraform) branch. Note in particular [inputs.tf](https://github.com/rancher/dartboard/blob/20221128_api_load_benchmarks/terraform/inputs.tf) for the main parameters. +Initial configuration is driven by [Cypress](https://www.cypress.io/) files in the [cypress/e2e](https://github.com/rancher/dartboard/tree/20221128_api_load_benchmarks/cypress/cypress/e2e) directory. +Benchmark Python scripts are available in the [util](https://github.com/rancher/dartboard/tree/20221128_api_load_benchmarks/util) directory. ## Reproduction Instructions @@ -92,7 +92,7 @@ Benchmark Python scripts are available in the [util](https://github.com/moio/sca - clone this project: ```shell -git clone https://github.com/moio/scalability-tests.git +git clone https://github.com/rancher/dartboard.git cd scalability-tests git checkout 20221128_api_load_benchmarks ``` diff --git a/docs/20221130 - can-i microbenchmark.md b/docs/20221130 - can-i microbenchmark.md index 738f09f1..e09ee250 100644 --- a/docs/20221130 - can-i microbenchmark.md +++ b/docs/20221130 - can-i microbenchmark.md @@ -45,9 +45,9 @@ References: ## Full configuration details -All infrastructure is defined via [Terraform](https://www.terraform.io/) files in the [20221130_can-i_microbenchmark](https://github.com/moio/scalability-tests/tree/20221128_api_load_benchmarks/terraform) branch. Note in particular [inputs.tf](https://github.com/moio/scalability-tests/blob/20221130_can-i_microbenchmark/terraform/inputs.tf) for the main parameters. -Initial configuration is driven by [Cypress](https://www.cypress.io/) files in the [cypress/e2e](https://github.com/moio/scalability-tests/tree/20221130_can-i_microbenchmark/cypress/cypress/e2e) directory. -Benchmark Python scripts are available in the [util](https://github.com/moio/scalability-tests/tree/20221130_can-i_microbenchmark/util) directory. +All infrastructure is defined via [Terraform](https://www.terraform.io/) files in the [20221130_can-i_microbenchmark](https://github.com/rancher/dartboard/tree/20221128_api_load_benchmarks/terraform) branch. Note in particular [inputs.tf](https://github.com/rancher/dartboard/blob/20221130_can-i_microbenchmark/terraform/inputs.tf) for the main parameters. +Initial configuration is driven by [Cypress](https://www.cypress.io/) files in the [cypress/e2e](https://github.com/rancher/dartboard/tree/20221130_can-i_microbenchmark/cypress/cypress/e2e) directory. +Benchmark Python scripts are available in the [util](https://github.com/rancher/dartboard/tree/20221130_can-i_microbenchmark/util) directory. ## Reproduction Instructions @@ -68,7 +68,7 @@ Benchmark Python scripts are available in the [util](https://github.com/moio/sca - clone this project: ```shell -git clone https://github.com/moio/scalability-tests.git +git clone https://github.com/rancher/dartboard.git cd scalability-tests git checkout 20221130_can-i_microbenchmark ``` diff --git a/docs/20221215 - kine locality test.md b/docs/20221215 - kine locality test.md index 5a22022b..f3cf05f9 100644 --- a/docs/20221215 - kine locality test.md +++ b/docs/20221215 - kine locality test.md @@ -29,8 +29,8 @@ No significant difference in list performance of small ConfigMaps, up to 256K of ## Full configuration details -All infrastructure is defined via [Terraform](https://www.terraform.io/) files in the [20221215_kine_locality_test](https://github.com/moio/scalability-tests/tree/20221215_kine_locality_test/terraform) branch. -Benchmark Python script is available in the [util](https://github.com/moio/scalability-tests/tree/20221215_kine_locality_test/util) directory. +All infrastructure is defined via [Terraform](https://www.terraform.io/) files in the [20221215_kine_locality_test](https://github.com/rancher/dartboard/tree/20221215_kine_locality_test/terraform) branch. +Benchmark Python script is available in the [util](https://github.com/rancher/dartboard/tree/20221215_kine_locality_test/util) directory. ## Reproduction Instructions @@ -50,7 +50,7 @@ Benchmark Python script is available in the [util](https://github.com/moio/scala - clone this project: ```shell -git clone https://github.com/moio/scalability-tests.git +git clone https://github.com/rancher/dartboard.git cd scalability-tests git checkout 20221215_kine_locality_test ``` diff --git a/docs/20230306 - steve vai tests.md b/docs/20230306 - steve vai tests.md index fbe8b4cc..9bdcc8bf 100644 --- a/docs/20230306 - steve vai tests.md +++ b/docs/20230306 - steve vai tests.md @@ -37,9 +37,9 @@ Under test conditions, according to collected measures described below: ## Full configuration details -All infrastructure is defined in [Terraform](https://www.terraform.io/) files in the [20230306_steve_vai_tests](https://github.com/moio/scalability-tests/tree/20230306_steve_vai_tests/terraform) branch. +All infrastructure is defined in [Terraform](https://www.terraform.io/) files in the [20230306_steve_vai_tests](https://github.com/rancher/dartboard/tree/20230306_steve_vai_tests/terraform) branch. -[k6](https://k6.io) load test scripts are defined in the [k6](https://github.com/moio/scalability-tests/tree/20230306_steve_vai_tests/k6) directory. +[k6](https://k6.io) load test scripts are defined in the [k6](https://github.com/rancher/dartboard/tree/20230306_steve_vai_tests/k6) directory. ## Reproduction Instructions @@ -57,7 +57,7 @@ All infrastructure is defined in [Terraform](https://www.terraform.io/) files in Deploy the k3d infrastructure and install Rancher: ```shell # clone this project -git clone https://github.com/moio/scalability-tests.git +git clone https://github.com/rancher/dartboard.git cd scalability-tests git checkout 20230306_steve_vai_tests @@ -155,7 +155,7 @@ In the example above, retrieving a page up to 100 resources from the local clust #### Analysis of results -Full results are available in the [20230306 - steve vai test results](https://github.com/moio/scalability-tests/tree/20230306_steve_vai_tests/docs/20230306%20-%20steve%20vai%20test%20results) directory, summary is in the table below: +Full results are available in the [20230306 - steve vai test results](https://github.com/rancher/dartboard/tree/20230306_steve_vai_tests/docs/20230306%20-%20steve%20vai%20test%20results) directory, summary is in the table below: ![table showing a summary of results](images/20230306-table.png) An [Excel file](https://mysuse-my.sharepoint.com/:x:/g/personal/moio_suse_com/ERaeDyfE25xLoQFKiMYa8bgBOb2z24wKNhTp0FVMVumDMA?e=nGOPMy) is available for SUSE employees. diff --git a/docs/20230503 - steve vai tests higher scale.md b/docs/20230503 - steve vai tests higher scale.md index fdb01323..9043ca00 100644 --- a/docs/20230503 - steve vai tests higher scale.md +++ b/docs/20230503 - steve vai tests higher scale.md @@ -39,9 +39,9 @@ Under test conditions, according to collected measures described below: ## Full configuration details -All infrastructure is defined in [Terraform](https://www.terraform.io/) files in the [20230503_steve_vai_tests_higher_scale](https://github.com/moio/scalability-tests/tree/20230503_steve_vai_tests_higher_scale/terraform) branch. +All infrastructure is defined in [Terraform](https://www.terraform.io/) files in the [20230503_steve_vai_tests_higher_scale](https://github.com/rancher/dartboard/tree/20230503_steve_vai_tests_higher_scale/terraform) branch. -[k6](https://k6.io) load test scripts are defined in the [k6](https://github.com/moio/scalability-tests/tree/20230503_steve_vai_tests_higher_scale/k6) directory. +[k6](https://k6.io) load test scripts are defined in the [k6](https://github.com/rancher/dartboard/tree/20230503_steve_vai_tests_higher_scale/k6) directory. ## Reproduction Instructions @@ -86,7 +86,7 @@ TAG=vai make quickbuild Deploy the k3d infrastructure, install Rancher, set up clusters for tests, import built images: ```shell # clone this project -git clone https://github.com/moio/scalability-tests.git +git clone https://github.com/rancher/dartboard.git cd scalability-tests git checkout 20230503_steve_vai_tests_higher_scale @@ -108,7 +108,7 @@ First, we create a given number of ConfigMaps in a test namespace via a k6 scrip Then, we simulate 10 virtual users listing all ConfigMaps in that namespace via another k6 script. Each user will repeat the listing 30 times (for statistical accuracy of measures). The page size is of 100, like in the current UI. We exercise both the k8s based pagination implementation, using the `limit`/`continue` parameters and currently used by the [dashboard](https://github.com/rancher/dashboard/) UI, as well as the new Steve-cache pagination implementation using the `page`/`pagesize` parameters. We test both local and downstream cluster. Tests are repeated for `baseline` and `vai` images. -Details on tests are available in the [bin/run_test.js](https://github.com/moio/scalability-tests/blob/20230503_steve_vai_tests_higher_scale/bin/run_tests.mjs) script source file. +Details on tests are available in the [bin/run_test.js](https://github.com/rancher/dartboard/blob/20230503_steve_vai_tests_higher_scale/bin/run_tests.mjs) script source file. #### Procedure @@ -133,7 +133,7 @@ Interpreting results: the script will output one `results.csv` file with the fol * `p(99)` 99th percentile - 99% of requests had a duration less than or equal to this value * `count` total number of requests -Full results are available in the [results.csv](https://github.com/moio/scalability-tests/tree/20230503_steve_vai_tests_higher_scale/docs/20230503%20-%20steve%20vai%20test%20higher%20scale%20results/results.csv) file, summary is in the table below: +Full results are available in the [results.csv](https://github.com/rancher/dartboard/tree/20230503_steve_vai_tests_higher_scale/docs/20230503%20-%20steve%20vai%20test%20higher%20scale%20results/results.csv) file, summary is in the table below: ![table showing a summary of results](images/20230503-table.png) An [Excel file](https://mysuse-my.sharepoint.com/:x:/g/personal/moio_suse_com/ETkus1LxojlBm7aYWdswNX0BmmkfrQt0NET3oO6QujnNgw?e=bexG44) is available for SUSE employees. diff --git a/docs/20231222 - RKE2 100 clusters 1vs3 nodes comparison.md b/docs/20231222 - RKE2 100 clusters 1vs3 nodes comparison.md index 64679277..31c43b0c 100644 --- a/docs/20231222 - RKE2 100 clusters 1vs3 nodes comparison.md +++ b/docs/20231222 - RKE2 100 clusters 1vs3 nodes comparison.md @@ -75,7 +75,7 @@ The increase in the CPU usage to almost 5 cores in _fig.1_ and 7 cores in _fig.2 Note that, in absence of resources (users, roles, projects, ConfigMaps, Secrets) the load is minimal and the CPU load varies from ~0.750 to ~0.950 cores. In those conditions, base OS processes running on the nodes have a higher CPU footprint. -You can find more screenshots of the resource usage of these two scenarios in the [100 clusters x 1 node grafana screenshots folder](https://github.com/moio/scalability-tests/tree/20231222_rke2_100_clusters_1vs3_nodes_comparison/docs/20231222%20-%20RKE2%20100%20clusters%201vs3%20nodes%20comparison/grafana-screenshots/RKE4x) and in the [100 clusters x 3 nodes grafana screenshots folder](https://github.com/moio/scalability-tests/tree/20231222_rke2_100_clusters_1vs3_nodes_comparison/docs/20231222%20-%20RKE2%20100%20clusters%201vs3%20nodes%20comparison/grafana-screenshots/RKE5x). +You can find more screenshots of the resource usage of these two scenarios in the [100 clusters x 1 node grafana screenshots folder](https://github.com/rancher/dartboard/tree/20231222_rke2_100_clusters_1vs3_nodes_comparison/docs/20231222%20-%20RKE2%20100%20clusters%201vs3%20nodes%20comparison/grafana-screenshots/RKE4x) and in the [100 clusters x 3 nodes grafana screenshots folder](https://github.com/rancher/dartboard/tree/20231222_rke2_100_clusters_1vs3_nodes_comparison/docs/20231222%20-%20RKE2%20100%20clusters%201vs3%20nodes%20comparison/grafana-screenshots/RKE5x). ### Rancher API response time benchmarks @@ -104,9 +104,9 @@ For more results, check the available data shared in the [available data section ## Full configuration details -All infrastructure is defined in [Terraform](https://www.terraform.io/) files in the [20231222_rke2_100_clusters_1vs3_nodes_comparison](https://github.com/moio/scalability-tests/tree/20231222_rke2_100_clusters_1vs3_nodes_comparison/terraform) branch. +All infrastructure is defined in [Terraform](https://www.terraform.io/) files in the [20231222_rke2_100_clusters_1vs3_nodes_comparison](https://github.com/rancher/dartboard/tree/20231222_rke2_100_clusters_1vs3_nodes_comparison/terraform) branch. -[k6](https://k6.io) load test scripts are defined in the [k6](https://github.com/moio/scalability-tests/tree/20231222_rke2_100_clusters_1vs3_nodes_comparison/k6) directory. +[k6](https://k6.io) load test scripts are defined in the [k6](https://github.com/rancher/dartboard/tree/20231222_rke2_100_clusters_1vs3_nodes_comparison/k6) directory. ## Reproduction Instructions @@ -128,7 +128,7 @@ Log into Azure via the CLI: Deploy the RKE2 environment, install Rancher, set up clusters for tests: ```shell # clone this project -git clone -b 20231222_rke2_100_clusters_1vs3_nodes_comparison https://github.com/moio/scalability-tests.git +git clone -b 20231222_rke2_100_clusters_1vs3_nodes_comparison https://github.com/rancher/dartboard.git cd scalability-tests export TERRAFORM_WORK_DIR=terraform/main/azure @@ -137,7 +137,7 @@ export TERRAFORM_WORK_DIR=terraform/main/azure ./bin/setup.mjs && ./bin/run_tests.mjs ```` >[!NOTE] ->by default the branch will setup the 100 clusters x 1 node scenario: if you want to run the 100 clusters x 3 nodes one you may want to change the server_count value in the [azure configuration file](https://github.com/moio/scalability-tests/blob/20231222_rke2_100_clusters_1vs3_nodes_comparison/terraform/main/azure/inputs.tf#L28) to 3 before running the /bin/setup.mjs && ./bin/run_tests.mjs command. +>by default the branch will setup the 100 clusters x 1 node scenario: if you want to run the 100 clusters x 3 nodes one you may want to change the server_count value in the [azure configuration file](https://github.com/rancher/dartboard/blob/20231222_rke2_100_clusters_1vs3_nodes_comparison/terraform/main/azure/inputs.tf#L28) to 3 before running the /bin/setup.mjs && ./bin/run_tests.mjs command. Once the system is provisioned, to get Rancher and clusters access info: @@ -175,9 +175,9 @@ important output data points are: ## Available data -All the data collected from the tests is shared in the [_RKE2 100 clusters 1vs3 nodes comparison_ results directory](https://github.com/moio/scalability-tests/tree/20231222_rke2_100_clusters_1vs3_nodes_comparison/docs/20231222%20-%20RKE2%20100%20clusters%201vs3%20nodes%20comparison/). +All the data collected from the tests is shared in the [_RKE2 100 clusters 1vs3 nodes comparison_ results directory](https://github.com/rancher/dartboard/tree/20231222_rke2_100_clusters_1vs3_nodes_comparison/docs/20231222%20-%20RKE2%20100%20clusters%201vs3%20nodes%20comparison/). There you will find: -* a list of the deployed scenarios ([_list.txt_ file](https://github.com/moio/scalability-tests/tree/20231222_rke2_100_clusters_1vs3_nodes_comparison/docs/20231222%20-%20RKE2%20100%20clusters%201vs3%20nodes%20comparison/list.txt)) -* the raw data from the single tests ([_tests_raw_data_ folder](https://github.com/moio/scalability-tests/tree/20231222_rke2_100_clusters_1vs3_nodes_comparison/docs/20231222%20-%20RKE2%20100%20clusters%201vs3%20nodes%20comparison/tests_raw_data)) -* screenshots from a Grafana dashboard showing CPU and Memory usage from the tested scenarios ([_grafana-screenshots_ folder](https://github.com/moio/scalability-tests/tree/20231222_rke2_100_clusters_1vs3_nodes_comparison/docs/20231222%20-%20RKE2%20100%20clusters%201vs3%20nodes%20comparison/grafana-screenshots)) -* an [OpenDocument spreadsheet](https://github.com/moio/scalability-tests/tree/20231222_rke2_100_clusters_1vs3_nodes_comparison/docs/20231222%20-%20RKE2%20100%20clusters%201vs3%20nodes%20comparison/scalability_tests_summary.ods) containing all the results with few graphs +* a list of the deployed scenarios ([_list.txt_ file](https://github.com/rancher/dartboard/tree/20231222_rke2_100_clusters_1vs3_nodes_comparison/docs/20231222%20-%20RKE2%20100%20clusters%201vs3%20nodes%20comparison/list.txt)) +* the raw data from the single tests ([_tests_raw_data_ folder](https://github.com/rancher/dartboard/tree/20231222_rke2_100_clusters_1vs3_nodes_comparison/docs/20231222%20-%20RKE2%20100%20clusters%201vs3%20nodes%20comparison/tests_raw_data)) +* screenshots from a Grafana dashboard showing CPU and Memory usage from the tested scenarios ([_grafana-screenshots_ folder](https://github.com/rancher/dartboard/tree/20231222_rke2_100_clusters_1vs3_nodes_comparison/docs/20231222%20-%20RKE2%20100%20clusters%201vs3%20nodes%20comparison/grafana-screenshots)) +* an [OpenDocument spreadsheet](https://github.com/rancher/dartboard/tree/20231222_rke2_100_clusters_1vs3_nodes_comparison/docs/20231222%20-%20RKE2%20100%20clusters%201vs3%20nodes%20comparison/scalability_tests_summary.ods) containing all the results with few graphs