From eb3a113441088e6c8fa5d9cd4532913c29986b8e Mon Sep 17 00:00:00 2001 From: Stefano Cunego <93382903+kukkok3@users.noreply.github.com> Date: Wed, 8 May 2024 12:40:25 +0200 Subject: [PATCH] feat: Testcase and testplan templates (#466) * wip * wip * wip * wip * wip * wip * wip * feat: add proposal template wip * fix * wip * wip * wip * wip * wip * wip * wip * wip * feat: add testplan template * fixes * markdown fix * markdown fix --- .config/dictionaries/project.dic | 5 +- .github/ISSUE_TEMPLATE/test_case.yml | 55 +++++++ .github/ISSUE_TEMPLATE/test_plan.yml | 31 ++++ docs/src/architecture/.pages | 3 +- docs/src/architecture/10_quality/.pages | 1 + docs/src/architecture/10_quality/index.md | 7 + .../quality_requirements.md} | 2 +- .../architecture/10_quality/test_reports.md | 3 + .../10_quality/testplans/template.md | 149 ++++++++++++++++++ docs/src/index.md | 4 - 10 files changed, 251 insertions(+), 9 deletions(-) create mode 100644 .github/ISSUE_TEMPLATE/test_case.yml create mode 100644 .github/ISSUE_TEMPLATE/test_plan.yml create mode 100644 docs/src/architecture/10_quality/.pages create mode 100644 docs/src/architecture/10_quality/index.md rename docs/src/architecture/{10_quality_requirements.md => 10_quality/quality_requirements.md} (99%) create mode 100644 docs/src/architecture/10_quality/test_reports.md create mode 100644 docs/src/architecture/10_quality/testplans/template.md diff --git a/.config/dictionaries/project.dic b/.config/dictionaries/project.dic index 69acc2fc5ac..a1d66ad7135 100644 --- a/.config/dictionaries/project.dic +++ b/.config/dictionaries/project.dic @@ -97,6 +97,8 @@ mitigations moderations msedgedriver multidex +multiplatform +Multiplatform myproject nanos NDEBUG @@ -154,6 +156,7 @@ SYSROOT tacho testcov testdocs +testplan testunit thiserror thollander @@ -186,5 +189,3 @@ xctest xctestrun xcworkspace yoroi -multiplatform -Multiplatform diff --git a/.github/ISSUE_TEMPLATE/test_case.yml b/.github/ISSUE_TEMPLATE/test_case.yml new file mode 100644 index 00000000000..0467663dd43 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/test_case.yml @@ -0,0 +1,55 @@ +name: Test Case +description: Create a test case +title: "[Test Case]: " +labels: ["testcase"] +projects: ["/input-output-hk/102"] +body: + - type: input + attributes: + label: Objective + description: A concise description of what we want to test. + validations: + required: true + + - type: textarea + attributes: + label: References + description: Any useful references like github issues, jira epics. + validations: + required: false + + - type: textarea + attributes: + label: Environment + description: The environment where to run the testcase + validations: + required: true + + - type: textarea + attributes: + label: Preconditions + description: Any step or setup that is needed before running the testcase + validations: + required: false + + - type: textarea + attributes: + label: Steps + description: The steps needed to complete the test + placeholder: | + | \# | Step description | Expected result | + | -- | ------------------------- | -------------------------------------------- | + | 1. | <what a tester should do> | <what a tester should see when they do that> | + | 2. | <what a tester should do> | <what a tester should see when they do that> | + validations: + required: true + + - type: textarea + attributes: + label: Notes + description: | + Any additional information + + Tip: You can attach images or log files by clicking this area to highlight it and then dragging files in. + validations: + required: false diff --git a/.github/ISSUE_TEMPLATE/test_plan.yml b/.github/ISSUE_TEMPLATE/test_plan.yml new file mode 100644 index 00000000000..889d205f99b --- /dev/null +++ b/.github/ISSUE_TEMPLATE/test_plan.yml @@ -0,0 +1,31 @@ +name: Test Plan +description: Create a test plan +title: "[Test plan]: <title>" +labels: ["testplan"] +projects: ["/input-output-hk/102"] +body: + - type: input + attributes: + label: Objective + description: A concise description of the objective of the test plan. + validations: + required: true + + - type: input + attributes: + label: Test plan document + placeholder: https://input-output-hk.github.io/catalyst-voices/architecture/10_quality/testplans/template.md + description: A link to the test plan document. + validations: + required: true + + - type: textarea + attributes: + label: Test cases + description: The list of the test cases that are part of the test plan + placeholder: | + -[] https://github.com/input-output-hk/catalyst-voices/issues/1 + -[] https://github.com/input-output-hk/catalyst-voices/issues/1 + -[] https://github.com/input-output-hk/catalyst-voices/issues/1 + validations: + required: true diff --git a/docs/src/architecture/.pages b/docs/src/architecture/.pages index 4120ce54fd7..ad4584a5e1b 100644 --- a/docs/src/architecture/.pages +++ b/docs/src/architecture/.pages @@ -9,7 +9,6 @@ nav: - 07_deployment_view.md - 08_concepts - 09_architecture_decisions - - 10_quality_requirements.md + - 10_quality - 11_technical_risks.md - 12_glossary.md - diff --git a/docs/src/architecture/10_quality/.pages b/docs/src/architecture/10_quality/.pages new file mode 100644 index 00000000000..7d11b5072aa --- /dev/null +++ b/docs/src/architecture/10_quality/.pages @@ -0,0 +1 @@ +title: Quality \ No newline at end of file diff --git a/docs/src/architecture/10_quality/index.md b/docs/src/architecture/10_quality/index.md new file mode 100644 index 00000000000..e0d95b87824 --- /dev/null +++ b/docs/src/architecture/10_quality/index.md @@ -0,0 +1,7 @@ +--- +icon: material/thought-bubble +--- + +# Quality strategies + +<!-- See: https://docs.arc42.org/section-10/ --> diff --git a/docs/src/architecture/10_quality_requirements.md b/docs/src/architecture/10_quality/quality_requirements.md similarity index 99% rename from docs/src/architecture/10_quality_requirements.md rename to docs/src/architecture/10_quality/quality_requirements.md index 19430759185..43b58d5f1ae 100644 --- a/docs/src/architecture/10_quality_requirements.md +++ b/docs/src/architecture/10_quality/quality_requirements.md @@ -2,7 +2,7 @@ icon: material/quality-high --- -# Quality Requirements (WIP) +# Quality Requirements <!-- See: https://docs.arc42.org/section-10/ --> diff --git a/docs/src/architecture/10_quality/test_reports.md b/docs/src/architecture/10_quality/test_reports.md new file mode 100644 index 00000000000..a39855b1aa6 --- /dev/null +++ b/docs/src/architecture/10_quality/test_reports.md @@ -0,0 +1,3 @@ +# Test reports + +* [Main branch test report](https://input-output-hk.github.io/catalyst-voices/allure-action/main/test-report/latest.html) diff --git a/docs/src/architecture/10_quality/testplans/template.md b/docs/src/architecture/10_quality/testplans/template.md new file mode 100644 index 00000000000..4b37e1f83f6 --- /dev/null +++ b/docs/src/architecture/10_quality/testplans/template.md @@ -0,0 +1,149 @@ +<!-- +Based on: https://books.google.it/books?id=vHlTOVTKHeUC&hl=it&source=gbs_navlinks_s + https://testing.googleblog.com/2016/06/the-inquiry-method-for-test-planning.html + https://testing.googleblog.com/2011/09/10-minute-test-plan.html +--> + +# Test Plan Template + +> :memo: **Note:** Substitute with test plan name and link to the github testplan issue. +> +# [Test Plan Name](https://github.com/input-output-hk/catalyst-voices/issues/1) + +> :memo: **Note:** This is meant to be only a guideline, the paragraphs can be removed or added as they fit. + +* [General informations](#general-informations) + * [Abstract](#abstract) + * [Stakeholders](#stakeholders) + * [Requirements](#requirements) + * [Tools](#tools) +* [ACC framework](#acc-framework) + * [Attributes](#attributes) + * [Components](#components) + * [Capabilities](#capabilities) +* [Unit tests](#unit-tests) + * [Unit tests strategy](#unit-tests-strategy) + * [Unit test cases](#unit-test-cases) +* [Functional tests](#functional-tests) + * [Functional tests strategy](#functional-tests-strategy) + * [Functional test cases](#functional-test-cases) +* [Regression tests](#regression-tests) + * [Regression tests strategy](#regression-tests-strategy) + * [Regression test cases](#regression-test-cases) +* [Integration tests](#integration-tests) + * [Integration tests strategy](#integration-tests-strategy) + * [Integration test cases](#integration-test-cases) +* [End-to-end tests](#end-to-end-tests) + * [End-to-end tests strategy](#end-to-end-tests-strategy) + * [End-to-end test cases](#end-to-end-test-cases) + +## General Informations + +### Abstract + +> :memo: **Note:** Describe the design and architecture of the system in a way that highlights possible points of failure + +### Stakeholders + +> :memo: **Note:** Insert the relevant stakeholders that need to understand, review and approve the test plan + +| Role/Name | Contact | Approval | +|-----------------|----------------|----------------| +| *Product-Owner* | | | +| *Developers* | | | +| *Admin* | | | +| *SRE* | | | +| *Testers* | | | + +### Requirements + +> :memo: **Note:** Business requirements, insert links to relevant Github or JIRA tickets, +> list what platforms are supported, what will not be tested, etc + +### Tools + +> :memo: **Note:** Describe what tools will be needed for the testing, if new tools are needed to be developed + +## ACC framework + +> :memo: **Note:** Use the [ACC framework](https://testing.googleblog.com/2011/09/10-minute-test-plan.html) +> to help you define the test cases + +### Attributes + +> :memo: **Note:** The adverbs and adjectives that describe the high level concepts testing is meant to ensure. +> Attributes such as fast, usable, secure, accessible and so forth. +> The quality metrics document in this repo should and can be used as a guideline + +### Components + +> :memo: **Note:** The nouns that define the major code chunks that comprise the product. +These are classes, module names and features of the application. + +### Capabilities + +> :memo: **Note:** The verbs that describe user actions and activities. +> Every capability should be testable. + +## Unit tests + +### Unit tests strategy + +> :memo: **Note:** Evaluate new features and bug fixes introduced in this release, and the extent of the unit tests + +### Unit test cases + +| \# | OBJECTIVE | INPUT | EXPECTED RESULTS | TO BE AUTOMATED | +| --- | --------- | ----- | ---------------- | --------------- | +| 1 | | | | | + +## Functional tests + +### Functional tests strategy + +> :memo: **Note:** Evaluate new features introduced in this release, and the extent of the functional tests + +### Functional test cases + +| \# | OBJECTIVE | INPUT | EXPECTED RESULTS | TO BE AUTOMATED | +| --- | --------- | ----- | ---------------- | --------------- | +| 1 | | | | | + +## Regression tests + +### Regression tests strategy + +> :memo: **Note:** Ensure that previously developed and tested software still performs after change. + +### Regression test cases + +| \# | OBJECTIVE | INPUT | EXPECTED RESULTS | TO BE AUTOMATED | +| --- | --------- | ----- | ---------------- | --------------- | +| 1 | | | | | + +## Integration tests + +### Integration tests strategy + +> :memo: **Note:** Evaluate all integrations with other functions, services etc. + +### Integration test cases + +| \# | OBJECTIVE | INPUT | EXPECTED RESULTS | TO BE AUTOMATED | +| --- | --------- | ----- | ---------------- | --------------- | +| 1 | | | | | + +## End-to-end tests + +### End-to-end tests strategy + +> :memo: **Note:** Evaluate how will test infrastructure, systems under test, and other dependencies be managed? +> How will they be deployed? +> How will persistence be set-up/torn-down? +> How will you handle required migrations from one datacenter to another? + +### End-to-end test cases + +| \# | OBJECTIVE | INPUT | EXPECTED RESULTS | TO BE AUTOMATED | +| --- | --------- | ----- | ---------------- | --------------- | +| 1 | | | | | diff --git a/docs/src/index.md b/docs/src/index.md index 52017f8ad64..d6e791c23d0 100644 --- a/docs/src/index.md +++ b/docs/src/index.md @@ -8,7 +8,3 @@ icon: material/chat * [MkDocs](https://www.mkdocs.org) * [Material for MkDocs](https://squidfunk.github.io/mkdocs-material/) - -## Test reports - -* [Main test report](https://input-output-hk.github.io/catalyst-voices/allure-action/main/test-report/latest.html)