Skip to content

Latest commit

 

History

History
94 lines (60 loc) · 7.92 KB

TESTING.md

File metadata and controls

94 lines (60 loc) · 7.92 KB

Testing

This document provides testing information for contributors to this repository which hosts the source for the https://www.coronawarn.app/ production webserver.

  • Cypress is used to run End-To-End tests of the website contents in the /public directory after it is built from source.
  • markdown-link-check is used to check Markdown documents, such as this one, which are used only on the GitHub repository.

Test strategy

Tests are conducted in three locations:

  1. Locally on a contributor's system using a cloned copy of the website (optional)
  2. On GitHub where the source code of the web resides (automatic)
  3. On the webserver https://www.coronawarn.app/. After the web is built on GitHub it is then deployed to this webserver (automatic)

Tests can be run locally and manually before submission. Automated tests, running in GitHub Actions workflows, are used to ensure quality of the website submissions from pull requests. As a final check, some tests are also applied to the production webserver https://www.coronawarn.app from a GitHub Actions workflow.

Running tests

Local tests

You can run tests locally using npm commands in the Test modules table below. You must have Node.js installed and have run npm install before running any tests. See the Development section in the README document for more details. You can run the full test suite using:

npm test

You can also interactively run individual tests through the Cypress app using:

npm run test:open

The production web https://www.coronawarn.app runs under the Ubuntu operating system with a case-sensitive file-system. Perform local tests preferably under Ubuntu, since other operating systems, such as Microsoft Windows, which access files in a case-insensitive mode, may not reveal problems in testing if there is a mismatch between the upper/lower-case file naming and the reference to the file.

The npm run checklinks script for checking Markdown documents uses Unix commands. On a Microsoft Windows operating system, the Unix commands can be provided by git for Windows with its embedded Git BASH emulation. If you are using Command Prompt or Windows PowerShell on Windows instead of the Git BASH shell, then you need to execute the following command:

npm config set script-shell "C:\\Program Files\\git\\bin\\bash.exe" --location user

otherwise you may see an error message similar to "'xargs' is not recognized as an internal or external command, operable program or batch file."

GitHub tests

You can access GitHub Actions workflows containing tests in the table below from the Actions menu on GitHub. Refer to the GitHub documentation Manually running a workflow for more information. The actions either run on the repository contents or on the production webserver as noted in the table.

Test modules

Test target Type Local manual test GitHub Actions (repository) GitHub Actions (production)
All web Cypress npm test -
General web Cypress npm run test:short build-and-test cypress-test-prod
Links in web Cypress npm run test:links test-check_links
Links in Markdown Markdown npm run checklinks test-checklinks-md

npm test

The command npm test runs all Cypress tests locally. It is the same as running both npm run test:short and npm run test:links (see below). There is no combined single equivalent GitHub Actions test.

npm run test:short

The command npm run test:short runs all Cypress tests from the top level directory cypress/e2e as in the following table. The equivalent GitHub test is build-and-test. The same set of tests is also included in cypress-test-prod which runs daily at 09:00 UTC and tests the production webserver https://www.coronawarn.app/.

Cypress test name Purpose
app_to_web Checks that all links to the website used by the Android and iOS apps are working.
blog_spec Checks if the blog archive is accessible.
check_anchor_links Detects broken anchor links throughout the website.
check_videos Detects broken videos throughout the website.
eventRegistration Checks that all fields in event-registration exist and fills them to create a QR code.
faq Simulates an FAQ search and checks the result for mobile and desktop view.
mime Checks that CSS and JS files have the correct MIME type.

npm run test:links

The command npm run test:links runs the Cypress test cypress/e2e/hybrid/check_links which detects broken links throughout the website. The equivalent GitHub Actions test is test-check_links. This test is separated from the other Cypress test for two reasons:

  • It may report link failures due to the unavailability of third party websites. The aim of testing is however to find out if there are errors in this website, not in others, unless they are permanent, in which case link changes may be necessary.
  • npm run test:links takes about 10 minutes to run locally, which is significantly longer that the elapsed time for other tests. When cypress/e2e/hybrid/check_links is run on GitHub it takes about 15 to 25 minutes to run.

test-check_links runs weekly on Mondays at 03:15 UTC.

Cypress

Cypress cache

Executing npm install, as described in the Getting started section, caches a separate copy of Cypress on your local system. If the cwa-website later specifies a different Cypress version and npm install is executed again, then any previously cached Cypress versions remain on your system. Each cached version uses about 0.5 GB of storage. To remove other versions, apart from the currently used version, execute:

npx cypress cache prune

For more detailed information refer to the Cypress cache command documentation.