Repository Setup

This documents the process done to setup the repository to automatically generate documentation with readthedocs, run unit tests at the right time, and integrate with CodeCov. Github Actions are used for the CI/CD setup which is relatively simple at the moment and can be configured through Github, Readthedocs and Codecov have to be activated through their respective sites and connected as webhooks to the repository.

This repo mostly follows GitFlow with a master branch which is always the current release of the code, and a dev branch that is the development version of the code. Branch protection rules are in place for master and dev which ensure that code can only be committed to these branches through reviewed PRs (although the rules are often left a little loose for dev out of convenience):

• Require pull request reviews before merging
• Require status checks to pass before merging ("build" action selected)
• Require branches to be up to date before merging
• Require linear history
• Include administrators

These are configured through the "Settings" -> "Branches" section of the repository. The basic idea to GitFlow is to have a master branch always representing the current release, new work being done on a dev branch with other branches derived from it for specific features. Merging into dev should be done through pull requests so that code can be reviewed, and typically master is only updated by pull request from dev when a new version of the software is ready to be released. These branch rules are designed to help enforce this.

Documentation

Documentation is automatically generated using Readthedocs for data in the docs directory.

Readthedocs uses Sphinx to generate documentation which requires reStructuredText (.rst) files containing documentation and layout information. Additionally it requires a configuration file conf.py to store template information and state what plugins to use. The basic setup was created using these commands:

sphinx-quickstart
sphinx-apidoc -o . ../openep

The generated conf.py file was modified to add the current and parent directories to sys.path, add 'sphinx.ext.autodoc' to the list of used extensions, and changing the html theme. This will generate .rst files for existing modules and it's into these that contributors should write explanatory information and manage other content. As new modules are added these may require new .rst files to be added. Docstrings in the actual code will also be imported and used to generate documentation automatically. For more info on Sphinx here is the quickstart.

Readthedocs additionally depends on the .readthedocs.yaml file in the project root. The default one is used here which states what config file to use, what Python version, and where requirements are found.

Readthedocs needs to be linked to a repository, this is done by logging into the website with Github credentials and selecting to setup the appropriate repository. Once the instructions have been followed it should show up as a webhook in the repository's settings.

Unit tests

Basic unit tests are implemented here as a Github action, which runs at particular trigger times in Github hosted VMs. These are setup through YAML files in the .github/workflows directory. A new workflow can be created by clicking on the Actions tab of the repository and selecting "New workflow". The one used here is based off the basic "Python package" workflow provided by Github. In the .github/workflows/python-app.yml file there is only one job "build" which has steps for setting up the environment, running flake8 code checker, running the unit tests with coverage, and uploading the coverage report.

The line which runs the unit tests is the following:

coverage run -m unittest tests/test_*.py

What this does is run the unittest module with all source files in the tests directory starting with test_ passed as arguments. This means that the only files run as part of the testing operation are those with that prefix, and all these are expected to contain unit test code written with the unittest module. The best place to start on how to use this module is the documentation here, but in short unit tests are methods of a subclass of unittest.TestCase. This basic example runs tests to verify some basic properties:

import unittest

class BasicTest(unittest.TestCase):
    def test1(self):
        """Test to ensure a string literal is not None."""
        self.assertNotNone("Hello, tester")  # a string literal is obviously not None

    def test2(self):
        """Test to check an exception is raised if trying to convert a bad number."""
        with self.assertRaises(ValueError):
            s=int("I'm not a number") 

The theory and purpose to unit tests is a large subject area but the objective is to test low level components in a thorough manner so that there's a reasonable expectation they are correct. Input data must be created to test classes and routines so that minor changes that would introduce errors can be caught before they become problems. Running unit tests whenever a commit is made or PR proposed to the master or dev branches ensures there's always a minimum level of correctness that's being enforced whenever new code is introduced.

Codecov

Code coverage is the metric of how much of the system's code was actual run when unit tests are executed. This helps identify code that isn't properly tested, such as conditional branches that aren't used by tests or dead code that's not being used at all. Having a high coverage metric indicates your tests are thorough so new code should be accompanied with new tests providing as close to 100% coverage as possible.

Codecov is another webhook that will use the coverage information from the coverage tool used in the build Github Action to present the coverage information in a convenient web interface. Codecov is setup by logging into its site with Github credentials, enabling access to the desired repository, specifically enabling it as a webhook. To send the coverage information a step is added to the build job at the end:

- name: Upload coverage
  uses: codecov/codecov-action@v1
  with:
    file: ./coverage.xml

This can be found in .github/workflows/python-app.yml after the unit tests are run. The tests are executed by the coverage which is then used to produce the coverage.xml file. Tests, or executing any script file, can be done locally with coverage followed by coverage report to see a simple output on the command line. See the coverage docs for more information.

PyPI Release

Whenever a new release is made this is uploaded automatically to PyPI using the default Github workflow "Publish Python Package". To upload to PyPI these steps explain the process. For this repo the basic steps are:

1. Create account on pypi.org
2. Create a wheel file with python setup.py bdist_wheel, this creates dist/RandomDataset-0.1.0-py3-none-any.whl
3. Upload this package manually to PyPI with python -m twine upload dist/* (assuming you have twine already installed)
4. Get the API token for the new package and set it to the secret PYPI_API_TOKEN in the repository's settings
5. Add the workflow file .github/workflows/python-publish.yml from here.
6. Commit changes and create a release for the project, this should upload to PyPI automatically