Unit Testing

We are using PyTest for unit testing. It is very easy to use and can hook into the built in unittest module if desired. Additionally Tox Is used to automatically run the tests e.g. by Travis whenever there is a commit on a branch or a pull request.

PyTest

Running Tests

Just cd into the project and run:

make test

Simple!

For more information, see https://docs.pytest.org/en/latest/. By default, tests will run in parallel, with the number of workers equal to the number of CPUs available. If you want to run tests in serial mode, use the following command in the terminal:

pytest

For more information on parallel execution, check pytest-xdist.

Creating Tests

PyTest can automatically discover tests by the names given to functions and Classes. It is set up to detect any function which has a name starting with test_ and any class which has a name starting with Test.

Inside the test function use assert statements to check for conditions that should be True for the test to pass.

For examples see https://docs.pytest.org/en/latest/example/index.html.

When creating a test, ensure that it can run in parallel mode. For instance, avoid modifying files that other tests may use. Use temporary directory fixtures to guarantee that a file will not be edited by another test. Also, use thread-safe functions and methods.

When to Use a Function vs a Class

Use one function per self-contained test. Use a class to group related tests together. See https://docs.pytest.org/en/latest/getting-started.html for more.

Fixtures

Sometimes you need to do some setup or have some re-usable functionality for a test and this could easily be repeated across a whole bunch of tests. To avoid this (remember to write DRY code) use a fixture.

A short example from the PyTest documentation here shows how to create one:

# content of ./test_smtpsimple.py
import pytest

@pytest.fixture
def smtp_connection():
    import smtplib
    return smtplib.SMTP("smtp.gmail.com", 587, timeout=5)



def test_ehlo(smtp_connection):
    response, msg = smtp_connection.ehlo()
    assert response == 250
    assert 0  # for demo purposes

  The fixture can be used simply by putting the function name as an argument to the test function. PyTest will automatically pass this to the test function for you. 

For more, including pre-made fixtures, see https://docs.pytest.org/en/latest/fixture.html#fixtures.

Click (CLI) Tests

Click comes with some of its own testing 'equipment', since command line interfaces need a bit of extra work in order to set up unit testing. For more information on this see the click documentation at https://click.palletsprojects.com/en/7.x/testing/. Below is a small snippet from the click documentation showing a basic click command and its corresponding test.

hello.py

import click

@click.command()
@click.argument('name')
def hello(name):
    click.echo('Hello %s!' % name)

test_hello.py

from click.testing
import CliRunner
from hello import hello

def test_hello_world():
    runner = CliRunner()
    result = runner.invoke(hello, ['Peter'])
    assert result.exit_code == 0
    assert result.output == 'Hello Peter!\n'

Tox

Tox is used to set up virtual environments for automatically running tests.