Writing unit tests

How to build and run the existing tests

See Setting up the environment and Testing/Unit Tests

Concept

Every picoTCP component is shipped with its own set of unit tests. Unit tests are based on libcheck. The first version of the unit test platform is maintained in a single file: test/units.c. This module provides a set of unit tests for the core of the stack and the generic tools.

Unit test creation

Unit tests for modules are implemented in their own test/unit/modunit_$(MODULENAME).c file. This file can be created automatically using the script mkunits.sh.

Suppose that we just created a simple picotcp module, which exports a function that broadcasts one udp datagram to port 555.

/* pico_poke.c */
#include <pico_stack.h>
#include <pico_socket.h>

static int socket_cb(uint16_t ev, struct pico_socket *s)
{
}

static int do_pico_poke(char *msg, int len)
{
    struct pico_socket *s;
    int ret;
    pico_err_t err;
    struct pico_ip4 bcast = {
        .addr = 0xFFFFFFFF;
    };

    if ((len < 1) || (msg == NULL)) {
        pico_err = PICO_ERR_EINVAL;
        return -1;
    }

    s = pico_socket_open(PICO_PROTO_IPV4, PICO_PROTO_UDP, socket_cb);
    if (!s)
        return -1;
    ret = pico_socket_sendto(s, msg, len, &bcast, short_be(555));
    if (ret < 0)
        err = pico_err;
    pico_socket_close(s);
    pico_err = err;
    return ret;
}

int pico_poke(char *msg, int len)
{
    return do_pico_poke(msg, len);
}

We can generate unit tests using the script as follows:

test/mkunits.sh modules/pico_poke.c

Warning: the script creates incomplete code, and sometimes may fail recognizing your functions signatures. mkunits.sh is only intended to automate the very boring parts of creating a new unit test module, such as the needed include files, the suite creation, etc.

The file modunit_pico_skull.c file has been created, and contains a skeleton for our unit tests:

#include <pico_stack.h>
#include <pico_socket.h>
#include "modules/pico_poke.c"
#include "check.h"


START_TEST(tc_socket_cb)
{
   /* TODO: test this: static int socket_cb(uint16_t ev, struct pico_socket *s) */
}
END_TEST

START_TEST(tc_do_pico_poke)
{
   /* TODO: test this: static int do_pico_poke(char *msg, int len) */
}
END_TEST


Suite *pico_suite(void)
{
    Suite *s = suite_create("PicoTCP");

    TCase *TCase_socket_cb = tcase_create("Unit test for socket_cb");
    TCase *TCase_do_pico_poke = tcase_create("Unit test for do_pico_poke");


    tcase_add_test(TCase_socket_cb, tc_socket_cb);
    suite_add_tcase(s, TCase_socket_cb);
    tcase_add_test(TCase_do_pico_poke, tc_do_pico_poke);
    suite_add_tcase(s, TCase_do_pico_poke);
return s;
}

int main(void)
{
    int fails;
    Suite *s = pico_suite();
    SRunner *sr = srunner_create(s);
    srunner_run_all(sr, CK_NORMAL);
    fails = srunner_ntests_failed(sr);
    srunner_free(sr);
    return fails;
}

Conditional assertions

The Check library supports a number of interfaces for evaluating the test cases. picoTCP's team favourite are

fail_if(condition [,error_message]);
fail_unless(condition [,error_messages]);

The result is the same, but of course the logic is inverted between the two cases.

Mocking

Some functions in picoTCP can be mocked, meaning that they are defined as weak symbols in the testing environment. These functions are those defined with the modifier MOCKABLE in front. This is the case of the pico_socket_sendto() function, which is often useful to check the payload that is being transmitted by the module under test to the socket interface. The function is indeed defined in stack/pico_socket.c with the signature:

int MOCKABLE pico_socket_sendto(struct pico_socket *s, const void *buf, const int len, void *dst, uint16_t remote_port)

In our poke_test case, we can define a mock of this function, which will allow calling the sockets even if the stack is not initialized or ticking.

static int sendto_mock_called = 0;

int pico_socket_sendto(struct pico_socket *s, const void *buf, const int len, void *dst, uint16_t remote_port)
{
   sendto_mock_called++;
   return len;
}

In this particular case, several use cases can be tested in the unit test function created by the script. Let's replace the TODO comment

   /* TODO: test this: static int do_pico_poke(char *msg, int len) */

with:

   /* Use case 0: normal call */
   sendto_mock_called = 0;
   fail_if(pico_poke("Hello", 5) != 5); /* Assert appropriate return value (len) */
   fail_if(sendto_mock_called != 1);  /* Assert that sendto has been called exactly once */
   
   /*Use case 1: NULL msg */
   sendto_mock_called = 0;
   fail_if(pico_poke(NULL, 3) != -1); /* Assert expected fail */
   fail_if(pico_err != PICO_ERR_EINVAL ); /* Assert expected error code */
   fail_if(sendto_mock_called != 0);  /* Assert that sendto has not been called */

   /* etc ... */

The unit test function can now be completed.

Compiling

In the main Makefile, under the target units:, add a new line to compile and link your unit test. If the test uses functions from the stack library, be sure to add the libpicotcp.a to the linking process. In order to compile modunit_pico_poke.c into a modunit_poke.elf binary, one would add:

    @$(CC) -o $(PREFIX)/test/modunit_poke.elf $(CFLAGS) -I. test/unit/modunit_pico_poke.c  -lcheck -lm -pthread -lrt $(UNITS_OBJ) $(PREFIX)/lib/libpicotcp.a

on the bottom of the Makefile units: target.

Running in the CI

The main script to run the unit tests is located in test/units.sh. In order to integrate the new unit test, add a line like

./build/test/modunit_poke.elf || exit 1

among the existing tests.

This will ensure that the unit test is run by Jenkins upon each future commit, and you can get a report on regressions whenever the module is changed.