Skip to content

Latest commit

 

History

History
303 lines (252 loc) · 14.8 KB

README.md

File metadata and controls

303 lines (252 loc) · 14.8 KB
Raw

AceUnit-3

Welcome to AceUnit, the Advanced C and Embedded Unit test framework. AceUnit is a comfortable (test discovery!) unit test framework for C, in the best tradition of xUnit frameworks (like JUnit). Its target audience are developers using the C programming language to develop firmware, drivers, operating systems, and other C programs, like command line programs. It is tiny and thus suitable even for extremely resource-constraint environments.

AceUnit can also be used to study or teach Test-Driven Development. Although, you don't really need a framework for that in C, did you know that? A framework just helps making your life more comfortable.

The purpose of AceUnit is to be portable, small, and usable in resource-constraint environments as well as on PCs.

This is the third version of AceUnit, and a complete rewrite from scratch. The "template" for this version of AceUnit is no longer JUnit 4 but JUnit 5.

#include "leapyear.h"
#include <assert.h>

void testLeapYears(void) {
    assert(isLeapYear(0));
    assert(isLeapYear(4));
    assert(isLeapYear(400));
}

void testNonLeapYears(void) {
    assert(!isLeapYear(1));
    assert(!isLeapYear(100));
}

Attributes and Design Goals

  • JUnit 5.x-style naming.
  • Consumes only very little resources.
  • Works for hosted, freestanding, and even exotic freestanding environments.
  • Supports C89/C90, C99, C11, C17, and C23.
  • Configurable.
  • Can be run with pure C89/C90 and thus can be used in environments for which C++ is not available or used (i.e. 80x51).
  • Minimal framework noise in the test code. Especially, no macro noise and no fixture management noise.
  • Make use of existing C features from the hosted environment and POSIX, but without requiring a hosted environment or POSIX. It will also work just fine on the freestanding environment of an embedded controller.

Quick Guide

For each fixture (object file with test cases), AceUnit looks for functions that start with the following names:

  • beforeAll() (0‥1) One-time Setup
  • beforeEach() (0‥1) Setup per test case
  • test() (n) Test cases
  • afterEach() (0‥1) Teardown per test case
  • afterAll() (0‥1) One-time Teardown In addition to that, functions can be prefixed with a user-defined prefix. For example, if you have modules and use a prefix convention, you can use this in your tests, too. You could name functions like HeapTest_beforeAll(), HeapTest_beforeEach(), and HeapTest_test1() by adding -p HeapTest_ to the command line options of aceunit.

Compilers

This new version of AceUnit has been tested extensively using the following compilers:

  • GCC 11 w/ the following settings on x86_64: c90 c99 c11 c17 c2x gnu90 gnu99 gnu11 gnu17 gnu2x
  • GCC 12 as cross-compiler w/ the following target platforms: aarch64-linux-gnu, alpha-linux-gnu, arm-linux-gnueabi, hppa-linux-gnu, mips64-linux-gnuabi64, mips-linux-gnu, powerpc64le-linux-gnu, powerpc64-linux-gnu, powerpc-linux-gnu, riscv64-linux-gnu, s390x-linux-gnu (hppa64-linux-gnu, m68k-linux-gnu, sh4-linux-gnu, sparc64-linux-gnu exist as targets but are currently broken due to bugs in gcc, qemu, or both)
  • clang 14.0.6

The following compilers are planned to be tested soon:

  • GCC for hppa64, i686, m68k, mips, sh4, sparc64
  • Clang for aarch64, arm, avr, hexagon, mips, mips64, thumb, wasm32, wasm64
  • Keil / ARM ARMCC on ARM7, Cortex-M0, Cortex-M3, SC000, SC100, SC300
  • Keil C51 and C251 for 8051 and 80251
  • Keil C166 for Infineon C16x and STMicroelectronics ST10
  • Samsung ucc on Calm16 and SecuCalm
  • Open64

Tested Targets

Hosted environments

  • aarch64-linux-gnu
  • alpha-linux-gnu
  • amd64-unknown-openbsd7.2
  • arm-linux-gnueabi
  • hppa-linux-gnu
  • m68k-amigaos
  • mips64-linux-gnuabi64
  • mips-linux-gnu
  • powerpc64le-linux-gnu
  • powerpc64-linux-gnu
  • powerpc-linux-gnu
  • riscv64-linux-gnu
  • s390x-linux-gnu
  • x86-dos (bcc - Bruce's C Compiler, tested using dosbox)
  • x86_64-unknown-linux-gnu
  • x86_64-apple-darwin (Mac OS X; use aceunit.zsh instead of aceunit, see .github/workflows/clang-macos.yml)
  • x86_64-unknown-freebsd13.1
  • x86_64-unknown-haiku
  • x86_64-unknown-netbsd9.0 (NetBSD 9.3)

Note

Users on macOS should either change the interpreter in bin/aceunit from bash to zsh or install a newer bash, as /bin/bash in macOS is extremely old and does not support the needed constructs. The ./configure.sh script does exactly that, it looks at your OS and available shells, and patches a supported shell into bin/aceunit. The shell script in bin/aceunit works for both, bash and zsh, but requires Bash 5 when using bash.

Note

If you use bcc for x86-dos, run aceunit with the flags -t nm -b nm86 -s _.

How to Build

(In these instructions, replace make with your actual GNU make command. This is typically make on GNU/Linux systems, and gmake on other systems.) To build and run AceUnit, you need a GNU bash shell, GNU make, and a C compiler with objdump, readelf, or nm to extract symbol tables from object files. To build AceUnit, simply run make. This builds and tests AceUnit.

AceUnit defaults to objdump. On some BSD, like FreeBSD, objdump is not installed by default. If you want to stick to the default, install the binutils package (which includes objdump). Alternatively, you can remove the objdump.ac module from share/aceunit, or use -t nm or -t readelf when running aceunit.

How to Install

If you want to use AceUnit for testing on your POSIX system, simply run make && sudo make install. This will install AceUnit into /usr/local/. You can override the installation location using the PREFIX variable, like this: sudo make install PREFIX=/opt/aceunit/. The PREFIX variable only has an effect during installation, you do not need to rebuild.

The same way, you can remove a previous installation by running sudo make uninstall.

How to build with a different compiler.

By default, AceUnit will be built with/for the C compiler that your make tool uses, usually whatever cc is found on the PATH. If you want to build and test with a different compiler, you can use make CC=compilername, for example, make CC=clang or make CC=armcc.

You can also easily cross-compile for multiple targets in parallel. The Makefile in aceunit/lib can be used from other directories. See test/cross-hosted/ for examples of how that works.

Runners

AceUnit provides different runners for different needs. Out of the box, AceUnit comes with 4 runners: Simple, SetJmp, Abort, and Fork. AceUnit is well-documented, it should be easy to create your own runner if necessary.

SimpleRunner

The SimpleRunner just executes the tests without any special handling. If a test case fails, the runner stops.

The SimpleRunner is good to get started and for learning and practicing TDD.

For real-life projects, the SimpleRunner is less suited. As it will stop at the first error, there will be no information about the total number of test cases and errors. But do not fret, there are other runners.

SetJmpRunner

The SetJmpRunner uses setjmp() / longjmp() to intercept failed assertion. The assert() macro is defined to call AceUnit_fail() when the condition fails. And AceUnit_fail() performs a longjmp() back.

This runner works in freestanding environments as long as setjmp()/longjmp() are available. According to the C standard, freestanding environments are not required to provide <setjmp.h>. However, most freestanding environments do so.

AbortRunner

The AbortRunner is like the SetJmpRunner and additionally has a signal handler to catch the abort() signal. If a test case fails by raising SIGABRT, the runner will catch it. If you use <assert.h> provided by the system/compiler, it will usually call abort(), and that will usually raise SIGABRT.

ForkRunner

The ForkRunner uses fork() to execute test cases in child processes. A test case is marked failed if the child ended due to a signal, or if it exited with an exit value other than EXIT_SUCCESS (0).

Writing your own Runner

If the runners provided out of the box do not suit you, you can simply write your own. Just look at the source code of the existing runners to get inspired. They're simple and should be easy to understand.

Workflow

  1. Build AceUnit. Skip this step if you've installed AceUnit in your system and you're testing for the same system.
  2. Build your object files as usual.
  3. Build your test object files (can be included in the previous step).
  4. Run aceunit on the test object files to generate the fixtures source code.
  5. Build your fixtures object file.
  6. Build a test executable linking the object files together with the aceunit library of your choice.
  7. Run the tests.

Here's an example of how such a Workflow could look like on the command-line:

# Build AceUnit
$ cd aceunit
$ make
# Build your object files and test object files
$ cd ../myproject
$ cc -I ../aceunit/include/ -c *.c
# Run aceunit on the test object files to generate the fixtures source code.
$ ../aceunit/bin/aceunit *_test.o >testcases.c
# Build your fixtures object file.
$ cc -I ../aceunit/include/ -c testcases.c
# Build a test executable linking the object files together with the aceunit library of your choice.
$ cc *.o ../aceunit/lib/libaceunit-abort.a
# Run the tests
$ ./a.out
AceUnit: 2 test cases, 2 successful, 0 failed.

Here's an example of a Makefile that implements such a workflow:

ACEUNIT_HOME=../aceunit
ACEUNIT_LIBRARY=$(ACEUNIT_HOME)/lib/libaceunit-abort.a
CPPFLAGS+=-I $(ACEUNIT_HOME)/include

.PHONY: all
all: test

.PHONY: test
test: leapyear_test
	./$^

leapyear_test: leapyear_test.o leapyear.o testcases.o $(ACEUNIT_LIBRARY)

$(ACEUNIT_LIBRARY):
	$(MAKE) -C $(dir $(ACEUNIT_LIBRARY))

testcases.c: leapyear_test.o
	$(ACEUNIT_HOME)/bin/aceunit $^ >$@

.PHONY: clean
clean::
	$(RM) *.[adios] *.bc leapyear_test testcases.c

Assertion/Failure Features

There are multiple ways how you can make a test-case fail. The following table shows which type of assertion is supported by which type of runner.

Assertion SimpleRunnerSetJmpRunnerAbortRunnerForkRunner
<stdlib.h> longjmp() no yes yes no
<assert.h> assert() stop no yes yes
<stdlib.h> abort() stop no yes yes
<stdlib.h> exit() stop no no yes
<AceUnit.h> assert() stop yes yes yes
abnormal test case termination (like SIGSEGV)no no no yes

Legend:

  • "no" means that this runner doesn't support the specified assertion type. This means that this assertion type will not result in a test case fail. It also means that the test execution will be aborted. For example, using exit(1) will simply abort test execution for all runners except for the ForkRunner.
  • "yes" means that this runner supports the specified assertion type. The assertion will result in a test case fail. And the runner will continue execution with the next test case.
  • "stop" means that this runner supports the specified assertion type. The assertion will result in a test case fail. However, the runner will not continue execution with the next test case but instead stop.

The ForkRunner does not support longjmp() as there is no need for this. Using longjmp() for assertions in a test case only serves the purpose of implementing a kind of exception handling in the absence of proper exception handling. The ForkRunner uses POSIX fork() for exception handling, so using longjmp() or catching SIGABORT as a workaround to have exception handling in C is not required.

Test Fixtures

What is different from previous versions?

The whole code has been rewritten from scratch, test-driven and with ease-of-use on mind. The nomenclature has been updated to match that of JUnit5. The generator has been changed from a Java program to a shell script. Also, the generator no loner works on the source file, which is fragile. Instead, the generator uses tools like objdump, nm, or readelf to extract the symbol table and thus introspect the code to discover fixtures and test cases.

Trouble Shooting

Error about declare

If you see the following error:

./aceunit: line 3: declare: -g: invalid option

It means that you're running the aceunit shell script with an unsupported shell. If you're using macOS, open the aceunit shell script and replace #!/usr/bin/env bash with #!/usr/bin/env zsh. AceUnit requires Bash 5 or zsh 5.

Glossary

AfterAll
A function that AceUnit shall run once after it runs any of the test cases of a fixture.
AfterEach
A function that AceUnit shall run after each of the test cases of a fixture.
Assertion
A piece of code that verifies expectations inside a test function, and if the expectation isn't met, aborts the test case and reports back to the runner.
BeforeAll
A function that AceUnit shall run once before it runs any of the test cases of a fixture.
BeforeEach
A function that AceUnit shall run before each of the test cases of a fixture.
Fixture
An object file with test cases.
Runner
The part of AceUnit that executes test functions by calling them.
Test Case
A function that AceUnit should execute as a test case.
Test Function
Any of these: AfterAll, AfterEach, BeforeAll, BeforeEach, Test Case.

Dropped Features

The previous versions of AceUnit had a number of features that have been deliberately dropped to keep things simple:

Annotation-style processing
Previous versions of AceUnit used annotation-style processing like A_Test, A_BeforeClass, and so on. This was a bit fragile and required an extra parser. Annotations will come back once C has annotations. They're on the way, give it a few more years. This also made it possible to get rid of Java for the generator and replace it with a simple shell script.
Multiple fixture methods
Previous versions of AceUnit allowed for multiple setup and teardown functions per fixture. I don't think this was really used. This has been dropped to save a bit of memory and to make things less complicated.