-
Notifications
You must be signed in to change notification settings - Fork 0
Integration Tests
Automating test at integration level is not easy. However because of complexity of the uAAL platform it was decided that some mechanisms of automation are needed at this level for purpose of effective error detection and code quality assurance. The following requirements for this mechanism were specified in the course of work on D2.3:
- Configuration of integration test should be simple and easy to maintain, even with frequent development changes (e.g. artifact versions changes, component releases).
- Integration tests of given artifact should be executed during each artifact build.
- It should be possible to execute integration tests in the scope of Continuous Integration.
In itests, it is assumed that a given integration test belongs to only one software artifact. Additionally, this software artifact is always a maven project which is packaged into an OSGi bundle. As a consequence, the uAAL platform is a set of bundles which are started in an OSGi container. An integration test is a logic which is added to its software artifact (bundle) only for the test’s time being. For purpose of launching the test, an ad-hoc OSGi container is started (currently Apache Felix) and the tested artifact together with all its dependencies (bundles) is started. Then, the test’s logic is executed “from inside” its bundle (tested software artifact). Thanks to that the logic can operate on all the bundle´s classes and not only the exported packages. Developer has to be aware that during test execution, all mechanisms of OSGi container are available, and therefore all parts of implementation which could not be tested in regular unit tests can be thoroughly tested now. Additionally, all dependency bundles are started and are ready for use; therefore if for example, a given artifact needs the service bus to perform some communication, it can be done in the course of the integration test without any problem.
Created framework was designed in a way that it can be easily integrated with uaal-maven-plugin (http://forge.universaal.org/wiki/support:Composites_Generation). Uaal-maven-plugin generates a file artifact-test.composite which can be used by itests to read from there a list of bundles (in appropriate order) which should be started in the ad-hoc OSGi container launched for the test. Thanks to that, the first requirement (configuration and maintenance ease) can be achieved – thanks to uaal-maven-plugin developer does not need to update launch configurations for the integration test. Exposing integration test as JUnit ensures that the test can be executed in each maven build (all JUnits placed in the src/test/java folder are executed in the maven test phase). As a natural consequences, the tests will be automatically included in Continuous Integration, which performs the whole maven build for each uAAL artifact. Thanks to that, it is possible to fulfil requirements two and three.
In order to develop an integration test, one has to complete two tasks: First, configure maven project to be tested, second, write a test JUnit class. In the following paragraphs the POM file of maven project to be tested is simply referred to as the POM. The first task is very simple. Developer has to add the following snippet to the POM file:
<dependency> <groupId>org.universAAL.support</groupid> <artifactId>itests</artifactid> </dependency>
Please note that the version is not provided. It is possible thanks to the fact that this version is managed in the parent POM of middleware (in dependencyManagement section). Therefore if your POM (or its parent) imports the middleware POM (which is very often the case), then you do not need to specify the version. The important thing that has to be remembered is that dependency to itests should be located as the first one in the POM. Otherwise strange interferences with the OSGi framework can happen.
It is also possible that one of your parent POMs has already specified itests as dependency. In such case, the first configuration task can be skipped.
The second task comes to creating a class in which logic of integration testing will be implemented. This class has to be located in src/test/java and has to extend the following class form itests framework: org.universAAL.itests.IntegrationTest.
The default behaviour of a test is looking for artifact-test.composite file (in the target folder). If the file is found then bundles from it are started and the test is executed. Please note, that this default behaviour assumes that uaal-maven-plugin is properly configured in the project (refer to http://forge.universaal.org/wiki/support:Composites_Generation for guidelines for configuring uaal-maven-plugin). The easiest way to execute the test is just simply invoking “Run As JUnit” action in Eclipse. The other way is to invoke mvn test. Then the test will be executed together with other unit tests located in src/test/java.
If developer does not want, for some reasons, use the default itests behaviour which relies on uaal-maven-plugin or simply wants to customize it, he/she can invoke the following methods of IntegretionTest class (from which the test class is inheriting). It is most convenient to invoke these methods from the construction of the integration test class.
- setPaxArtifactUrls(final String... urls) – this method accepts a one or more URLs. Each URL can reference either composite or OSGi bundle. The composites (if provided) are recursively resolved and all gathered bundles are started in order in which they appear in the method’s parameter list (and also in referenced composite files). So the default itests behaviour is the same as developer would invoke setPaxArtifactUrls("file:target/artifact-test.composite").
- IntegrationTest(final String eclipseLaunchFile) – it is a constructor which can be invoked from the integration test class. The argument has to be a path to the eclipse launch configuration file which will be used for setting up the OSGi container for the test. Invoking this constructor indicates that provided eclipse launch configuration is used for:
- specifying set of bundles which should be started,
- specifying run arguments;
- specifying bundles.configuration.location (uAAL runtime configuration directory)
- setBundleConfLocation(final String path) – sets the path to bundles.configuration.location (uAAL runtime configuration directory).
- setRunArguments(final Properties p) – sets arguments for the universAAL platform (sample of such arguments can be looked up in the launch file of Lighting Example).
- setLogLevel(final int level) - Sets felix.log.level property which specifies the logging level of uAAL platform. Acceptable values are as follows (Default value is LOG_DEBUG):
- 1 - LOG_ERROR: Used for error messages
- 2 - LOG_WARNING: Used for warning messages.
- 3 - LOG_INFO: Used for informational messages
- 4 - LOG_DEBUG: Used for debug messages
- setUseOnlyLocalRepo(final boolean useOnlyLocalRepo) – Sets flag which can force mvn URL resolution to look up only local maven repository. Since by default all artifacts needed for testing are resolved by uaal-maven-plugin the default flag value is false.
- setIgnoreVersionMismatch(final boolean ignoreVersionMismatch) – If set to true than version mismatch between bundle version specified in the POM and bundle version specified in the manifest is ignored.
A sample integration test implemented for Lighting example is provided below:
/**
* LightingTest is an example of JUnit OSGi integration test. The TestCase uses
* LightingConsumer to verify if the lighting sample works correctly. Each
* integration TestCase has to extend MiddlewareIntegrationTest provided in the
* itests maven artifact. Thanks to using JUnit the TestCase will be executed
* during each maven build. However please mind that the tests are by default
* disabled in the main middleware pom file (mw.pom). To enable them an argument
* "-DskipTests=false" has to be added to the "mvn" invocation in the command
* line.
*
* @author rotgier
*
*/
public class LightingTest extends MiddlewareIntegrationTest {
/**
* Constructor of each integration TestCase has to call constructor of upper
* class providing path to the launch configuration and path to the
* configuration directory of the uAAL runtime. Launch configuration will be
* used to setup uAAL runtime for the purpose of TestCase. All bundles
* needed for the TestCase have to be included in the launch configuration.
*/
public LightingTest() {
super("../../pom/launches/LightingExample_Complete_0_3_2.launch",
"../../../itests/rundir/confadmin");
}
/**
* Helper method for logging.
*
* @param msg
*/
private void LOGInfo(String format, Object ... args) {
StackTraceElement callingMethod = Thread.currentThread().getStackTrace()[2];
String logMsg = null;
if (args != null) {
logMsg = String.format(format, args);
} else {
logMsg = format;
}
LogUtils.logInfo(Activator.mc, LightingTest.class, callingMethod.getMethodName(), new Object [] {logMsg}, null);
}
/**
* Verifies that runtime platform has correctly started. It prints basic
* information about framework (vendor, version) and lists installed
* bundles.
*
* @throws Exception
*/
public void testOsgiPlatformStarts() throws Exception {
LOGInfo("FRAMEWORK_VENDOR %s", bundleContext.getProperty(Constants.FRAMEWORK_VENDOR));
LOGInfo("FRAMEWORK_VERSION %s", bundleContext.getProperty(Constants.FRAMEWORK_VERSION));
LOGInfo("FRAMEWORK_EXECUTIONENVIRONMENT %s", bundleContext.getProperty(Constants.FRAMEWORK_EXECUTIONENVIRONMENT));
LOGInfo("!!!!!!! Listing bundles in integration test !!!!!!!");
for (int i = 0; i < bundleContext.getBundles().length; i++) {
LOGInfo("name: " + bundleContext.getBundles()[i].getSymbolicName());
}
LOGInfo("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
}
/**
* Verifies the lighting sample with the use of LightingConsumer. Following
* operations are tested: getControlledLamps, turnOn, turnOff, dimToValue.
*
* @throws Exception
*/
public void testLightingClient() throws Exception {
LOGInfo("!!!!!!! Testing Lighting Client !!!!!!!");
LOGInfo("!!!!!!! Getting controlled lamps and checking their amount !!!!!!!");
/* There should be four lamps available. */
Device[] controlledLamps = LightingConsumer.getControlledLamps();
Assert.isTrue(controlledLamps.length == 4);
int i = 0;
for (Device lamp : controlledLamps) {
LOGInfo("!!!!!!! Testing Lamp %s!!!!!!!", i);
String lampUri = lamp.getURI();
/* turnOn should end with success */
LOGInfo("!!!!!!! Testing Lamp %s turnOn!!!!!!!", i);
Assert.isTrue(LightingConsumer.turnOn(lampUri));
/* when repeated turnOn should also end with success */
LOGInfo("!!!!!!! Testing Lamp %s turnOn!!!!!!!", i);
Assert.isTrue(LightingConsumer.turnOn(lampUri));
/* turnOff should end with success */
LOGInfo("!!!!!!! Testing Lamp %s turnOff!!!!!!!", i);
Assert.isTrue(LightingConsumer.turnOff(lampUri));
/* when repeated turnOff should also end with success */
LOGInfo("!!!!!!! Testing Lamp %s turnOff!!!!!!!", i);
Assert.isTrue(LightingConsumer.turnOff(lampUri));
/* dimToValue with argument other than 0, 100 should fail */
LOGInfo("!!!!!!! Testing Lamp %s dimToValue 45!!!!!!!", i);
Assert.isTrue(!LightingConsumer.dimToValue(lampUri, 45));
/* dimToValue with argument 100 should end with success */
LOGInfo("!!!!!!! Testing Lamp %s dimToValue 100!!!!!!!", i);
Assert.isTrue(LightingConsumer.dimToValue(lampUri, 100));
/* dimToValue with argument 0 should end with success */
LOGInfo("!!!!!!! Testing Lamp %s dimToValue 0!!!!!!!", i);
Assert.isTrue(LightingConsumer.dimToValue(lampUri, 0));
i++;
}
}
}
The test was added to the http://forge.universaal.org/svn/support/trunk/samples/lighting/client smp.lighting.client project.
After launching the test its output should be visible in the console and it should be also logged to the log file of the universAAL platform. If the logging information related to the starting phase of universAAL middleware (installing and starting bundles) is not sufficient then a log4j.properties file with a following contents can be added to the src/test/resources directory:
log4j.logger.org.universAAL.samples.lighting.client.test=DEBUG, Appender1 log4j.appender.Appender1=org.apache.log4j.ConsoleAppender log4j.appender.Appender1.layout=org.apache.log4j.PatternLayout log4j.appender.Appender1.layout.ConversionPattern=%-4r %L [%t] %-5p %c %x - %m%n
However this additional logging information will be available only on the console (not the log file).