Testing

Introduction

Testing of Angular projects can be done on multiple levels.

Unit tests work on the lowest level and test small units, such as components and services. They are easy to implement and have fast execution times, so they can provide early feedback without incurring high cost.

UI integration tests operate on the running application, automating browser interaction, and hence test not only individual units but the integration of these units including UI. Backend APIs and external services are mocked. UI integration tests require more implementation effort than unit tests and have longer execution times.

System tests are UI tests that test the entire system, including backend APIs and external services. They provide feedback on whether the Angular application works as expected, embedded in the system with APIs and services. Unlike unit tests and UI integration tests, system tests require time to set up the infrastructure before executing the tests. They tend to be flakier because they depend on external dependencies and working infrastructure.

Typically, following the test pyramid, the proportion of tests should put unit tests at the base and have fewer tests the higher you get in the test pyramid layers. However, some people argue that unit tests do not provide the best cost-benefit ratio for web applications, and UI tests should be the focus. We do not give a recommendation here as this highly depends on the testing requirements of a project.

Unit Tests

For unit testing in Angular projects we use Karma as a test runner in combination with Jasmine as testing framework, because these are the default testing tools provided by Angular. There is extensive documentation on unit testing on the Angular website.

The test runner spawns an HTTP server and generates the test runner HTML file from the tests written in Jasmine (or other testing frameworks like Mocha, QUnit, Chai etc).

On the other hand, the testing framework provides a syntax for assertions in JavaScript / TypeScript. Some testing frameworks, e.g. Jest, include a test runner in their framework.

Testing Observables

Testing code with RxJS Observables is tricky. Whereas simple scenarios can be tested subscribing to the observable, asserting and then calling the done() callback, this approach has its limitations, especially when the observable emits multiple values or at a certain time. For this reason we recommend to use marble testing to have stable and readable tests with an intuitive visual representation of the stream.

In our example application you can find an example for a marble test.

UI Integration Tests

Overview

There are several frameworks on the market for automating UI integration tests for Angular web applications.

Cypress

• most used framework in Angular community
• first stable version in 2017
• own test runner and tool set

Playwright

• developed by Microsoft
• not based on WebDriver API
• pretty fast

Webdriver.io

• on the market since 2014
• based on WebDriver API
• big feature set with helper methods and syntactic sugar

TestCafé

• less used in the community
• not based on WebDriver API
• own URL proxy to emulate DOM API and JavaScript into the browser

Comparison

When choosing the right framework for testing, it always depends on your project needs. Consider the following:

• Which browsers/versions need to be supported?
• Is mobile support needed?
• Is the feature set sufficient to test all of the business logic?

Criterion	Cypress	Playwright	Webdriver.io	TestCafé
Browser support (and versions)	o	+	++	++
Mobile support	++	++	++	++
Feature set	+	++	++	+
Execution time	+	++	o	+
Future-proofness	+	+	+	o

Recommendation

Based on our experiences with the testing frameworks, we recommend Playwright for projects with low testing requirements (such as supporting only the most recent browser version) due to its fast execution and big feature set.

For projects with more extensive testing requirements, we recommend either Webdriver.io or Playwright. While Playwright offers fast execution times, Webdriver.io offers more testing capabilities as specified by the WebDriver protocol.

Both of these testing frameworks are used in our example application (WebdriverIO test - Playwright test).

Cross Browser and Cross Platform Testing

If there is a need in the project for testing different browsers, operating systems, and platforms you can use BrowserStack.

The service provides the ability to execute UI integration tests for the desired capabilities (combination of browser, OS and platform) on real remote devices.

Playwright and Webdriver.io can also be used with the BrowserStack Automate service.

Layout Tests

In case there is the need in the project for layout testing, especially for responsiveness testing, we can recommend the framework Galen.

It's a Java based framework for layout and responsiveness testing providing an own domain-specific language (DSL).

Expectations can be formulated as properties (e.g. height / width) of a certain DOM element and its relations to other elements, e.g., positioning, spacing etc.

Example of a Galen specification (gspec)

@objects
  header        app-product-master header
    title       h1
    subtitle    h2
 
== Header ==
 
  header:
    width 100 % of screen/width
 
  @on desktop
    header:
      height ~ 279 px
 
  @on mobile
    header:
      height 350 to 400 px
 
  header.title:
    centered horizontally inside header
 
  header.subtitle:
    centered horizontally inside header
    below header.title

See a Galen specification from the example application here.

Additionally, it is also possible to connect Galen with Browserstack to perform cross-platform and cross browser tests. Related configurations can be done within Galens DSL like in the following example:

Example of a Galen test

@@ set
  url                 http://bs-local.com:4200
  browserstack_hub    http://${browserstack.username}:${browserstack.key}@hub-cloud.browserstack.com/wd/hub
  desktop_size        1280x720
 
@@ parameterized
    | deviceName             | tags      | capabilities |
    | Google Pixel 3         | mobile    | --dc.device "Google Pixel 3" --dc.os_version "9.0" --dc.real_mobile true |
    | OS X Mojave Chrome 76  | desktop   | --size ${desktop_size} --browser chrome --dc.browser_version 76 --dc.os "OS X" --dc.os_version Mojave |
    | Win 10 Edge 18         | desktop   | --size ${desktop_size} --browser edge --dc.browser_version "18.0" --dc.os Windows --dc.os_version 10 |
Product page on Browserstack - ${deviceName}
  selenium grid ${browserstack_hub} --page ${url} --dc.project "angular-styleguide" --dc.browserstack.local true ${capabilities}
    check layout-tests/specs/product-master.gspec --include ${tags}

You can view the Galen test file in the example application repo.

Mock Servers

In the layer of UI Integration tests it is necessary to mock some dependencies, such as the own backend or other external services.

For this purpose, there are many frameworks for mocking an HTTP server. We are currently using and recommending two frameworks:

Mock HTTP Server

Can be used for simple scenarios.

Example HTTP Mock Server

mockHttpServer.on({
  method: 'GET',
  path: '/products',
  reply: {
    status: 200,
    headers: { ...defaultHeaders, ...noCachingHeader },
    body: JSON.stringify({ products: result, productCount: result.length }),
  },
  delay,
});

Mock Server

• Advantages:
• Client libraries already available (java, node/javascript/typescript)
• REST API for all other cases
• Many options for deployment (docker image, war file, inline testing by starting mock server together with test runner)
• Nice dashboard for debugging at runtime
• HTTPS support

Stubbing:

Example HTTP Mock Server

await httpServerMock.mockAnyResponse({
  httpRequest: {
    method: 'POST',
    path: '/Product',
  },
  httpResponse: {
    statusCode: 201,
    body: JSON.stringify({ id: 'productId' }),
  },
});

Verification:

Example HTTP Mock Server

await httpServerMock.verify({
  method: 'POST',
  path: '/Product',
  headers: expectedRequestHeaders,
  body: expectedRequestProduct,
});

Page Objects

The goal of using page objects is to abstract any page information away from the actual tests. Ideally, you should store all selectors or specific instructions that are unique for a certain page in a page object, so that you still can run your test after you've completely redesigned your page.

System Tests

For testing the entire system with all its dependencies, we are using the same frameworks as for UI integration tests, but without any mocking. Therefore, no separate recommendations are needed here.