Initial version of basic service worker

[Nooshu]

Draft PR created for the initial work to add a service worker to the parliament petitions website.

User Need

As has been shown in the past couple of weeks, when a popular petition is live it can put the server and infrastructure under extreme load. Many times this could be because a user is revisiting the website to look at the updated number count. Under these conditions these users can be requesting assets from the server multiple times, effectively DDoSing the server. By using the service worker API we have very fine-grain control over the caching of assets.

If a CacheFirst strategy is used, once assets are cached the requests to the server will never be made. The service worker will intercept the request and serve it directly from the service worker cache. NOTE: This is different to a standard browser cache which will periodically purge itself (depending on many different factors).

The use of a service worker can also open up other areas of improvement for users including:

• offline access
• a fully fledged PWA with add to home screen functionality (A2HS)
• background synchronisation of data
• ability to subscribe to notifications when a petition is updated

All this functionality can be considered in the future.

The primary purpose for this PR is to improve web performance on both desktop and mobile devices. According to Caniuse 86.88% of users in the UK use a browser that supports for API. Since this is a progressive enhancement, those who don't will see no difference but will still have full access to the website.

Implementation

The main focus of this PR is to work on the workflow integration into a RoR application. The two key technologies used are:

• Workbox.js
• serviceworker-rails

The basic serviceworker-rails setup has been modified to allow the use of Workbox and all the fine-grain control it allows us. As workbox is an npm module the workflow has needed to be changed slightly to generate the initial serviceworker.js.erbfile. npm start is used to generate the service worker file (from the workbox-config.js), and then start the rails application. NOTE: The config only needs to be run once, the compiled service worker file can then be committed.

Caching strategies

The following caching strategies have been used for the initial version:

• HTML files: NetworkOnly since we always want a user to have the freshest version of the HTML
• Analytics JS: NetworkOnly to avoid issues if these 3rd party files were to be modified
• count.json: NetworkOnly so a user will always receive the latest number count update from the server. StaleWhileRevalidate could be another option here, serve a cached version first then check with the server for an update (then update in the background).
• CSS / JS: CacheFirst so the browser will always check the service worker cache first. If found, a request to the network isn't required. If not it will fallback to the network.
• Image files: CacheFirst so the browser will always check the service worker cache first. If found, a request to the network isn't required. If not it will fallback to the network.

The exact strategies used and expiration times can be controlled. This seem like the safest setup for the initial implementation. CacheFirst works well in this instance because filenames are appended with a unique SHA. If a file changes the SHA will change and a new request will be created, (bypassing the service worker).

Screenshots (Chrome 73)

Service worker installed in DevTools application tab.

Cache used for CSS and JS.

Cache used for images.

Homepage assets being served from the service worker.

Potentail future plans

Create an offline version of the site

By caching the HTML it is possible to create a version of the site that will work offline. Should the server fail, a user that has previously visited the site will receive a cached version of the site (with an old submission count). In this case it would be worth telling a user they are viewing an offline version of the site and when the figure is from.

An advantage to this approach is all assets will be cached locally. A network request would only be required if the homepage is updated and when there is a request for a new count.json.

This would require a little bit of refactoring / design to include the offline notice and date.

TODO

• Make sure the serviceworker.js isn't being cached max-age=0, no-cache
• Is serving Workbox from the Google CDN cache an acceptable dependency? If not then Workbox can be served locally but would require RoR workflow updates.
• Is the current workflow acceptable for the development / production process?
• Question to only render application/service_worker in production (for easier development)?

[Nooshu]

Verified the serviceworker.js file is being served with the correct Cache-Control headers:

[pixeltrix]

@Nooshu thanks for the PR, about your comment regarding fitting into the current workflow. We are looking at some point in the near future upgrading both the app and infrastructure and one thing we'll be doing is upgrading to Rails 6.0 to take advantage of the native multi-database support so that we can use an Aurora PostgreSQL cluster. As part of that we'll probably switch over to using webpack via the webpacker gem and I was thinking it may be better to integrate this as part of that work - what do you think?

[Nooshu]

Hi @pixeltrix, yes I think that is a great idea. The current implementation does feel a little clunky. Almost like npm / workbox-cli has been crowbarred into the build process. Workbox.js actually comes with 2 webpack plugins: one that generates a complete service worker for you and one that generates a list of assets to precache that is injected into a service worker file.

It would require some reconfiguring to get working but the base workbox-config.js can be used as a guide.