Decision Records

Below is a list of decisions taken on the project, with justification.

Date	Who	Decision	Justification
05/07/24	Team	Pollutant values of zero, 9999 and negative values will not be stored in the database	There is a high occurrence of zero values, we believe that is sometimes used for no data. For 9999, this is significantly higher than any observed value, generally readings don't go above several hundred, and very rarely breaches 1000. We have also seen in the OpenAQ code an example of 9999 being used to signify that no data exists, a practice known as Not a Number (NaN) (see in the OpenAQ code here). Some negative values are very negative, some are very common (e.g. -1) and some are potentially due to calibration drift - during the weekly meeting on 5th July 2024 it was decided that we'd ignore all negative values.
04/07/24	Rich	Two ETL Processes will be used as opposed to one	We have decided to use two ETL scripts, as due to the times the data is available we want to have different cron jobs calling them. This also has the benefit of ensuring one breaking doesn't affect the other in the event that happens
11/06/24	Team	UI times will all be in UTC	Given it's an international application all times will be displayed in UTC
11/06/24	Team	Open AQ measurements in ppm will be used	After discussions with the mentors we will use the PPM measurements and put in place some functionality to convert
04/06/24	Ajay	React Query will be used in UI for managing API data fetching, caching. Redux will not be used	This app is very simple and does not require the overhead and complexity of Redux for state management
31/05/24	Team	Apache ECharts will be used as the front end charting library	After a POC, it was decided ECharts offered all the capabilities needed and had a wide range of charts/options for future proofing
30/05/24	Team	The ETL processes, API, database and UI will all be run in Docker containers on the ECMWF provided host	Docker allows us easy portability to any other infrastructure and is widely used and understood already
29/05/24	Team	OpenAQ measurements in ppm units are not used	ppm measurements were found to make up a much lower proportion of the measurements received from OpenAQ and didn't have a huge impact on the overall data found for cities. As it is fairly complex to convert ppm to µg/m³, we have discarded these results for now.
20/05/24	Team	In situ measurements are stored without AQI or aggregation	Originally the idea was to store in situ measurements for each city in the same way as forecast data, i.e. aggregated to calculate an AQI for each city from many measurements. This aggregation will now be done dynamically when requested from front end as this will allow users to dynamically select which measurements make up a location's AQI
22/04/24	Ajay	MongoDB will be used as the primary database	Stored data is not relational and mongo will allow horizontal scalability through sharding if mentors wish to store data historically