Persist application's setup/config information

[mehrdadfeller]

We need a way to persist application setup/config information such as access keys, user specific configuration, as well as application data. This is a large task and we can potentially break it into smaller ones.

It should be possible also for the user to recover their system if for example MicroSD card burns; we either need a cloud-based storage or secondary storage (such as EEPROM or EMMc).

EEPROM is ideal for storing small amount of data (I believe currently its max size is 64KB but we can use a larger memory such as 128KB if necessary down the road). This would be great for config data or persisting access tokens.

We need to consider security implications and potentially perform encryption at rest.

[sassanh]

@mehrdadfeller We can use hashicorp vault for storing security-sensitive data, it has a docker image too: https://hub.docker.com/r/hashicorp/vault

We can implement 3 layers of security:

1. Minimum security: secrets are saved as raw data in the home directory, we just set the Linux file permissions and protect the file with SELinux.
2. A little more security: if user installs hashicorp vault docker image, we run it in dev mode and store secrets in it.
3. High security: We let user configure the vault in production mode and show them the QRCode of unseal key and initial root token, they store these keys either as qrcode or text in a secure storage (like iPhone's vault, etc). Then they need to unseal the vault either in the webui of the docker or by directly entering the key as qrcode.

For user configuration we can store them in the home directory (XDG_CONFIG_HOME)

We can provide configuration options for enabling back up for their config and/or secrets in the EEPROM or a secondary storage of their choice, we can add support for cloud solutions later.

[sassanh]

OK, we now have the simplest implementation, which implements abstractions for storing config as well as storing secrets.
It also includes a dummy implementations for each of them, the config are saved in .config and the secrets are stored in a file with owner-only-access file permissions.

Next steps would be:

1. Implement EEPROM storage for config
2. Optionally let the user store their secrets in EEPROM too
3. Save the secrets in a vault
4. Use the hardware cryptography module to store the private key and use it as the vault's key

[sassanh]

fsspec provides an abstraction layer for file system and supports ssh-fs, Google Drive, S3, Google Cloud, Azure, Alibaba Cloud. We better use its interface since the beginning to make it easier to add cloud support in the future.

[mehrdadfeller]

I love the idea of using Hashicop Vault. I watched some of their high level videos a while ago but never used it in a project. fsspec would be also great for abstracting storage.

[zytegalaxy]

This is the datasheet for current EEPROM:

https://www.onsemi.com/products/timing-logic-memory/memory/eeprom-memory/cat24c64

Features

• Supports Standard, Fast and Fast−Plus I2C Protocol
• 1.7 V to 5.5 V Supply Voltage Range
• 32−Byte Page Write Buffer
• Hardware Write Protection for Entire Memory
• Schmitt Triggers and Noise Suppression Filters on I2C Bus Inputs
(SCL and SDA)
• Low Power CMOS Technology
• 1,000,000 Program/Erase Cycles
• 100 Year Data Retention
• Industrial and Extended Temperature Range

[sassanh]

That’s good news!
1,000,000 / 10 years gives us 100,000 writes per year, or roughly 300 writes per day, which is ~12 writes per hour.
So I think it should be safe to debounce writes in 5 minutes windows, on leading and trailing edges: first write after a while will take place immediately, next writes will be aggregated for 2 minutes and their last will be written to EEPROM.
We can use it as the default value for the debouncing window length and let the user change it in the settings to a value as low as 2 seconds.
Eventually we may consider different debouncing window lengths for different kinds of config, for example changes initiated by user directly interacting with the device have more priority than the ones caused by network events.