Skip to content

Commit

Permalink
patch envelopes Add documentation
Browse files Browse the repository at this point in the history
This commit changes following documents:

- `docs/PATCH-ENVELOPES.md` describing what Patch Envelopes are
- `docs/ECO-METADATA.md` describing how to use them from Edge Applications
- `pkg/pillar/persistcache/README.md` add missing link to Patch Envelopes doc

Signed-off-by: Pavel Abramov <[email protected]>
  • Loading branch information
uncleDecart authored and eriknordmark committed Oct 4, 2023
1 parent fd0c722 commit 2120f7b
Show file tree
Hide file tree
Showing 3 changed files with 119 additions and 16 deletions.
48 changes: 48 additions & 0 deletions docs/ECO-METADATA.md
Original file line number Diff line number Diff line change
Expand Up @@ -311,3 +311,51 @@ EVE is generating diagnostic output on a console (if there is one) which summari

This can be done using a GET to `/eve/v1/diag` endpoint.
The returned object is of Context-Type text - the same text which is sent to the console.

### Patch Envelope endpoints

Applications might want to get some updates/configurations in runtime, this can be done via Patch Envelopes.
More information on what Patch Envelopes are you can find in [PATCH-ENVELOPES.md](.PATCH-ENVELOPES.md) doc.
There are several endpoints which allow application to handle Patch Envelopes

Get list of available Patch Envelopes `/eve/v1/patch/description.json`
For example:

```bash
curl -X GET -v http://169.254.169.254/eve/v1/patch/description.json
[

{
"PatchId":"699fbdb2-e455-448f-84f5-68e547ec1305",
"BinaryBlobs":[
{
"file-name":"textfile1.txt",
"file-sha":"%FILE_SHA",
"file-meta-data":"YXJ0aWZhY3QgbWV0YWRhdGE=",
"url":"/persist/patchEnvelopesCache/textfile1.txt"
},
{
"file-name":"textfile2.txt",
"file-sha":"%FILE_SHA%",
"file-meta-data":"YXJ0aWZhY3QgbWV0YWRhdGE=",
"url":"/persist/patchEnvelopesCache/textfile2.txt"
}
],
"VolumeRefs":null
}

]
```

Files represented in BinaryBlobs section can be downloaded via this endpoint
`/eve/v1/patch/download/{patch}/{file}`

Where `patch` is Patch Envelope uuid and `file` is file name of binary blob.
In example above files are `textfile1.txt` and `textfile2.txt`
For example:

```bash
curl -X GET http://169.254.169.254/eve/v1/patch/download/699fbdb2-e455-448f-84f5-68e547ec1305/textfile1.txt

%base64-encoded file contents%
```
36 changes: 36 additions & 0 deletions docs/PATCH-ENVELOPES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Patch Envelopes

## Overview

Patch Envelopes are objects which are exposed to app instances via EVE meta-data server.
This objects can be useful to update any kind of information in a secure, isolated manner
(for instance, configuration parameters) on app instance without the need of rebooting it.
Alternatively, one can create new image for app instance, upload it to EVE, purge and restart
application instance with the new image, even if there was a small change in some configuration
file. To top that there will be a down-time during reboot of app instance. To summarise it,
patch envelopes goal is to make fleet management of app instances easier.

## Patch Envelope structure

Patch Envelope are created on controller and propagated to EVE via protobuf's `EvePatchEnvelope` message.
It consists of:

- *uuid*: uinque identified to reffer Patch Envelope object
- *action*: way this object should be treated
- *artifacts*: array of binary artifacts related to this Patch Envelope
- *appInstIdsAllowed*: list of app instances ids that can access this Patch Envelope

And other fields, for more information about additional fields in protobuf message reffer to API definition [here](https://github.com/lf-edge/eve-api/blob/main/proto/config/patch_envelope.proto)

Binary artifacts are objects that app instance can download and use (for instance, configuration files).
This artifacts are *opaque* to EVE: information is just transferred, never parsed, decoded, etc.
Currently, there are two types of Binary artifacts: *Inline* and *External*.

*Inline artifacts* are small (less or equal than 100KB) base64-encoded (not encrypted) strings with
optional meta data. They are part of Edge Device configuration.
*External artifacts* are referrencining volumes created on EVE. Size of volumes is not limited to 10KB

## How to use Patch Envelopes

When Patch Envelopes are created on controller and exposed to EVE via API, app instance can access
Patch Envelopes available to it from meta-data server using API defined in [metadata server](.ECO-METADATA.md)
51 changes: 35 additions & 16 deletions pkg/pillar/persistcache/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,35 +2,54 @@

## Use-case

This package implements a persistance cache. The main use-case is to store data on filesystem and
This package implements a persistence cache. The main use-case is to store data on filesystem and
provide access to it via API. Limitation of current implementation is that data should be represented
as string. This was developed for configuration services feature in EVE. Since there is an option for
user to provide object to store via inline query, EVE has to store it even after reboot.
as string. This was developed for patch envelope feature in [EVE](https://github.com/lf-edge/eve/blob/master/docs/PATCH-ENVELOPES.md). Since there is an option for
user to provide object to store via inline query, EVE has to store it even after reboot.

From library user perspective this library works as `map[string]string` with respective files created for each key in root folder.

## API & Usage

Main structure is `pesistCache` outside of `persistcache` package it could be created via `Load` function:
it is loads values from cache if they are present or created folder if there was none.
`persistCache` structure has 3 operations:
Create `persistCache` structure by calling `New`

- *Get* value from in-memory cache. Note: lazy initialisation is used.
- *Put* value to in-memory cache and store it in filesystem
- *Delete* value from in-memory cache and filesystem
- *Close* persistCache, should be used before freeing resource
```golang
rootFilePath := "/my/root/file/path"
pc := persistcache.New(rootFilePath)
```

### Notes:
- persistCache create lock file which does not allow using same folder in different threads, goroutines, apps, before creating instance on the same resource one should *Close* first instance. In case of multithreading use it is advised to use same object (basic mutex is implemented, so persistCache is not lock-free object, but you stil can share it between goroutines)
- Lazy initalisation is used. That means no values are loaded during *New* call, rather *Get* will load value from filesystem if it is not in the cache
In case there are any objects (files) in specified `rootFilePath`, they will lazily initialized,
that means, they will actually be loaded only when `Get` function is called

Add objects to store by calling `Put`

```golang
pc.Put("myValidKey", []byte("myValidValue"))
```

check `isValidKey` and `isValidValue` to see limitations on valid keys and values.
Objects will be stored both in-memory and on filesystem.

Retrieve objects by calling `Get`

```golang
pc.Get("myValidKey")
```

Remove object from filesystem and from in-memory cache by calling `Delete`

```golang
pc.Delete("myValidKey")
```

## Design decisions

### Why we are storing separate files and not saving whole structure as file?

+ If objects stored are large it takes less time to save/update them and less code
+ Access to cache file is easier
+ Avoids a Put of one key from affecting the storage of another key, which could happen if it is a single file and the device is powered off before everything has been sync'ed to disk.

### Why store `[]byte` and not `interface{}` or `string`

### Why store `string` and not `interface{}`
This way library user bears responsibility of marshalling and unmarshalling object on his side, keeping this
library simple
This way library user bears responsibility of marshalling and unmarshalling object on its side, keeping persistcache library simple. In case of `string` it would imply a valid UTF-8 which is not required for `[]byte`.

0 comments on commit 2120f7b

Please sign in to comment.