Skip to content

Commit

Permalink
Updated docs
Browse files Browse the repository at this point in the history
  • Loading branch information
dominicletz committed Oct 1, 2024
1 parent a88ef54 commit b14c378
Show file tree
Hide file tree
Showing 3 changed files with 98 additions and 7 deletions.
54 changes: 52 additions & 2 deletions ARCHITECTURE_MS1.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,32 @@ On average the size of a diode message is 236 bytes. This includes meta messages

## Architecture

Overview of the canister interface:

```mermaid
graph TD
subgraph Canister[Diode Message Canister]
SM[Store Message]
GM[Get Messages]
GK[Get Messages Indexed by Encryption Key]
IX[Heap Indexes]
subgraph DS[Data Storage]
IR[Inbox Region]
KIR[Key Inbox Regions]
PR[Cipher Text Region]
end
end
C1[Client 1] -->|add_message| SM
C2[Client 2] -->|get_message_by_id| GM
C3[Client 3] -->|get_*_message_id_by_key| GK
SM -->|Write| DS[Stable Storage]
GM -->|Read| DS
GK -->|Read| DS
```

The canister will be made out of one actor implemented in Motoko. The actor will be responsible for all interactions with the outside world. To leverage the stable storage the new [Region Memory API](https://internetcomputer.org/docs/current/motoko/main/base/Region) will be used.

As of time of writing the ic allows storing 4gb "heap memory" as well as 400gb "stable storage" (https://internetcomputer.org/docs/current/developer-docs/smart-contracts/maintain/resource-limits). On upgrade of an actor the Motoko runtime will serialize "stable" structures into stable storage" and restore after the upgrade the state into "heap memory". Discussion on the forum indicates that this can be a costly process effectively limiting the maximum usable "heap memory" to 2gb as greater values lead to execution timeouts during code upgrades.
Expand Down Expand Up @@ -87,6 +113,20 @@ for a given destination key.

```mermaid
erDiagram
%% HEAP Maps
KEY_INBOX_MAP ||--o{ KEY_INBOX : maps
KEY_INBOX_MAP {
bytes24 key_id
Region region
uint64 end_offset
}
MESSAGE_INDEX ||--o{ INBOX : indexes
MESSAGE_INDEX {
bytes32 hash
uint32 message_id
}
%% REGION Data
KEY_INBOX ||--o{ INBOX : selects
KEY_INBOX {
uint32 id
Expand All @@ -97,20 +137,30 @@ erDiagram
uint32 id
uint32 timestamp
bytes24 destination
uint32 hash
uint32 offset
bytes32 hash
uint64 offset
uint32 len
}
PAYLOAD {
bytes cipher_text
}
%% Markers
HEAP_MARKER[HEAP_MEMORY]
REGION_MARKER[STABLE_MEMORY]
%% Relationships to markers
HEAP_MARKER ||--|{ KEY_INBOX_MAP : contains
HEAP_MARKER ||--|{ MESSAGE_INDEX : contains
REGION_MARKER ||--|{ KEY_INBOX : contains
REGION_MARKER ||--|{ INBOX : contains
REGION_MARKER ||--|{ PAYLOAD : contains
```

### Data Layout Considerations

- `inbox` (and thus `payload_region`) could be made later into a ring buffer by adding stable `head` and `tail` pointers to the actor. For this the `id` field is introduced now.
-

## Milestone 1

Expand Down
51 changes: 46 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,52 @@
# Diode Message Availability
# Diode Zone Availability

## Prerequisites
Diode is a serverless P2P collaboration suite. It offers chat, file sharing, network sharing and more features usually found in centralized cloud applications such as slack, microsoft teams or discord. With it's peer to peer model similiar to BitTorrent it's decentralized and resilient. But it also has a unique availability problem compared to centralized applications when no peers are online.

This project addresses this issue by defining the Diode Zone Availability canisters to keep Diode zones data available even if no diode peer is online. For this it's using the [IC](https://internetcomputer.org/) decentralized large storage capabilities.

Currently the cost for Storing 1GiB of data on the IC is ($5.36 USD/year)[https://internetcomputer.org/docs/current/developer-docs/gas-cost#storage]. With just a fraction of this cost availability for encrypted messages can be achieved for zones improving the Diode user experience by quite a lot.

It might also be interesting to store additional file data encrypted on the IC for zones. Zone metadata like avatar pictures, zone info, etc. would add to the user experience a lot while being small enough to be stored for a long time.

Further we would like to investigate the usage of the IC for storing all files and folders of a file sharing Diode zone. This would increase the cost quite a bit more but again might be useful for certain use cases.

# Architecture

The Diode App is structured into "Zones" each zone has it's own set of owners, members, messages, files, etc. Zones are fully isolated from each other. To map this correspondingly to the IC we define one Zone Availability Canister (ZAC) per zone. This ZAC is storing all the data for one zone.

![Diode Zone Availability Architecture](docs/diode_drive.png)

Diagram showing the zone availability canister and multiple client apps uploading and downloading zone data:

```mermaid
graph TD
subgraph IC[Internet Computer]
ZAC1[Zone 1 Availability Canister]
ZAC2[Zone 2 Availability Canister]
end
C1[Client App 1] <-->|Upload/Download| ZAC1
C2[Client App 2] <-->|Upload/Download| ZAC1
C3[Client App 3] <-->|Upload/Download| ZAC1
C3[Client App 3] <-->|Upload/Download| ZAC2
C5[Client App 2] <-->|Upload/Download| ZAC2
C6[Client App 3] <-->|Upload/Download| ZAC2
```

Detailed descrptions of the Architecture in each development milestone:

- [MS1: IC Capacity evaluation](./ARCHITECTURE_MS1.md) storing 100k encrytped messages in a canister and retrieving them.
- [MS2: Authentication](./ARCHITECTURE_MS2.md) only allowing Zone members to interact with the Canister.
- [MS3: Integration](./ARCHITECTURE_MS3.md) integration of the canisters into the Diode App.

## Build Prerequisites
This project requires an installation of:

- [x] mops https://cli.mops.one/
- [x] DFX version 0.22.0 or newer
- [x] Install the [IC SDK](https://internetcomputer.org/docs/current/developer-docs/setup/install/).
- nodejs >= 22.9
- mops https://cli.mops.one/
- DFX version 0.22.0 or newer
- The [Internet Computer SDK](https://internetcomputer.org/docs/current/developer-docs/setup/install/).

### Run the tests

Expand Down
Binary file added docs/diode_drive.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit b14c378

Please sign in to comment.