docs: updates

docs/client_guide.md

@@ -26,13 +26,13 @@ After determining the subfile CID, client should supply a local path for writing
 ### CLI example
 ```
 ➜  subfile-exchange git:(main) ✗ cargo run -p subfile-exchange downloader \
-   --ipfs-hash QmeaPp764FjQjPB66M9ijmQKmLhwBpHQhA7dEbH2FA1j3v \
+   --ipfs-hash QmHash \
    --indexer-endpoints http://localhost:5678,http://localhost:5677 \
-   --free-query-auth-token 'Bearer imfreeee' \
-   --mnemonic "sheriff obscure trick beauty army fat wink legal flee leader section suit" \
+   --free-query-auth-token 'Bearer auth_token' \
+   --mnemonic "seed phrase" \
    --chain-id 421614 \
    --verifier 0xfC24cE7a4428A6B89B52645243662A02BA734ECF \
-   --provider [arbitrum-sepolia-rpc-endpoint]
+   --provider "arbitrum-sepolia-rpc-endpoint"
 ```
 
 

docs/contracts.md

@@ -1,8 +1,13 @@
-# Contracts
+# On-chain Interactions
 
-Generated from https://github.com/graphprotocol/contracts at commit a667f7aba10d558af6485d34cf857204321e343c.
+## Network specification
 
-Copy and pasted `addresses.json` and abis is from `/build/abis`
+Note: Contract artifacts are generated from https://github.com/graphprotocol/contracts at commit a667f7aba10d558af6485d34cf857204321e343c. (location: `/build/abis`). Contract address book (`addresses.jon`) is copy and pasted from the same commit.
 
-> In theory, contract ABIs can be imported with a path using Etherscan or npmjs, with "online" / "abigen_online" feature
-> But... could not get the crate to find the file online, rip. So just importing the hardcoded files
+> In theory, contract ABIs can be imported with a path using Etherscan or npmjs, with "online" / "abigen_online" feature. But having issues to connect with this method. The current manual solution is only temporary.
+
+Currently we only utilizes `L2Staking` contract for allocations.
+
+## Available CLI commands
+
+[On-Chain Guide](onchain_guide.md)

docs/feature_checklist.md

@@ -10,9 +10,8 @@
   - [x] construct and write a chunk_file.yaml (root, nodes)
   - [x] Unit tests: same file same hash, different file different hash, big temp file same/modified
   - [x] last chunk lengths, 
-  - [ ] Analyze merkle tree vs hash list
-  - [ ] memory usage for hashing (profiling to O(n) where n is the size of the file)
-- [ ] Subfile builder / publisher - CLI
+- [ ] Analyze merkle tree vs hash list for memory usage and verification runtime
+- [x] Subfile builder / publisher - CLI
   - [x] Take a file, use File hasher to get the chunk_file, publish chunk_file to IPFS
     - [x] later, take a list of files, use File hasher to hash all files and get root hashes 
   - [x] Construct a subfile manifest with metainfo using YAML builder
@@ -54,7 +53,7 @@
     - [x] Construct response and respond
       - [ ] determine if streaming is necessary
   - [x] Start with free service and requiring a free query auth token
-    - [ ] then add default cost model, allow updates for pricing per byte
+    - [x] default pricing, allow updates for pricing per byte
   - [ ] Runs TAP agent for receipt management
 - [ ] Subfile Client 
   - [ ] Take private key/mneomic for wallet connections
@@ -70,11 +69,11 @@
       - [x] Parallel requests
       - [x] Use random endpoints
     - [x] Construct and send requests to indexer endpoints 
-      - [ ] Parallelize requests
+      - [x] Parallelize requests
       - [ ] Multiple connections (HTTPS over HTTP2)
   - [x] Wait for the responses (For now, assume that the response chunks correspond with the verifiable chunks)
     - [x] Keeps track of the downloaded and missing pieces, 
-    - [ ] continually requesting missing pieces until the complete file is obtained
+    - [x] continually requesting missing pieces until the complete file is obtained
     - [x] Upon receiving a response, verify the chunk data in the chunk_file
       - [x] if failed, blacklist the indexer
   - [x] Once all file has been received and verified, terminate

docs/onchain_guide.md

@@ -56,23 +56,23 @@ Grab the IPFS hash of the deployment you want to allocate to and decide the allo
     --provider "http://localhost:8545" \
     allocate \
         --tokens 100 \
-        --deployment-ipfs QmeaPp764FjQjPB66M9ijmQKmLhwBpHQhA7dEbH2FA1j3v \
+        --deployment-ipfs QmHash \
         --epoch 100
 ```
 
 With RUST_LOG turned on, you can expect the following logs upon success
 ```
-  2024-01-08T18:17:34.123941Z  INFO subfile_exchange::transaction_manager: Initialize transaction manager
+  INFO subfile_exchange::transaction_manager: Initialize transaction manager
     at subfile-exchange/src/transaction_manager/mod.rs:32
 
-  2024-01-08T18:17:34.650044Z  INFO subfile_exchange::transaction_manager::staking: allocate params, dep_bytes: [241, 64, 71, 78, 218, 63, 159, 91, 130, 173, 178, 168, 30, 254, 183, 20, 225, 131, 35, 230, 52, 85, 74, 196, 40, 255, 173, 61, 144, 126, 223, 33], tokens: Some(Allocate(AllocateArgs { tokens: 256, deployment_ipfs: "QmeaPp764FjQjPB66M9ijmQKmLhwBpHQhA7dEbH2FA1j3v", 
+  INFO subfile_exchange::transaction_manager::staking: allocate params, dep_bytes: [241, 64, 71, 78, 218, 63, 159, 91, 130, 173, 178, 168, 30, 254, 183, 20, 225, 131, 35, 230, 52, 85, 74, 196, 40, 255, 173, 61, 144, 126, 223, 33], tokens: Some(Allocate(AllocateArgs { tokens: 256, deployment_ipfs: "QmeaPp764FjQjPB66M9ijmQKmLhwBpHQhA7dEbH2FA1j3v", 
   epoch: 101 })), allocation_id: 0x75e11e0f2319913c28d0b1916b4b1d9a1ac3977b, metadata: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], proof: Bytes(0xac1b36d68ea95ebe9f9793850cf083a031d40806121dc2dac525423c50611d18053f195627f6bffe036b9325b4dfd273959457b5d3f1c1b53095c096182756bb1b)
     at subfile-exchange/src/transaction_manager/staking.rs:67
 
-  2024-01-08T18:17:34.765769Z DEBUG subfile_exchange::transaction_manager::staking: estimate gas, estimated_gas: 379109
+  DEBUG subfile_exchange::transaction_manager::staking: estimate gas, estimated_gas: 379109
     at subfile-exchange/src/transaction_manager/staking.rs:82
 
-  2024-01-08T18:17:42.224872Z  INFO subfile_exchange: Allocation transaction finished, allocation_id: 0x75e11e0f2319913c28d0b1916b4b1d9a1ac3977b, tx_receipt: Some(TransactionReceipt { transaction_hash: 0x835b790326abf1555545920265e54d5bfbaba150aef31820529736e6727c7a0a, ... })
+  INFO subfile_exchange: Allocation transaction finished, allocation_id: 0x75e11e0f2319913c28d0b1916b4b1d9a1ac3977b, tx_receipt: Some(TransactionReceipt { transaction_hash: 0x835b790326abf1555545920265e54d5bfbaba150aef31820529736e6727c7a0a, ... })
     at subfile-exchange/src/main.rs:75
 ```
 
@@ -88,15 +88,15 @@ Grab the ID of allocation you want to close, and populate the unallocate subcomm
 
 You can expect logs as follows
 ```
-  2024-01-08T20:17:58.886745Z  INFO subfile_exchange::transaction_manager: Initialize transaction manager
+  INFO subfile_exchange::transaction_manager: Initialize transaction manager
     at subfile-exchange/src/transaction_manager/mod.rs:32
 
-  2024-01-08T20:17:59.355035Z  INFO subfile_exchange::transaction_manager::staking: unallocate params, allocation_id: 0xe37b9ee6d657ab5700e8a964a8fcc8b39cdefd73
+  INFO subfile_exchange::transaction_manager::staking: unallocate params, allocation_id: 0xe37b9ee6d657ab5700e8a964a8fcc8b39cdefd73
     at subfile-exchange/src/transaction_manager/staking.rs:142
 
-  2024-01-08T20:17:59.436442Z DEBUG subfile_exchange::transaction_manager::staking: estimate gas, estimated_gas: 390965
+  DEBUG subfile_exchange::transaction_manager::staking: estimate gas, estimated_gas: 390965
     at subfile-exchange/src/transaction_manager/staking.rs:154
 
-  2024-01-08T20:18:06.929268Z  INFO subfile_exchange: Transaction result, result: Ok((0xe37b9ee6d657ab5700e8a964a8fcc8b39cdefd73, Some(TransactionReceipt { transaction_hash: 0xd5c7c4d3dbd4aa8f845f87f8225aef91e927fe7cd5a1cd02085b0d30a59f4743, transaction_index: 1, block_hash: Some(0xcb46a88b2a37648a38165ca3740248b9a2a41e01f3b56f65f59b33f5cbf00fd0), block_number: Some(5738566), from: 0xe9a1cabd57700b17945fd81feefba82340d9568f, to: Some(0x865365c425f3a593ffe698d9c4e6707d14d51e08), cumulative_gas_used: 345329, gas_used: Some(345329), contract_address: None, logs: [Log { address: 0x865365c425f3a593ffe698d9c4e6707d14d51e08, topics: [...], data: Bytes(...), block_hash: Some(0xcb46a88b2a37648a38165ca3740248b9a2a41e01f3b56f65f59b33f5cbf00fd0), block_number: Some(5738566), transaction_hash: Some(0xd5c7c4d3dbd4aa8f845f87f8225aef91e927fe7cd5a1cd02085b0d30a59f4743), transaction_index: Some(1), log_index: Some(0), transaction_log_index: None, log_type: None, removed: Some(false) }], status: Some(1), root: None, logs_bloom: ..., transaction_type: Some(2), effective_gas_price: Some(100000000), other: OtherFields { inner: {"gasUsedForL1": String("0x28a70"), "l1BlockNumber": String("0x4d09a3")} } })))
+  INFO subfile_exchange: Transaction result, result: Ok((0xe37b9ee6d657ab5700e8a964a8fcc8b39cdefd73, Some(TransactionReceipt { transaction_hash: 0xd5c7c4d3dbd4aa8f845f87f8225aef91e927fe7cd5a1cd02085b0d30a59f4743, transaction_index: 1, block_hash: Some(0xcb46a88b2a37648a38165ca3740248b9a2a41e01f3b56f65f59b33f5cbf00fd0), block_number: Some(5738566), from: 0xe9a1cabd57700b17945fd81feefba82340d9568f, to: Some(0x865365c425f3a593ffe698d9c4e6707d14d51e08), cumulative_gas_used: 345329, gas_used: Some(345329), contract_address: None, logs: [...], status: Some(1), root: None, logs_bloom: ..., transaction_type: Some(2), effective_gas_price: Some(100000000), other: OtherFields { inner: {"gasUsedForL1": String("0x28a70"), "l1BlockNumber": String("0x4d09a3")} } })))
     at subfile-exchange/src/main.rs:88
 ```

docs/server_guide.md

@@ -10,9 +10,6 @@ Jump to [Quick Start](###getting-started)
 
 The server utilizes HTTP2 over HTTPS for secure and efficient file transfer. This ensures data integrity and confidentiality while leveraging the performance benefits of HTTP2.
 
-## Server Discovery (TODO)
-
-The server registers its endpoint on a designated smart contract (can be indexed by a subgraph). This allows clients to discover the server and initiate file transfers.
 
 ## Access Control
 
@@ -36,7 +33,7 @@ The server utilizes a combination of open-source technologies for optimal perfor
 - Backend: Rust for robust and efficient server-side processing.
 - Database: (Current: In-memory for server management, file paths for local access). 
 - Database: (TODO: PostgreSQL for persisted server management, generic storage paths to allow cloud/object storage). 
-- Smart Contract: (TODO) Solidity for secure and transparent server registration and discovery.
+- Smart Contract: Solidity for secure and transparent server registration and discovery.
 - User Interface: CLI to start up the server, HTTP requests for managing files and accessing receipts (TODO: Terminal UI).
 
 ## System Requirements
@@ -59,14 +56,14 @@ CLI example
 ✗ cargo run -p subfile-exchange server \
   --host 0.0.0.0 \
   --port 5678 \
-  --mnemonic "abondon abondon abondon abondon abondon abondon abondon abondon abondon abondon abondon abondon" \
+  --mnemonic "seed phrase" \
   --admin-auth-token "imadmin" \
   --free-query-auth-token "imafriend" \
-  --subfiles "QmY9aHuMqSPoLixVRdcYQei2cAtChBQNbjdtL5VzaQdFzw:./example-file/,IPFS_HASH:SUBFILE_PATH"
+  --subfiles "QmHash00:./example-file/,QmHash01:SUBFILE_PATH"
 ```
 Run `cargo run -p subfile-exchange --help` for more configurations and the corresponding ENV variable names.
 
-3. Access the server via the admin endpoint.
+3. Access the server via the **admin** endpoint.
 
 HTTP request example to get, add, and remove subfile services
 ```
@@ -91,7 +88,9 @@ Subfile(s) added successfully%
 Subfile(s) removed successfully
 ```
 
-4. (TODO) Register the server endpoint on the smart contract.
+4. (TODO) Register the server endpoint on the smart contract. Currently we assume the service endpoint has been registered with indexer-agent (for subgraphs). 
+
+5. To be compatible with V1 and Scalar TAP, an indexer must maintain an allocation. This means the indexer should use the `wallet` subcommand to create allocations. Refer to [Onchain Guide](onchain_guide.md).
 
 You are open for business!