Skip to content

Commit

Permalink
Update README.md (#111)
Browse files Browse the repository at this point in the history
  • Loading branch information
linuskendall authored Aug 28, 2024
1 parent 513161d commit 097b9a8
Showing 1 changed file with 0 additions and 107 deletions.
107 changes: 0 additions & 107 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,113 +110,6 @@ Flags:
- `--verify`: Verify the indexes after generation. (optional)
- `--network=<network>`: Which network to use for the gsfa index. Defaults to `mainnet` (other options: `testnet`, `devnet`). (optional)

## RPC server proxying

The RPC server provides a proxy mode which allows it to forward traffic it can't serve to a downstream RPC server. To configure this, simply provide the command line argument `--proxy=/path/to/faithful-proxy-config.json` pointing it to a config file. The config file should look like this:

```json
{
"target": "https://api.mainnet-beta.solana.com",
"headers": {
"My-Header": "My-Value"
},
"proxyFailedRequests": true
}
```

The `proxyFailedRequests` flag will make the RPC server proxy not only RPC methods that it doesn't support, but also retry requests that failed to be served from the archives (e.g. a `getBlock` request that failed to be served from the archives because that epoch is not available).

### Log Levels

You can set the desired log verbosity level by using the `-v` flag. The levels are from 0 to 5, where 0 is the least verbose and 5 is the most verbose. The default level is 2.

Example:

```bash
faithful-cli rpc -v=5 455.yml
```

### RPC server from old-faithful.net

We are hosting data on old-faithful.net for testing and cloning purposes. This allows you to run a sample test server without downloading any data. You can run a fully remote server like this:

```
$ ./tools/run-rpc-server-remote.sh 0
```

This will create a server that hosts epoch 0.

### RPC server with local indexes

For ongoing testing, we strongly recommend that you download at least the indexes for best performance. If you have local indexes downloaded you can use the following helper script:

```
$ ./tools/run-rpc-server-local-indexes.sh 0 ./epoch0
```

There is a utility script in the `tools` folder that will download the indexes hosted on old-faithful.net. The indexes will also be available on Filecoin.

```
$ mkdir epoch0
$ cd epoch0
$ ../tools/download-indexes.sh 0 ./epoch0
$ ../tools/download-gsfa.sh 0 ./epoch0
```
### RPC server running fully locally
If you have a local copy of a CAR archive and the indexes and run a RPC server servicing data from them. For example:
```bash
/usr/local/bin/faithful-cli rpc \
--listen $PORT \
/path/to/epoch-455.yml
```

You can download the CAR files either via Filecoin or via the bucket provided by Triton. There are helper scripts in the `tools` folder. To download the full epoch data:

```bash
$ mkdir epoch0
$ cd epoch0
$ ../tools/download-epoch.sh 0
$ ../tools/download-indexes.sh 0
$ ../tools/download-gsfa.sh 0
```

Once files are downloaded there are also utility scripts to run the server:
```bash
$ ./tools/run-rpc-server-local.sh 0 ./epoch0
```

This will host epoch 0 from the data available in the folder epoch0.

### Filecoin RPC server

The filecoin RPC server allows provide getBlock, getTransaction and getSignaturesForAddress powered by Filecoin. This requires access to indexes. The indexes allow you to lookup transaction signatures, block numbers and addresses and map them to Filecoin CIDs.

You can run it in the following way:

```bash
faithful-cli rpc 455.yml
```

The config file points faithful to the location of the required indexes (`455.yaml`):
```bash
indexes:
slot_to_cid: './epoch-455.car.bafyreibkequ55hyrhyk6f24ctsofzri6bjykh76jxl3zju4oazu3u3ru7y.slot-to-cid.index'
sig_to_cid: './epoch-455.car.bafyreibkequ55hyrhyk6f24ctsofzri6bjykh76jxl3zju4oazu3u3ru7y.sig-to-cid.index'
sig_exists: './epoch-455.car.bafyreibkequ55hyrhyk6f24ctsofzri6bjykh76jxl3zju4oazu3u3ru7y.sig-exists.index'
gsfa: './epoch-455.car.gsfa.index'
```

Due to latency in fetching signatures, typically the getSignaturesForAddress index needs to be stored in a local directory, but the other indexes can be fetched via HTTP or via local file system access. If you provide a URL, you need to make sure that the url supports HTTP Range requests. S3 or similar APIs will support this.

There is a mode in which you can use a remote gSFA index, which limits it to only return signatures and not additional transaction meta data. In this mode, you can use a remote gSFA index. To enable this mode run faithful-cli in the following way:

```bash
faithful-cli rpc -gsfa-only-signatures=true 455.yml
```

### Filecoin fetch via CID

If you already know the CID of the data you are looking for you can fetch it via `faithful-cli fetch <cid>`. This requires no further indexes and can also be used to recursively fetch data for example for an epoch. To avoid fetching the full dataset for an epoch (100s of GB) you probably want to pass the parameter `--dag-scope=block` to fetch only the particular CID entity that you are interested in.
Expand Down

0 comments on commit 097b9a8

Please sign in to comment.