-
Notifications
You must be signed in to change notification settings - Fork 145
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* StreamingFast docs on Substreams-powered subgraphs * added a title * Add SpS documentation to the menu * Fix format * Addressing feedback * editing the _meta.js * fix * trying to fix prettier * fixing prettier * Fix title * `sps-intro` => `introduction` * Duplicate new pages in every language --------- Co-authored-by: Enol Álvarez <[email protected]> Co-authored-by: benface <[email protected]>
- Loading branch information
1 parent
d3d4099
commit 5fffe03
Showing
100 changed files
with
4,858 additions
and
39 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
import meta from '../../en/sps/_meta.js' | ||
|
||
export default { | ||
...meta, | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
--- | ||
title: Introduction to Substreams-powered Subgraphs | ||
--- | ||
|
||
By using a Substreams package (`.spkg`) as a data source, your subgraph gains access to a stream of pre-indexed blockchain data. This enables more efficient and scalable data handling, especially with large or complex blockchain networks. | ||
|
||
There are two methods of enabling this technology: | ||
|
||
Using Substreams [triggers](./triggers): Consume from any Substreams module by importing the Protobuf model through a subgraph handler and move all your logic into a subgraph. This method creates the subgraph entities directly in the subgraph. | ||
|
||
Using [Entity Changes](https://substreams.streamingfast.io/documentation/consume/subgraph/graph-out): By writing more of the logic into Substreams, you can consume the module's output directly into graph-node. In graph-node, you can use the Substreams data to create your subgraph entities. | ||
|
||
It is really a matter of where you put your logic, in the subgraph or the Substreams. Keep in mind that having more of your logic in Substreams benefits from a parallelized model, whereas triggers will be linearly consumed in graph-node. | ||
|
||
Visit the following links for How-To Guides on using code-generation tooling to build your first end-to-end project quickly: | ||
|
||
- [Solana](https://substreams.streamingfast.io/documentation/how-to-guides/intro-your-first-application/solana) | ||
- [EVM](https://substreams.streamingfast.io/documentation/how-to-guides/intro-your-first-application/evm) | ||
- [Injective](https://substreams.streamingfast.io/documentation/how-to-guides/intro-your-first-application/injective) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,137 @@ | ||
--- | ||
title: 'Tutorial: Set Up a Substreams-Powered Subgraph on Solana' | ||
--- | ||
|
||
## Prerequisites | ||
|
||
Before starting, make sure to: | ||
|
||
- Complete the [Getting Started Guide](https://github.com/streamingfast/substreams-starter) to set up your development environment using a Dev Container. | ||
- Be familiar with The Graph and basic blockchain concepts such as transactions and Protobufs. | ||
|
||
## Step 1: Initialize Your Project | ||
|
||
1. Open your Dev Container and run the following command to initialize your project: | ||
|
||
```bash | ||
substreams init | ||
``` | ||
|
||
2. Select the "minimal" project option. | ||
3. Replace the contents of the generated `substreams.yaml` file with the following configuration, which filters transactions for the Orca account on the SPL token program ID: | ||
|
||
```yaml | ||
specVersion: v0.1.0 | ||
package: | ||
name: my_project_sol | ||
version: v0.1.0 | ||
imports: # Pass your spkg of interest | ||
solana: https://github.com/streamingfast/substreams-solana-spl-token/raw/master/tokens/solana-spl-token-v0.1.0.spkg | ||
modules: | ||
- name: map_spl_transfers | ||
use: solana:map_block # Select corresponding modules available within your spkg | ||
initialBlock: 260000082 | ||
- name: map_transactions_by_programid | ||
use: solana:solana:transactions_by_programid_without_votes | ||
network: solana-mainnet-beta | ||
params: # Modify the param fields to meet your needs | ||
# For program_id: TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA | ||
map_spl_transfers: token_contract:orcaEKTdK7LKz57vaAYr9QeNsVEPfiu6QeMU1kektZE | ||
``` | ||
|
||
## Step 2: Generate the Subgraph Manifest | ||
|
||
Once the project is initialized, generate a subgraph manifest by running the following command in the Dev Container: | ||
|
||
```bash | ||
substreams codegen subgraph | ||
``` | ||
|
||
You will generate a`subgraph.yaml` manifest which imports the Substreams package as a data source: | ||
|
||
```yaml | ||
--- | ||
dataSources: | ||
- kind: substreams | ||
name: my_project_sol | ||
network: solana-mainnet-beta | ||
source: | ||
package: | ||
moduleName: map_spl_transfers # Module defined in the substreams.yaml | ||
file: ./my-project-sol-v0.1.0.spkg | ||
mapping: | ||
apiVersion: 0.0.7 | ||
kind: substreams/graph-entities | ||
file: ./src/mappings.ts | ||
handler: handleTriggers | ||
``` | ||
|
||
## Step 3: Define Entities in `schema.graphql` | ||
|
||
Define the fields you want to save in your subgraph entities by updating the `schema.graphql` file. Here is an example: | ||
|
||
```graphql | ||
type MyTransfer @entity { | ||
id: ID! | ||
amount: String! | ||
source: String! | ||
designation: String! | ||
signers: [String!]! | ||
} | ||
``` | ||
|
||
This schema defines a `MyTransfer` entity with fields such as `id`, `amount`, `source`, `designation`, and `signers`. | ||
|
||
## Step 4: Generate Protobuf Files | ||
|
||
To generate Protobuf objects in AssemblyScript, run the following command: | ||
|
||
```bash | ||
npm run protogen | ||
``` | ||
|
||
This command converts the Protobuf definitions into AssemblyScript, allowing you to use them in the subgraph's handler. | ||
## Step 5: Handle Substreams Data in `mappings.ts` | ||
With the Protobuf objects generated, you can now handle the decoded Substreams data in your `mappings.ts` file found in the `./src` directory. The example below demonstrates how to extract to subgraph entities the non-derived transfers associated to the Orca account id: | ||
```ts | ||
import { Protobuf } from 'as-proto/assembly' | ||
import { Events as protoEvents } from './pb/sf/solana/spl/token/v1/Events' | ||
import { MyTransfer } from '../generated/schema' | ||
export function handleTriggers(bytes: Uint8Array): void { | ||
const input: protoEvents = Protobuf.decode<protoEvents>(bytes, protoEvents.decode) | ||
for (let i = 0; i < input.data.length; i++) { | ||
const event = input.data[i] | ||
if (event.transfer != null) { | ||
let entity_id: string = `${event.txnId}-${i}` | ||
const entity = new MyTransfer(entity_id) | ||
entity.amount = event.transfer!.instruction!.amount.toString() | ||
entity.source = event.transfer!.accounts!.source | ||
entity.designation = event.transfer!.accounts!.destination | ||
if (event.transfer!.accounts!.signer!.single != null) { | ||
entity.signers = [event.transfer!.accounts!.signer!.single.signer] | ||
} else if (event.transfer!.accounts!.signer!.multisig != null) { | ||
entity.signers = event.transfer!.accounts!.signer!.multisig!.signers | ||
} | ||
entity.save() | ||
} | ||
} | ||
} | ||
``` | ||
## Conclusion | ||
You’ve successfully set up a trigger-based Substreams-powered subgraph for a Solana SPL token. You can now further customize your schema, mappings, and modules to suit your specific use case. | ||
For more advanced customization and optimizations, check out the official [Substreams documentation](https://substreams.streamingfast.io/tutorials/solana). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,37 @@ | ||
--- | ||
title: Substreams Triggers | ||
--- | ||
|
||
Custom triggers allow you to send data directly into your subgraph mappings file and entities (similar to tables and fields), enabling full use of the GraphQL layer. By importing the Protobuf definitions emitted by your Substreams module, you can receive and process this data within your subgraph’s handler, ensuring efficient and streamlined data management within the subgraph framework. | ||
|
||
> Note: If you haven’t already, visit one of the How-To Guides found [here](./introduction) to scaffold your first project in the Development Container. | ||
The following code demonstrates how to define a `handleTransactions` function in a subgraph handler. This function receives raw Substreams bytes as a parameter and decodes them into a `Transactions` object. For each transaction, a new subgraph entity is created. | ||
|
||
```tsx | ||
export function handleTransactions(bytes: Uint8Array): void { | ||
let transactions = assembly.eth.transaction.v1.Transactions.decode(bytes.buffer).trasanctions // 1. | ||
if (transactions.length == 0) { | ||
log.info('No transactions found', []) | ||
return | ||
} | ||
|
||
for (let i = 0; i < transactions.length; i++) { | ||
// 2. | ||
let transaction = transactions[i] | ||
|
||
let entity = new Transaction(transaction.hash) // 3. | ||
entity.from = transaction.from | ||
entity.to = transaction.to | ||
entity.save() | ||
} | ||
} | ||
``` | ||
|
||
Here's what you’re seeing in the `mappings.ts` file: | ||
|
||
1. The bytes containing Substreams data are decoded into the generated `Transactions` object, this object is used like any other AssemblyScript object | ||
2. Looping over the transactions | ||
3. Create a new subgraph entity for every transaction | ||
|
||
To go through a detailed example of a trigger-based subgraph, [click here](./triggers-example). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
import meta from '../../en/sps/_meta.js' | ||
|
||
export default { | ||
...meta, | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
--- | ||
title: Introduction to Substreams-powered Subgraphs | ||
--- | ||
|
||
By using a Substreams package (`.spkg`) as a data source, your subgraph gains access to a stream of pre-indexed blockchain data. This enables more efficient and scalable data handling, especially with large or complex blockchain networks. | ||
|
||
There are two methods of enabling this technology: | ||
|
||
Using Substreams [triggers](./triggers): Consume from any Substreams module by importing the Protobuf model through a subgraph handler and move all your logic into a subgraph. This method creates the subgraph entities directly in the subgraph. | ||
|
||
Using [Entity Changes](https://substreams.streamingfast.io/documentation/consume/subgraph/graph-out): By writing more of the logic into Substreams, you can consume the module's output directly into graph-node. In graph-node, you can use the Substreams data to create your subgraph entities. | ||
|
||
It is really a matter of where you put your logic, in the subgraph or the Substreams. Keep in mind that having more of your logic in Substreams benefits from a parallelized model, whereas triggers will be linearly consumed in graph-node. | ||
|
||
Visit the following links for How-To Guides on using code-generation tooling to build your first end-to-end project quickly: | ||
|
||
- [Solana](https://substreams.streamingfast.io/documentation/how-to-guides/intro-your-first-application/solana) | ||
- [EVM](https://substreams.streamingfast.io/documentation/how-to-guides/intro-your-first-application/evm) | ||
- [Injective](https://substreams.streamingfast.io/documentation/how-to-guides/intro-your-first-application/injective) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,137 @@ | ||
--- | ||
title: 'Tutorial: Set Up a Substreams-Powered Subgraph on Solana' | ||
--- | ||
|
||
## Prerequisites | ||
|
||
Before starting, make sure to: | ||
|
||
- Complete the [Getting Started Guide](https://github.com/streamingfast/substreams-starter) to set up your development environment using a Dev Container. | ||
- Be familiar with The Graph and basic blockchain concepts such as transactions and Protobufs. | ||
|
||
## Step 1: Initialize Your Project | ||
|
||
1. Open your Dev Container and run the following command to initialize your project: | ||
|
||
```bash | ||
substreams init | ||
``` | ||
|
||
2. Select the "minimal" project option. | ||
3. Replace the contents of the generated `substreams.yaml` file with the following configuration, which filters transactions for the Orca account on the SPL token program ID: | ||
|
||
```yaml | ||
specVersion: v0.1.0 | ||
package: | ||
name: my_project_sol | ||
version: v0.1.0 | ||
imports: # Pass your spkg of interest | ||
solana: https://github.com/streamingfast/substreams-solana-spl-token/raw/master/tokens/solana-spl-token-v0.1.0.spkg | ||
modules: | ||
- name: map_spl_transfers | ||
use: solana:map_block # Select corresponding modules available within your spkg | ||
initialBlock: 260000082 | ||
- name: map_transactions_by_programid | ||
use: solana:solana:transactions_by_programid_without_votes | ||
network: solana-mainnet-beta | ||
params: # Modify the param fields to meet your needs | ||
# For program_id: TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA | ||
map_spl_transfers: token_contract:orcaEKTdK7LKz57vaAYr9QeNsVEPfiu6QeMU1kektZE | ||
``` | ||
|
||
## Step 2: Generate the Subgraph Manifest | ||
|
||
Once the project is initialized, generate a subgraph manifest by running the following command in the Dev Container: | ||
|
||
```bash | ||
substreams codegen subgraph | ||
``` | ||
|
||
You will generate a`subgraph.yaml` manifest which imports the Substreams package as a data source: | ||
|
||
```yaml | ||
--- | ||
dataSources: | ||
- kind: substreams | ||
name: my_project_sol | ||
network: solana-mainnet-beta | ||
source: | ||
package: | ||
moduleName: map_spl_transfers # Module defined in the substreams.yaml | ||
file: ./my-project-sol-v0.1.0.spkg | ||
mapping: | ||
apiVersion: 0.0.7 | ||
kind: substreams/graph-entities | ||
file: ./src/mappings.ts | ||
handler: handleTriggers | ||
``` | ||
|
||
## Step 3: Define Entities in `schema.graphql` | ||
|
||
Define the fields you want to save in your subgraph entities by updating the `schema.graphql` file. Here is an example: | ||
|
||
```graphql | ||
type MyTransfer @entity { | ||
id: ID! | ||
amount: String! | ||
source: String! | ||
designation: String! | ||
signers: [String!]! | ||
} | ||
``` | ||
|
||
This schema defines a `MyTransfer` entity with fields such as `id`, `amount`, `source`, `designation`, and `signers`. | ||
|
||
## Step 4: Generate Protobuf Files | ||
|
||
To generate Protobuf objects in AssemblyScript, run the following command: | ||
|
||
```bash | ||
npm run protogen | ||
``` | ||
|
||
This command converts the Protobuf definitions into AssemblyScript, allowing you to use them in the subgraph's handler. | ||
## Step 5: Handle Substreams Data in `mappings.ts` | ||
With the Protobuf objects generated, you can now handle the decoded Substreams data in your `mappings.ts` file found in the `./src` directory. The example below demonstrates how to extract to subgraph entities the non-derived transfers associated to the Orca account id: | ||
```ts | ||
import { Protobuf } from 'as-proto/assembly' | ||
import { Events as protoEvents } from './pb/sf/solana/spl/token/v1/Events' | ||
import { MyTransfer } from '../generated/schema' | ||
export function handleTriggers(bytes: Uint8Array): void { | ||
const input: protoEvents = Protobuf.decode<protoEvents>(bytes, protoEvents.decode) | ||
for (let i = 0; i < input.data.length; i++) { | ||
const event = input.data[i] | ||
if (event.transfer != null) { | ||
let entity_id: string = `${event.txnId}-${i}` | ||
const entity = new MyTransfer(entity_id) | ||
entity.amount = event.transfer!.instruction!.amount.toString() | ||
entity.source = event.transfer!.accounts!.source | ||
entity.designation = event.transfer!.accounts!.destination | ||
if (event.transfer!.accounts!.signer!.single != null) { | ||
entity.signers = [event.transfer!.accounts!.signer!.single.signer] | ||
} else if (event.transfer!.accounts!.signer!.multisig != null) { | ||
entity.signers = event.transfer!.accounts!.signer!.multisig!.signers | ||
} | ||
entity.save() | ||
} | ||
} | ||
} | ||
``` | ||
## Conclusion | ||
You’ve successfully set up a trigger-based Substreams-powered subgraph for a Solana SPL token. You can now further customize your schema, mappings, and modules to suit your specific use case. | ||
For more advanced customization and optimizations, check out the official [Substreams documentation](https://substreams.streamingfast.io/tutorials/solana). |
Oops, something went wrong.