Substreams-Powered Subgraphs

Boost your Subgraph’s efficiency and scalability by using Substreams to stream pre-indexed blockchain data.

Use a Substreams package (.spkg) as a data source to give your Subgraph access to a stream of pre-indexed blockchain data. This enables more efficient and scalable data handling, especially with large or complex blockchain networks.

Enable Substreams

Enable Substreams-Powered Subgraphs for a blockchain network in Graph Node. See Deploy Graph Node.

Manage Substreams

Prerequisites

All commands should be executed as the root user to ensure proper installation and permissions.

Nodejs

Nodejs is required for dependency management and build steps in the subgraph process.

apt install nodejs

Substreams CLI

The Substreams CLI is used for connecting to blockchain endpoints, streaming real-time data, and packaging custom modules.

LINK=$(curl -s https://api.github.com/repos/streamingfast/substreams/releases/latest | \
awk "/download.url.*linux_$(uname -m | sed s/aarch64/arm64/)/ {print \$2}" | sed 's/"//g')

echo Downloading $LINK

curl -L $LINK | tar zxf -

mv substreams /usr/bin/substreams

Buf

Buf is used to generate typed structures from Protobuf definitions, simplifying integration across various languages.

An internet connection is required for generating Rust bindings.

1. Direct Download

LINK=$(curl -s https://api.github.com/repos/bufbuild/buf/releases/latest | \
awk "/download.url.*buf-Linux-$(uname -m)\"/ {print \$2}" | sed 's/"//g')

echo Downloading $LINK

curl -L $LINK -o /usr/bin/buf

1. Using npm

npm install @bufbuild/buf

Rust

A complete Rust environment is necessary for developing Substreams modules.

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

source $HOME/.cargo/env

Create Substreams-Powered Subgraphs

There are two methods of creating Substreams-Powered Subgraphs:

1. Using Substreams triggers: Consume from any Substreams module by importing the Protobuf model through a Subgraph handler and move all your logic into a Subgraph. This method creates the Subgraph entities directly in the Subgraph.

2. Using Entity Changes⁠: By writing more of the logic into Substreams, you can consume the module’s output directly into Subgraph. In Subgraph you can use the Substreams data to create your Subgraph entities.

You can choose where to place your logic, either in the Subgraph or Substreams. However, consider what aligns with your data needs, as Substreams has a parallelized model, and triggers are consumed linearly.

Deploy Substreams-Powered Subgraphs

Before deploying a substream-powered subgraph, ensure substream are enabled for the blockchain network in Graph Node.

1. Add Rust Target

rustup target add wasm32-unknown-unknown

1. Generate Protobuf Types

substreams protogen

This command generates types in /src/pb.

1. Build Substreams

substreams build

This command creates a Substream package (.spkg), which can be used as a data source for a subgraph to access pre-indexed blockchain data streams.

1. Generate Subgraph Project

substreams codegen subgraph

1. Navigate to the Subgraph Directory

Edit the following files as needed:

• mappings.ts

• schema.graphql

• subgraph.yaml

1. Install Dependencies

npm install

1. Generate AssemblyScript & Protobuf Bindings

npm run generate

1. Build Subgraph

npm run build

The final step is to deploy the subgraph using the deployment command. See Manage Subgraph