This guide is intended to be used by contributors to learn about how to develop RisingWave. The instructions about how to submit code changes are included in contributing guidelines.
If you have questions, you can search for existing discussions or start a new discussion in the Discussions forum of RisingWave, or ask in the RisingWave Community channel on Slack. Please use the invitation link to join the channel.
To report bugs, create a GitHub issue.
- Developer guide
Before you start to make code changes, ensure that you understand the design and implementation of RisingWave. We recommend that you read the design docs listed in docs/README.md first.
- The
src
folder contains all of the kernel components, refer to src/README.md for more details. - The
docker
folder contains Docker files to build and start RisingWave. - The
e2e_test
folder contains the latest end-to-end test cases. - The
docs
folder contains the design docs. If you want to learn about how RisingWave is designed and implemented, check out the design docs here. - The
dashboard
folder contains RisingWave dashboard v2.
RiseDev is the development mode of RisingWave. To develop RisingWave, you need to build from the source code and run RiseDev. RiseDev can be built on macOS and Linux. It has the following dependencies:
- Rust toolchain
- CMake
- protobuf
- OpenSSL
- PostgreSQL (psql) (>= 14.1)
- Tmux
To install the dependencies on macOS, run:
brew install postgresql cmake protobuf openssl tmux
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
To install the dependencies on Debian-based Linux systems, run:
sudo apt install make build-essential cmake protobuf-compiler curl openssl libssl-dev libcurl4-openssl-dev pkg-config postgresql-client tmux lld
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
Then you'll be able to compile and start RiseDev!
You can now build RiseDev and start a dev cluster. It is as simple as:
./risedev d # shortcut for ./risedev dev
psql -h localhost -p 4566 -d dev -U root
If you detect memory bottlenecks while compiling, either allocate some disk space on your computer as swap memory, or lower the compilation parallelism with CARGO_BUILD_JOBS
, e.g. CARGO_BUILD_JOBS=2
.
The default dev cluster includes metadata-node, compute-node and frontend-node processes, and an embedded volatile in-memory state storage. No data will be persisted. This configuration is intended to make it easier to develop and debug RisingWave.
To stop the cluster:
./risedev k # shortcut for ./risedev kill
To view the logs:
./risedev l # shortcut for ./risedev logs
To clean local data and logs:
./risedev clean-data
There are a few components that you can configure in RiseDev.
Use the ./risedev configure
command to start the interactive configuration mode, in which you can enable and disable components.
- Hummock (MinIO + MinIO-CLI): Enable this component to persist state data.
- Prometheus and Grafana: Enable this component to view RisingWave metrics. You can view the metrics through a built-in Grafana dashboard.
- Etcd: Enable this component if you want to persist metadata node data.
- Kafka: Enable this component if you want to create a streaming source from a Kafka topic.
- Jaeger: Use this component for tracing.
To manually add those components into the cluster, you will need to configure RiseDev to download them first. For example,
./risedev configure enable prometheus-and-grafana # enable Prometheus and Grafana
./risedev configure enable minio # enable MinIO
Note
Enabling a component with the
./risedev configure enable
command will only download the component to your environment. To allow it to function, you must revise the corresponding configuration setting inrisedev.yml
and restart the dev cluster.
For example, you can modify the default section to:
default:
- use: minio
- use: meta-node
enable-dashboard-v2: false
- use: compute-node
- use: frontend
- use: prometheus
- use: grafana
- use: zookeeper
persist-data: true
- use: kafka
persist-data: true
Note
The Kafka service depends on the ZooKeeper service. If you want to enable the Kafka component, enable the ZooKeeper component first.
Now you can run ./risedev d
to start a new dev cluster. The new dev cluster will contain components as configured in the yaml file. RiseDev will automatically configure the components to use the available storage service and to monitor the target.
You may also add multiple compute nodes in the cluster. The ci-3cn-1fe
config is an example.
If you do not need to start a full cluster to develop, you can issue ./risedev p
to start the playground, where the metadata node, compute nodes and frontend nodes are running in the same process. Logs are printed to stdout instead of separate log files.
./risedev p # shortcut for ./risedev playground
For more information, refer to README.md
under src/risedevtool
.
To start the playground (all-in-one process) from IDE or command line, you can use:
cargo run --bin risingwave -- playground
Then, connect to the playground instance via:
psql -h localhost -p 4566 -d dev -U root
Currently, RisingWave has two versions of dashboards. You can use RiseDev config to select which version to use.
The dashboard will be available at http://127.0.0.1:5691/
on meta node.
Dashboard v1 is a single HTML page. To preview and develop this version, install Node.js, and run this command:
cd src/meta/src/dashboard && npx reload -b
Dashboard v1 is bundled by default along with meta node. When the cluster is started, you may use the dashboard without any configuration.
The development instructions for dashboard v2 are available here.
RiseDev supports several observability components.
risectl
is the tool for providing internal access to the RisingWave cluster. See
cargo run --bin risectl -- --help
... or
./risingwave risectl --help
for more information.
Uncomment grafana
and prometheus
lines in risedev.yml
to enable Grafana and Prometheus services.
Compute nodes support streaming tracing. Tracing is not enabled by default. You need to
use ./risedev configure
to download the tracing components first. After that, you will need to uncomment jaeger
service in risedev.yml
and start a new dev cluster to allow the components to work.
You may use RisingWave Dashboard to see actors in the system. It will be started along with meta node.
The Rust components use tokio-tracing
to handle both logging and tracing. The default log level is set as:
- Third-party libraries: warn
- Other libraries: debug
If you need to adjust log levels, change the logging filters in utils/runtime/lib.rs
.
Before you submit a PR, fully test the code changes and perform necessary checks.
The RisingWave project enforces several checks in CI. Every time the code is modified, you need to perform the checks and ensure they pass.
RisingWave requires all code to pass fmt, clippy, sort and hakari checks. Run the following commands to install test tools and perform these checks.
./risedev install-tools # Install required tools for running unit tests
./risedev c # Run all checks. Shortcut for ./risedev check
RiseDev runs unit tests with cargo-nextest. To run unit tests:
./risedev install-tools # Install required tools for running unit tests
./risedev test # Run unit tests
If you want to see the coverage report, run this command:
./risedev test-cov
RisingWave's SQL frontend has SQL planner tests. For more information, see Planner Test Guide.
Use sqllogictest-rs to run RisingWave e2e tests.
sqllogictest installation is included when you install test tools with the ./risedev install-tools
command. You may also install it with:
cargo install --git https://github.com/risinglightdb/sqllogictest-rs --bin sqllogictest
Before running end-to-end tests, you will need to start a full cluster first:
./risedev d
Then to run the end-to-end tests, you can use one of the following commands according to which component you are developing:
# run all streaming tests
./risedev slt-streaming -p 4566 -d dev -j 1
# run all batch tests
./risedev slt-batch -p 4566 -d dev -j 1
# run both
./risedev slt-all -p 4566 -d dev -j 1
Note
Use
-j 1
to create a separate database for each test case, which can ensure that previous test case failure won’t affect other tests due to table cleanups.
Alternatively, you can also run some specific tests:
# run a single test
./risedev slt -p 4566 -d dev './e2e_test/path/to/file.slt'
# run all tests under a directory (including subdirectories)
./risedev slt -p 4566 -d dev './e2e_test/path/to/directory/**/*.slt'
After running e2e tests, you may kill the cluster and clean data.
./risedev k # shortcut for ./risedev kill
./risedev clean-data
RisingWave's codebase is constantly changing. The persistent data might not be stable. In case of unexpected decode errors, try ./risedev clean-data
first.
Basically, CI is using the following two configurations to run the full e2e test suite:
./risedev dev ci-3cn-1fe
You can adjust the environment variable to enable some specific code to make all e2e tests pass. Refer to GitHub Action workflow for more information.
For shell code, please run:
brew install shellcheck
shellcheck <new file>
For Protobufs, we rely on buf for code formatting and linting. Please check out their documents for installation. To check if you violate the rules, please run the commands:
buf format -d --exit-code
buf lint
See README for more information.
We use skywalking-eyes to manage license headers. If you added new files, please follow the installation guide and run:
license-eye -c .licenserc.yaml header fix
To avoid rebuild some common dependencies across different crates in workspace, use
cargo-hakari to ensure all dependencies
are built with the same feature set across workspace. You'll need to run cargo hakari generate
after deps get updated.
Use cargo-udeps to find unused dependencies in workspace.
And use cargo-sort to ensure all deps are get sorted.
Instructions about submitting PRs are included in the contribution guidelines.