community-development.md

Development

This section is for developers who want to build ReadySet from source as they work on the ReadySet codebase.

Install prerequisites

1. Install ReadySet dependencies.

   macOS with homebrew:

   brew install lz4 [email protected]

   Add the following to your cargo config to make it discover lz4:

   [env]
   LIBRARY_PATH = "/opt/homebrew/lib"

   Ubuntu:

   sudo apt update && sudo apt install -y build-essential libssl-dev pkg-config llvm clang liblz4-dev
   sudo apt-get -y install cmake

   Arch:

   sudo pacman -S base-devel clang lz4

   CentOS/Amazon Linux:

   sudo yum -y update
   sudo yum -y groupinstall "Development Tools"
   sudo yum -y install clang lz4-devel openssl-devel

   Nix:

   nix-shell

2. Install Rust via rustup.rs.

   ReadySet is written entirely in Rust.

3. Install Docker via Get Docker and Docker Compose via Install Docker Compose.

   ReadySet runs alongside a backing Postgres or MySQL database and, when run in distributed fashion, uses Consul for leader election, failure detection, and internal cluster state management. You'll use Docker Compose to create and manage these resources for local development.

Build and run ReadySet

1. Clone the repo using git and navigate into it:

   git clone https://github.com/readysettech/readyset.git
   cd readyset

2. Start a backing database:

   docker-compose up -d

   This starts both Postgres and MySQL in containers, although you will run ReadySet against only one at a time. If you don't want to run both databases, edit docker-compose.yml and comment out the mysql or postgres fields.

3. Compile and run ReadySet, replacing <deployment name> with a unique identifier for the deployment.

   Run against Postgres:

   cargo run --bin readyset --release -- --standalone --database-type=postgresql --upstream-db-url=postgresql://postgres:[email protected]:5432/testdb --username=postgres --password=readyset --address=0.0.0.0:5433 --deployment=<deployment name> --prometheus-metrics --query-log --query-log-ad-hoc

   Run against MySQL:

   cargo run --bin readyset --release -- --standalone --database-type=mysql --upstream-db-url=mysql://root:[email protected]:3306/testdb --username=root --password=readyset --address=0.0.0.0:3307 --deployment=<deployment name> --prometheus-metrics --query-log --query-log-ad-hoc

   This runs the ReadySet Server and Adapter as a single process, a simple, standard way to run ReadySet that is also the easiest approach when developing. For production deployments, however, it is possible to run the Server and Adapter as separate processes. See the scale out deployment pattern in the docs.

   For details about ReadySet options, see the CLI docs (coming soon).

4. With ReadySet up and running, you can now connect the Postgres or MySQL shell:

   Postgres:

   PGPASSWORD=readyset psql \
   --host=127.0.0.1 \
   --port=5433 \
   --username=postgres \
   --dbname=testdb

   MySQL:

   mysql \
   --host=127.0.0.1 \
   --port=3307 \
   --user=root \
   --password=readyset \
   --database=testdb

Testing

To run tests for the project, run the following command:

cargo test --skip integration_serial

Note:

• Certain tests cannot be run in parallel with others, and these tests are typically in files postfixed with _serial. Running the entire set of tests for a package (e.g., cargo test -p readyset-server) may fail if serial tests are included.

• Running tests may require increasing file descriptor limits. You can do so by running ulimit -Sn 65535.