pmemkv-bench

Benchmark for libpmemkv and its underlying libraries, based on leveldb's db_bench.

⚠️ Discontinuation of the project

The pmemkv-bench project will no longer be maintained by Intel.

• Intel has ceased development and contributions including, but not limited to, maintenance, bug fixes, new releases, or updates, to this project.
• Intel no longer accepts patches to this project.
• If you have an ongoing need to use this project, are interested in independently developing it, or would like to maintain patches for the open source software community, please create your own fork of this project.
• You will find more information here.

Introduction

The pmemkv_bench utility provides some standard read, write & remove benchmarks. It's based on the db_bench utility included with LevelDB and RocksDB, although the list of supported parameters is slightly different.

It is up-to-date with libpmemkv. Please don't fully rely on API and the results produced by this tool.

Note:

It is the user's responsibility to clear/remove DB (e.g. using pmempool rm). So running bench with --benchmarks=fillseq,fillrandom might have different results than running with --benchmarks=fillseq, removing DB and running bench one more time with --benchmarks--fillrandom, because in the first case fillrandom writes to the already filled DB.

Table of contents

1. Build
   • Prerequisites
   • Setting up environment
   • Compile
2. Execute
   • Various Pools
   • Runtime Parameters
3. Contact us

Build

Prerequisites

• Linux 64-bit (OSX and Windows are not yet supported)
• libpmemkv - library tested in this benchmark; currently version from master branch is required
• libpmempool, which is part of PMDK - tool to manage persistent pools
• python3 - for executing additional python scripts (and tests)
• Used only for development:
  • clang-format-11 - to format and check cpp coding style, version 11 is required
  • black 21.4b2 - to format and check python coding style, version 21.4b2 is required

See our Dockerfile (used e.g. on our CI system) to get an idea what packages are required to build and run pmemkv-bench.

Setting up environment

Makefile tries to find required packages (especially libpmemkv and libpmempool) using pkg-config. If dependencies are not installed in the system PKG_CONFIG_PATH variable has to be set to point to .pc files.

For execution, additionally LD_LIBRARY_PATH variable should be set to the directory which contains libpmemkv.so.1 file, e.g.:

export PKG_CONFIG_PATH=${PKG_CONFIG_PATH}:</path/to/libpmemkv/pkgconfig/dir>
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:</path/to/libpmemkv/lib/dir>

Compile

To build pmemkv_bench, you can simple run:

make bench

Execute

To simply run pmemkv-bench on DRAM:

PMEM_IS_PMEM_FORCE=1 ./pmemkv_bench --db=/dev/shm/pmemkv --db_size_in_gb=1

or using make command (with custom testing path):

make run_bench [ KV_BENCH_TEST_PATH=<test_path> ]

Alternatively you can run building selected version of pmemkv (and required by it a libpmemobj-cpp library) and run benchmark scenarios using python script (available in project's root directory):

python3 run_benchmark.py <build_config_file> <bench_scenarios_file>

Example usage of this script is also shown in one of our tests.

Various Pools

Benchmarking on filesystem DAX (fsdax, mounted e.g. on /mnt/pmem):

./pmemkv_bench --db=/mnt/pmem/pmemkv --db_size_in_gb=1

Benchmarking on device DAX (dev-dax):

./pmemkv_bench --db=/dev/dax1.0

Benchmarking with poolset (to read more, see PMDK's manpage poolset(5)):

./pmemkv_bench --db=~/pmemkv.poolset

Remember:

The pmemkv-bench won't clear the pool passed in --db parameter on start or at finish. It is the user's responsibility to clear it if desired.

Runtime Parameters

All of supported pmemkv-bench's parameters are described in command's help (see it e.g. in bench's sources).

Nonetheless, the most important runtime parameters are:

--engine=<name>            (storage engine name, default: cmap)
--db=<location>            (path to persistent pool, default: /dev/shm/pmemkv_test_db)
                           (note: file on DAX filesystem, DAX device, or poolset file)
--db_size_in_gb=<integer>  (size of persistent pool to create in GiB, default: 0)
                           (note: for existing poolset or device DAX configs use 0 or leave default value)
                           (note: when pool path is non-existing, value should be > 0)
--histogram=<0|1>          (show histograms when reporting latencies)
--num=<integer>            (number of keys to place in database, default: 1000000)
--reads=<integer>          (number of read operations, default: 1000000)
--threads=<integer>        (number of concurrent threads, default: 1)
--key_size=<integer>       (size of keys in bytes, default: 16)
--value_size=<integer>     (size of values in bytes, default: 100)
--benchmarks=<name>,       (comma-separated list of benchmarks to run)
    fillseq                (load N values in sequential key order)
    fillrandom             (load N values in random key order)
    readseq                (read N values in sequential key order)
    readrandom             (read N values in random key order)
    readmissing            (read N missing values in random key order)
    deleteseq              (delete N values in sequential key order)
    deleterandom           (delete N values in random key order)
    readwhilewriting       (1 writer, N threads doing random reads)
    readrandomwriterandom  (N threads doing random-read, random-write)

To run the equivalent of "overwrite" benchmark, run fillrandom on already filled DB.

Contact us

If you read the blog post and still have some questions (especially about discontinuation of the project), please contact us using the dedicated e-mail: [email protected].