Data Parsing & Data Optimization for NATS

[0xterminator]

Goals

In the context of fuel-streams and NATS in particular, there are two important aspects related to data that we need to analyze well before we attempt to optimize the data transport for production.

1. data serialization
2. data compression (compaction)

In the context of NATS, we generally aim at having small and easily transportable data packages sent over the wire especially as the NATS cluster is likely to be on a different cloud infra and publishers/consumers be located at different geographical locations. There could also be data delays, latencies, relays etc. slowing down the data transport from the moment it gets published over the wire and the time by which it gets consumed and unwrapped in its expected for ready-use format. To minimize the latter, compression and serialization often contribute to keeping the data packages small and compact, thus reducing the load on the message bus system, increasing delivery times and at the same time reducing the storage required for keeping the data persistently. There are two major criteria we need to consider when evaluating the success of our optimization:

When publishing:

1. time taken to serialize the data package
2. time taken to apply compression (if any) to the serialized data
3. time taken to publish the serialized and compressed data to NATS

When consuming:

1. time taken to consume the published above data over NATS
2. time taken to deserialize the data package
3. time taken to apply decompression (if any) to the deserialized data

The main mathematical goal of our discussion and testing will be to minimize the aggregated sums of all times for the publishing and respectively consuming parts. Of course, each of these could be compared in isolation to see which serialization / compression works best on their own (and also combined) given the published data morphology we have. We might also quickly have a look at the transport time based on package size.

Parsing Technology

These are some good articles with comparisons and benchmarks that shed some light on the topics of data serialization and compression:

1. Data serialization : https://blog.logrocket.com/rust-serialization-whats-ready-for-production-today/
2. Data compression: https://blog.logrocket.com/rust-compression-libraries/

Since we are using an e2e Rust solution, it is highly recommended to utilize a native, sound and robust Rust - based solution. Since we also heavily tokio-oriented, we shall also make use of a tokio-based solution.

For the current implementation and based on the scope o the articles mentioned we shall examine the following crates for serialization/deserialization:

# serialization/deserialization
postcard = "1.0.8"
prost = "0.13.1"
bincode = "1.3.3"
serde = { version = "1.0", features = ["derive"] }
serde_json = { workspace = true }

which support the serde-based bincode, serde json, postcard schemes. We also provide support for prost (protobuf) schemas via the prost crate.

and also consider the following for compression/decompression:

async-compression = { version = "0.4.11", features = ["all"] }

async-compression is an async tokio-based library with support for the most famous compression types: Zlib, Gzip, Brotli, Bz, Lzma, Deflate, Zstd.

Implementation

A data parser builder is suggested as in the MR: #110. The idea behind it is to have exposed interfaces for setting CompressionType ,CompressionLevel and SerializationType.

#[derive(Debug, Clone)]
pub(crate) struct DataParserBuilder {
    compression_type: Option<CompressionType>,
    compression_level: Option<Level>,
    serialization_type: Option<SerializationType>,
}

impl DataParserBuilder {
    /// Sets the compression type and returns the builder
    pub fn with_compression(mut self, compression_type: CompressionType, ) -> Self;

    /// Sets the compression level and returns the builder
    pub fn with_compression_level(mut self, compression_level: Level) -> Self;

    /// Sets the serialization type and returns the builder
    pub fn with_serialization(mut self, serialization_type: SerializationType, ) -> Self;

    /// Builds the DataParser instance using the specified settings or defaults
    pub fn build(self) -> DataParser;
}

The SerializationType and CompressionType enums allow setting the schema types using an appropriate variant:

pub enum SerializationType {
    /// Bincode serialization
    Bincode,
    /// Postcard serialization
    Postcard,
    /// json serialization
    Json,
}

pub enum CompressionType {
    /// No compression to be applied
    None,
    /// Zlib compression
    Zlib,
    /// Gzip compression
    Gzip,
    /// Brotli compression
    Brotli,
    /// Bz compression
    Bz,
    /// Lzma compression
    Lzma,
    /// Deflate compression
    Deflate,
    /// Zstd compression
    Zstd,
}

The latter allows building a universal data parser as such:

        let data_parser = DataParserBuilder::new()
            .with_compression(CompressionType::Gzip)
            .with_compression_level(Level::Fastest)
            .with_serialization(SerializationType::Bincode)
            .build();

having the following interfaces:

impl DataParser {
    /// Serializes and compresses the data
    pub async fn serialize_and_compress(&self, raw_data: impl serde::Serialize, ) -> Result<Vec<u8>, Error> { ... }

    /// Decompresses and deserializes the data
    pub async fn decompress_and_deserialize<T: serde::de::DeserializeOwned>(&self, raw_data: &[u8],) -> Result<T, Error> { ... }

    /// Serializes the data
    pub async fn serialize(&self, raw_data: impl serde::Serialize,) -> Result<Vec<u8>, Error> { ... }

    /// Deserializes the data
    pub async fn deserialize<'a, T: serde::Deserialize<'a>>(&self,  raw_data: &'a [u8], ) -> Result<T, Error> { ... }

    /// Compresses the data
    pub async fn compress(&self, raw_data: &[u8]) -> Result<Vec<u8>, Error> { ... }

    /// Decompresses the data
    pub async fn decompress(&self, raw_data: &[u8]) -> Result<Vec<u8>, Error> { ... }
}

We suggest having combined methods for serializing and compression, deserializing and decompression as well as these operations ones. The data parser wraps internally various compression/decompression and serialization/deserialization methods which can be easily set by the enums CompressionType, Leval and SerializationType suggested above. As its to be seen, serialization and deserialization on structures that implement serde::Serialize and serde::DeserializeOwned (which most of the fuel-core-types have) can be handled by all wrapped serializers.

A test example would look like:

// define a test data structure
        #[derive(Clone, Debug, Serialize, SerdeDeserialize, Eq, PartialEq)]
        struct MyTestData {
            ids: Vec<String>,
            version: u64,
            receipts: Vec<String>,
            assets: Vec<AssetId>,
            chain_id: ChainId,
        }

        // construct the data parser
        let data_parser = DataParserBuilder::new()
            .with_compression(CompressionType::Gzip)
            .with_compression_level(Level::Fastest)
            .with_serialization(SerializationType::Bincode)
            .build();

        let test_data = MyTestData {
            ids: vec!["1".to_string(), "2".to_string(), "3".to_string()],
            version: 1u64,
            receipts: vec!["receipt_1".to_string(), "receipt_2".to_string(), "receipt_3".to_string()],
            assets: vec![AssetId::zeroed()],
            chain_id: ChainId::new(1),
        };

        // compress and serialize
        let ser_compressed_data = data_parser.serialize_and_compress(&test_data).await.unwrap();

        // deserialize and decompress
        let my_test_data_recreated = data_parser.decompress_and_deserialize::<MyTestData>(&ser_compressed_data).await.unwrap();

Data Types

The data types which ought to be serialized/deserialized are all described in the notion document:

https://www.notion.so/fuellabs/Fuel-Streams-Tech-Specs-f18465f40b5d433fbae90427649bea9a#c3b979f13e004dcea75df54766b4a9ae

alongside with the data format specification. One can use this structure and data morphology when applying an appropriate compression.

Testing Strategy

1. Objective

The primary objective of the testing strategy is to establish a benchmarking protocol using the crate criterion to identify the most efficient serialization and compression methods for optimizing data payloads sent over NATS. The goal is to minimize the total time taken for:

• Publishing data (including serialization and compression times)
• Consuming data (including decompression and deserialization times)

2. Scope

This benchmark will assess multiple serialization and compression combinations to determine which provides the best balance of speed and payload size for communication via NATS. The scope includes:

• Serialization Formats: see above.
• Compression Algorithms: see above

3. Benchmarking Setup

Hardware and Environment

• Specify the hardware specifications where the tests will be run (CPU, RAM, Network specs).
• Ensure a consistent testing environment to avoid external influences on performance results.

Test Data
Define a standard test dataset based on a given test struct which includes diverse data types (strings, integers, custom structs).
Ensure the data is a real data taken from fuel-core-types.

4. Test Methodology

Testing Phases

• Serialization Time: Time taken to convert the test data structure to the specified serialization format.
• Compression Time: Time taken to compress the serialized data.
• Total Publish Time: Sum of serialization, compression, and the time taken to publish the data to NATS.
• Data Size: The size of the payload after serialization and after compression, as it affects transmission time and network usage.
• Consumption Time: Time taken to retrieve the message from NATS.
• Decompression Time: Time taken to decompress the received message.
• Deserialization Time: Time taken to convert the compressed data back to the test data.

Each test should be run multiple times to calculate the average and standard deviation for each phase.

5. Evaluation Criteria

• Performance: Evaluate the time taken for each phase of the data handling process.
• Efficiency: Assess the size of the data payload after serialization and compression.
• Reliability: Consider the stability and error rates of each serialization and compression combination.

6. Reporting

Present the results in a clear format, using tables and graphs to compare the time and efficiency of each method.
Include raw data logs for transparency and further analysis.

7. Recommendations

Based on the benchmark results, recommend the best serialization and compression combinations for different types of data payloads.
Suggest configuration settings for optimal performance with NATS.

[pedronauck]

Really nice work here @0xterminator, congrats 👏🏻

---

tldr: Let's keep it simple and focus on one compression and one serialization method for now, plus JSON for SDK use. We can add more options later if needed.

---

I appreciate the concept of a "universal data parsing" system that can leverage multiple serialization and compression methods. However, from a product perspective, and considering our current stage of development, I have some concerns.

At this phase, introducing multiple options might create unnecessary complexity and overhead that we should avoid. Instead, I suggest we take a more targeted approach. Let's select a single, high-performance algorithm for both compression and serialization that best meets our current needs.

We should, however, include JSON as an additional option - not for inter-service messaging, but for SDK integration and high-level usage after data consumption. This approach doesn't mean we're closing doors for the future. As we gather more user feedback and encounter specific use cases requiring different algorithms, we can always add support for them incrementally.

By focusing on a single method for each, we can deeply optimize our implementation, ensure robust performance, and simplify our codebase, testing procedures, and ongoing maintenance efforts. It'll also allow us to provide more detailed and focused documentation for developers.

This strategy allows us to move forward efficiently while leaving room for future enhancements based on real-world usage and requirements. It strikes a balance between functionality, simplicity, and adaptability - which I believe is crucial at our current stage.

[luizstacio]

I liked the multiple options. As such, we focus on benchmarking and have the most optimal one based on what you described.

The same goes for data encoded/decoded. We should only keep in mind portability as the client for the stream can also be built on the langs like javascript.

[SilentCicero]

Why not consider using the Fuel ABI encoding / decoding for this? All our SDK's already come with that encoder/decoder and it's fairly compact all things considered.

I'd be open to seeing which options are the best over the wire, I'd be curious to hear what @Voxelot has to say.

[0xterminator]

After running series of benchmark tests, I am pasting a block of results that depict the overall conclusions quite nicely:

deserialize/[bincode]   time:   [34.471 µs 35.322 µs 36.160 µs]
                        change: [+42.605% +44.623% +46.758%] (p = 0.00 < 0.05)
                        Performance has regressed.
deserialize/[postcard]  time:   [33.175 µs 33.388 µs 33.628 µs]
                        change: [+21.513% +23.091% +25.036%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 14 outliers among 100 measurements (14.00%)
  7 (7.00%) high mild
  7 (7.00%) high severe
deserialize/[json]      time:   [191.49 µs 192.68 µs 194.13 µs]
                        change: [+19.537% +20.902% +22.628%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
  7 (7.00%) high mild
  4 (4.00%) high severe

     Running benches/deserialize_decompress.rs (target/release/deps/deserialize_decompress-edb0201d63f265d0)
Gnuplot not found, using plotters backend
decompress_deserialize/["bincode"]["none"][Default]
                        time:   [31.448 µs 32.267 µs 33.133 µs]
                        change: [+22.777% +25.023% +27.172%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
  10 (10.00%) high mild
  1 (1.00%) high severe
decompress_deserialize/["bincode"]["none"][Fastest]
                        time:   [32.072 µs 32.321 µs 32.609 µs]
                        change: [+21.362% +22.741% +24.367%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  6 (6.00%) high mild
  4 (4.00%) high severe
decompress_deserialize/["bincode"]["none"][Best]
                        time:   [32.075 µs 32.320 µs 32.615 µs]
                        change: [+23.474% +24.808% +26.158%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) high mild
decompress_deserialize/["bincode"]["zlib"][Default]
                        time:   [48.757 µs 49.030 µs 49.355 µs]
                        change: [+21.180% +22.724% +24.230%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 15 outliers among 100 measurements (15.00%)
  11 (11.00%) high mild
  4 (4.00%) high severe
decompress_deserialize/["bincode"]["zlib"][Fastest]
                        time:   [47.468 µs 47.694 µs 47.975 µs]
                        change: [+21.031% +22.605% +24.333%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 15 outliers among 100 measurements (15.00%)
  6 (6.00%) high mild
  9 (9.00%) high severe
decompress_deserialize/["bincode"]["zlib"][Best]
                        time:   [45.299 µs 45.554 µs 45.849 µs]
                        change: [+15.417% +16.903% +18.427%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 15 outliers among 100 measurements (15.00%)
  6 (6.00%) high mild
  9 (9.00%) high severe
decompress_deserialize/["bincode"]["gzip"][Default]
                        time:   [44.506 µs 44.817 µs 45.205 µs]
                        change: [+20.422% +21.783% +23.463%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 17 outliers among 100 measurements (17.00%)
  9 (9.00%) high mild
  8 (8.00%) high severe
decompress_deserialize/["bincode"]["gzip"][Fastest]
                        time:   [43.556 µs 43.847 µs 44.209 µs]
                        change: [+19.561% +21.063% +22.571%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 18 outliers among 100 measurements (18.00%)
  2 (2.00%) high mild
  16 (16.00%) high severe
decompress_deserialize/["bincode"]["gzip"][Best]
                        time:   [41.320 µs 41.676 µs 42.071 µs]
                        change: [-19.282% -15.410% -11.326%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) high mild
decompress_deserialize/["bincode"]["brotli"][Default]
                        time:   [81.616 µs 82.301 µs 83.080 µs]
                        change: [-7.3889% -4.0759% -0.8759%] (p = 0.02 < 0.05)
                        Change within noise threshold.
Found 16 outliers among 100 measurements (16.00%)
  8 (8.00%) high mild
  8 (8.00%) high severe
decompress_deserialize/["bincode"]["brotli"][Fastest]
                        time:   [197.86 µs 199.84 µs 202.07 µs]
                        change: [+9.2502% +12.729% +16.106%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  7 (7.00%) high mild
  3 (3.00%) high severe
decompress_deserialize/["bincode"]["brotli"][Best]
                        time:   [82.074 µs 82.643 µs 83.314 µs]
                        change: [+2.8911% +7.7824% +12.653%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
  4 (4.00%) high mild
  7 (7.00%) high severe
decompress_deserialize/["bincode"]["bz"][Default]
                        time:   [142.43 µs 143.42 µs 144.58 µs]
                        change: [+25.156% +26.488% +28.012%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  5 (5.00%) high mild
  3 (3.00%) high severe
decompress_deserialize/["bincode"]["bz"][Fastest]
                        time:   [142.52 µs 143.67 µs 144.91 µs]
                        change: [+18.507% +20.242% +22.058%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  5 (5.00%) high mild
  2 (2.00%) high severe
decompress_deserialize/["bincode"]["bz"][Best]
                        time:   [142.56 µs 144.05 µs 145.89 µs]
                        change: [+23.470% +25.342% +27.043%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high mild
decompress_deserialize/["bincode"]["lzma"][Default]
                        time:   [41.470 µs 41.773 µs 42.122 µs]
                        change: [+13.065% +15.098% +17.082%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  4 (4.00%) high mild
  4 (4.00%) high severe
decompress_deserialize/["bincode"]["lzma"][Fastest]
                        time:   [41.526 µs 41.942 µs 42.440 µs]
                        change: [-9.0262% -6.5167% -4.1143%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  5 (5.00%) high mild
  3 (3.00%) high severe
decompress_deserialize/["bincode"]["lzma"][Best]
                        time:   [60.570 µs 60.985 µs 61.523 µs]
                        change: [+15.610% +18.387% +21.023%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 13 outliers among 100 measurements (13.00%)
  5 (5.00%) high mild
  8 (8.00%) high severe
decompress_deserialize/["bincode"]["deflate"][Default]
                        time:   [43.622 µs 43.957 µs 44.371 µs]
                        change: [+16.651% +18.641% +20.558%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 12 outliers among 100 measurements (12.00%)
  5 (5.00%) high mild
  7 (7.00%) high severe
decompress_deserialize/["bincode"]["deflate"][Fastest]
                        time:   [43.159 µs 43.556 µs 44.053 µs]
                        change: [+17.340% +19.153% +21.132%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  6 (6.00%) high mild
  4 (4.00%) high severe
decompress_deserialize/["bincode"]["deflate"][Best]
                        time:   [40.598 µs 40.967 µs 41.386 µs]
                        change: [-2.2495% +1.7913% +5.6127%] (p = 0.38 > 0.05)
                        No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
  3 (3.00%) high mild
  3 (3.00%) high severe
decompress_deserialize/["bincode"]["zstd"][Default]
                        time:   [34.216 µs 34.511 µs 34.873 µs]
                        change: [+5.3552% +7.3982% +9.5693%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
  4 (4.00%) high mild
  5 (5.00%) high severe
decompress_deserialize/["bincode"]["zstd"][Fastest]
                        time:   [34.205 µs 34.437 µs 34.714 µs]
                        change: [+5.6519% +7.5407% +9.3619%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  7 (7.00%) high mild
  3 (3.00%) high severe
decompress_deserialize/["bincode"]["zstd"][Best]
                        time:   [51.384 µs 51.801 µs 52.287 µs]
                        change: [+11.348% +13.737% +16.099%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 13 outliers among 100 measurements (13.00%)
  6 (6.00%) high mild
  7 (7.00%) high severe
decompress_deserialize/["postcard"]["none"][Default]
                        time:   [33.395 µs 33.638 µs 33.927 µs]
                        change: [+6.8819% +9.4441% +11.836%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
  7 (7.00%) high mild
  4 (4.00%) high severe
decompress_deserialize/["postcard"]["none"][Fastest]
                        time:   [33.177 µs 33.354 µs 33.574 µs]
                        change: [+13.546% +15.269% +17.015%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 14 outliers among 100 measurements (14.00%)
  6 (6.00%) high mild
  8 (8.00%) high severe
decompress_deserialize/["postcard"]["none"][Best]
                        time:   [32.857 µs 33.007 µs 33.180 µs]
                        change: [+14.861% +17.179% +19.543%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 14 outliers among 100 measurements (14.00%)
  4 (4.00%) high mild
  10 (10.00%) high severe
decompress_deserialize/["postcard"]["zlib"][Default]
                        time:   [55.944 µs 56.359 µs 56.871 µs]
                        change: [+22.156% +23.663% +25.257%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high mild
decompress_deserialize/["postcard"]["zlib"][Fastest]
                        time:   [46.169 µs 46.729 µs 47.373 µs]
                        change: [+20.985% +22.465% +24.328%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) high mild
  5 (5.00%) high severe
decompress_deserialize/["postcard"]["zlib"][Best]
                        time:   [56.544 µs 57.144 µs 57.868 µs]
                        change: [+16.989% +19.194% +21.305%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
  5 (5.00%) high mild
  4 (4.00%) high severe
decompress_deserialize/["postcard"]["gzip"][Default]
                        time:   [53.092 µs 53.540 µs 54.067 µs]
                        change: [+20.855% +22.560% +24.307%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) high mild
  5 (5.00%) high severe
decompress_deserialize/["postcard"]["gzip"][Fastest]
                        time:   [43.147 µs 43.412 µs 43.755 µs]
                        change: [+18.133% +20.690% +23.069%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 14 outliers among 100 measurements (14.00%)
  6 (6.00%) high mild
  8 (8.00%) high severe
decompress_deserialize/["postcard"]["gzip"][Best]
                        time:   [53.305 µs 53.635 µs 54.012 µs]
                        change: [+20.547% +22.512% +25.079%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 16 outliers among 100 measurements (16.00%)
  7 (7.00%) high mild
  9 (9.00%) high severe
decompress_deserialize/["postcard"]["brotli"][Default]
                        time:   [75.593 µs 76.229 µs 76.995 µs]
                        change: [+21.243% +22.846% +24.727%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 13 outliers among 100 measurements (13.00%)
  5 (5.00%) high mild
  8 (8.00%) high severe
decompress_deserialize/["postcard"]["brotli"][Fastest]
                        time:   [187.09 µs 188.98 µs 191.16 µs]
                        change: [+22.113% +25.996% +29.492%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  5 (5.00%) high mild
  3 (3.00%) high severe
decompress_deserialize/["postcard"]["brotli"][Best]
                        time:   [76.865 µs 77.403 µs 78.038 µs]
                        change: [+0.3354% +2.1605% +4.2463%] (p = 0.02 < 0.05)
                        Change within noise threshold.
Found 11 outliers among 100 measurements (11.00%)
  4 (4.00%) high mild
  7 (7.00%) high severe
decompress_deserialize/["postcard"]["bz"][Default]
                        time:   [126.34 µs 127.21 µs 128.19 µs]
                        change: [-0.7927% +0.7690% +2.5295%] (p = 0.38 > 0.05)
                        No change in performance detected.
Found 14 outliers among 100 measurements (14.00%)
  5 (5.00%) high mild
  9 (9.00%) high severe
decompress_deserialize/["postcard"]["bz"][Fastest]
                        time:   [126.18 µs 127.33 µs 128.70 µs]
                        change: [-3.5493% -2.1011% -0.6171%] (p = 0.01 < 0.05)
                        Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
  3 (3.00%) high mild
  6 (6.00%) high severe
decompress_deserialize/["postcard"]["bz"][Best]
                        time:   [127.31 µs 128.26 µs 129.43 µs]
                        change: [-2.0329% -0.2629% +1.5215%] (p = 0.78 > 0.05)
                        No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
  6 (6.00%) high mild
  2 (2.00%) high severe
decompress_deserialize/["postcard"]["lzma"][Default]
                        time:   [52.888 µs 53.340 µs 53.903 µs]
                        change: [+1.0358% +3.3603% +5.9474%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 12 outliers among 100 measurements (12.00%)
  7 (7.00%) high mild
  5 (5.00%) high severe
decompress_deserialize/["postcard"]["lzma"][Fastest]
                        time:   [52.598 µs 52.962 µs 53.404 µs]
                        change: [+0.5833% +2.2617% +4.1633%] (p = 0.01 < 0.05)
                        Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
  7 (7.00%) high mild
  2 (2.00%) high severe
decompress_deserialize/["postcard"]["lzma"][Best]
                        time:   [71.352 µs 74.313 µs 78.037 µs]
                        change: [+1.6384% +4.2754% +7.2749%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  4 (4.00%) high mild
  6 (6.00%) high severe
decompress_deserialize/["postcard"]["deflate"][Default]
                        time:   [52.188 µs 53.292 µs 54.659 µs]
                        change: [-1.6258% +0.0301% +1.7492%] (p = 0.97 > 0.05)
                        No change in performance detected.
Found 13 outliers among 100 measurements (13.00%)
  5 (5.00%) high mild
  8 (8.00%) high severe
decompress_deserialize/["postcard"]["deflate"][Fastest]
                        time:   [41.950 µs 42.203 µs 42.496 µs]
                        change: [-0.2979% +1.3655% +3.1796%] (p = 0.13 > 0.05)
                        No change in performance detected.
Found 12 outliers among 100 measurements (12.00%)
  6 (6.00%) high mild
  6 (6.00%) high severe
decompress_deserialize/["postcard"]["deflate"][Best]
                        time:   [52.809 µs 53.608 µs 54.549 µs]
                        change: [-0.3699% +1.0682% +2.4032%] (p = 0.15 > 0.05)
                        No change in performance detected.
Found 10 outliers among 100 measurements (10.00%)
  5 (5.00%) high mild
  5 (5.00%) high severe
decompress_deserialize/["postcard"]["zstd"][Default]
                        time:   [35.690 µs 36.030 µs 36.461 µs]
                        change: [-14.385% -12.052% -9.7168%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  5 (5.00%) high mild
  4 (4.00%) high severe
decompress_deserialize/["postcard"]["zstd"][Fastest]
                        time:   [35.751 µs 36.129 µs 36.599 µs]
                        change: [+6.2511% +10.202% +14.273%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
decompress_deserialize/["postcard"]["zstd"][Best]
                        time:   [51.783 µs 52.205 µs 52.671 µs]
                        change: [-11.153% -9.1704% -7.0977%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  5 (5.00%) high mild
  3 (3.00%) high severe
decompress_deserialize/["json"]["none"][Default]
                        time:   [196.79 µs 198.00 µs 199.38 µs]
                        change: [-1.3286% +0.6807% +2.9340%] (p = 0.56 > 0.05)
                        No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
  5 (5.00%) high mild
  6 (6.00%) high severe
decompress_deserialize/["json"]["none"][Fastest]
                        time:   [198.53 µs 199.96 µs 201.64 µs]
                        change: [-26.653% -22.241% -17.676%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  5 (5.00%) high mild
  3 (3.00%) high severe
decompress_deserialize/["json"]["none"][Best]
                        time:   [199.01 µs 200.60 µs 202.47 µs]
                        change: [-30.343% -26.922% -23.365%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  6 (6.00%) high mild
  3 (3.00%) high severe
decompress_deserialize/["json"]["zlib"][Default]
                        time:   [224.27 µs 225.67 µs 227.23 µs]
                        change: [-20.411% -18.288% -16.088%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  5 (5.00%) high mild
  4 (4.00%) high severe
decompress_deserialize/["json"]["zlib"][Fastest]
                        time:   [231.29 µs 233.09 µs 235.30 µs]
                        change: [-19.846% -17.088% -14.482%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  5 (5.00%) high mild
  3 (3.00%) high severe
decompress_deserialize/["json"]["zlib"][Best]
                        time:   [223.89 µs 225.00 µs 226.27 µs]
                        change: [-23.830% -19.295% -14.649%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 15 outliers among 100 measurements (15.00%)
  5 (5.00%) high mild
  10 (10.00%) high severe
decompress_deserialize/["json"]["gzip"][Default]
                        time:   [215.72 µs 217.31 µs 219.11 µs]
                        change: [-32.620% -29.532% -26.225%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe
decompress_deserialize/["json"]["gzip"][Fastest]
                        time:   [220.22 µs 222.24 µs 224.61 µs]
                        change: [-5.6452% -3.1932% -0.9976%] (p = 0.01 < 0.05)
                        Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
  4 (4.00%) high mild
  5 (5.00%) high severe
decompress_deserialize/["json"]["gzip"][Best]
                        time:   [214.47 µs 216.05 µs 217.87 µs]
                        change: [-5.6198% -3.7388% -1.9579%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
  5 (5.00%) high mild
  7 (7.00%) high severe
decompress_deserialize/["json"]["brotli"][Default]
                        time:   [315.84 µs 317.49 µs 319.45 µs]
                        change: [-5.8149% -3.1677% -0.7021%] (p = 0.01 < 0.05)
                        Change within noise threshold.
Found 14 outliers among 100 measurements (14.00%)
  4 (4.00%) high mild
  10 (10.00%) high severe
decompress_deserialize/["json"]["brotli"][Fastest]
                        time:   [432.23 µs 435.94 µs 440.36 µs]
                        change: [-6.7012% -4.4419% -2.1054%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  6 (6.00%) high mild
  5 (5.00%) high severe
decompress_deserialize/["json"]["brotli"][Best]
                        time:   [319.15 µs 321.48 µs 324.24 µs]
                        change: [-4.7977% -2.9979% -1.3419%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  6 (6.00%) high mild
  3 (3.00%) high severe
decompress_deserialize/["json"]["bz"][Default]
                        time:   [684.79 µs 688.58 µs 692.97 µs]
                        change: [-6.4365% -4.7863% -2.9952%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  4 (4.00%) high mild
  7 (7.00%) high severe
decompress_deserialize/["json"]["bz"][Fastest]
                        time:   [705.32 µs 721.56 µs 741.04 µs]
                        change: [-2.3871% +0.0291% +2.3926%] (p = 0.98 > 0.05)
                        No change in performance detected.
Found 14 outliers among 100 measurements (14.00%)
  10 (10.00%) high mild
  4 (4.00%) high severe
decompress_deserialize/["json"]["bz"][Best]
                        time:   [683.61 µs 687.41 µs 691.73 µs]
                        change: [-6.6836% -4.7447% -2.8090%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  2 (2.00%) high mild
  8 (8.00%) high severe
decompress_deserialize/["json"]["lzma"][Default]
                        time:   [249.90 µs 251.56 µs 253.63 µs]
                        change: [-6.2144% -4.2615% -2.3689%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 13 outliers among 100 measurements (13.00%)
  6 (6.00%) high mild
  7 (7.00%) high severe
decompress_deserialize/["json"]["lzma"][Fastest]
                        time:   [256.39 µs 258.08 µs 260.13 µs]
                        change: [-8.9678% -6.4974% -3.9587%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  4 (4.00%) high mild
  4 (4.00%) high severe
decompress_deserialize/["json"]["lzma"][Best]
                        time:   [288.00 µs 289.90 µs 292.05 µs]
                        change: [-30.806% -27.674% -24.537%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
  3 (3.00%) high mild
  8 (8.00%) high severe
decompress_deserialize/["json"]["deflate"][Default]
                        time:   [211.31 µs 212.64 µs 214.21 µs]
                        change: [-32.441% -29.873% -27.420%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 14 outliers among 100 measurements (14.00%)
  7 (7.00%) high mild
  7 (7.00%) high severe
decompress_deserialize/["json"]["deflate"][Fastest]
                        time:   [216.52 µs 217.97 µs 219.73 µs]
                        change: [-25.077% -21.534% -17.954%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 13 outliers among 100 measurements (13.00%)
  5 (5.00%) high mild
  8 (8.00%) high severe
decompress_deserialize/["json"]["deflate"][Best]
                        time:   [213.03 µs 215.01 µs 217.32 µs]
                        change: [-16.555% -14.309% -12.004%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) high mild
  5 (5.00%) high severe
decompress_deserialize/["json"]["zstd"][Default]
                        time:   [204.07 µs 205.18 µs 206.46 µs]
                        change: [-9.8219% -7.4921% -5.3273%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  6 (6.00%) high mild
  2 (2.00%) high severe
decompress_deserialize/["json"]["zstd"][Fastest]
                        time:   [203.37 µs 204.90 µs 206.71 µs]
                        change: [-18.696% -16.024% -13.204%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  4 (4.00%) high mild
  6 (6.00%) high severe
decompress_deserialize/["json"]["zstd"][Best]
                        time:   [241.05 µs 243.23 µs 245.72 µs]
                        change: [-22.445% -19.685% -16.800%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  4 (4.00%) high mild
  5 (5.00%) high severe

     Running benches/serialize.rs (target/release/deps/serialize-b4600b20850254ee)
Gnuplot not found, using plotters backend
serialize/[bincode]     time:   [9.4884 µs 9.5516 µs 9.6240 µs]
                        change: [-6.8320% -5.2554% -3.5869%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
  5 (5.00%) high mild
  7 (7.00%) high severe
serialize/[postcard]    time:   [15.789 µs 15.893 µs 16.018 µs]
                        change: [-6.0536% -3.7971% -1.6089%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 14 outliers among 100 measurements (14.00%)
  8 (8.00%) high mild
  6 (6.00%) high severe
serialize/[json]        time:   [313.36 µs 315.40 µs 317.81 µs]
                        change: [-18.101% -13.455% -7.8949%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  2 (2.00%) high mild
  7 (7.00%) high severe

     Running benches/serialize_compress.rs (target/release/deps/serialize_compress-8ec4d1374537f08e)
Gnuplot not found, using plotters backend
serialize_compress/["bincode"]["none"][Default]
                        time:   [11.017 µs 11.249 µs 11.557 µs]
                        change: [-13.461% -11.075% -8.7415%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  7 (7.00%) high mild
  2 (2.00%) high severe
serialize_compress/["bincode"]["none"][Fastest]
                        time:   [11.034 µs 11.090 µs 11.157 µs]
                        change: [-17.916% -15.884% -13.874%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  6 (6.00%) high mild
  4 (4.00%) high severe
serialize_compress/["bincode"]["none"][Best]
                        time:   [10.949 µs 11.026 µs 11.114 µs]
                        change: [-14.450% -12.623% -10.704%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  6 (6.00%) high mild
  4 (4.00%) high severe
serialize_compress/["bincode"]["zlib"][Default]
                        time:   [10.900 µs 10.951 µs 11.010 µs]
Found 15 outliers among 100 measurements (15.00%)
  7 (7.00%) high mild
  8 (8.00%) high severe
serialize_compress/["bincode"]["zlib"][Fastest]
                        time:   [10.853 µs 11.341 µs 12.003 µs]
Found 15 outliers among 100 measurements (15.00%)
  4 (4.00%) high mild
  11 (11.00%) high severe
serialize_compress/["bincode"]["zlib"][Best]
                        time:   [10.951 µs 11.191 µs 11.607 µs]
Found 14 outliers among 100 measurements (14.00%)
  6 (6.00%) high mild
  8 (8.00%) high severe
serialize_compress/["bincode"]["gzip"][Default]
                        time:   [10.800 µs 10.878 µs 10.970 µs]
Found 10 outliers among 100 measurements (10.00%)
  5 (5.00%) high mild
  5 (5.00%) high severe
serialize_compress/["bincode"]["gzip"][Fastest]
                        time:   [10.869 µs 10.942 µs 11.028 µs]
Found 9 outliers among 100 measurements (9.00%)
  3 (3.00%) high mild
  6 (6.00%) high severe
serialize_compress/["bincode"]["gzip"][Best]
                        time:   [10.904 µs 10.950 µs 11.003 µs]
Found 20 outliers among 100 measurements (20.00%)
  2 (2.00%) low mild
  6 (6.00%) high mild
  12 (12.00%) high severe
serialize_compress/["bincode"]["brotli"][Default]
                        time:   [10.817 µs 10.887 µs 10.972 µs]
Found 16 outliers among 100 measurements (16.00%)
  5 (5.00%) high mild
  11 (11.00%) high severe
serialize_compress/["bincode"]["brotli"][Fastest]
                        time:   [10.920 µs 10.993 µs 11.081 µs]
Found 11 outliers among 100 measurements (11.00%)
  4 (4.00%) high mild
  7 (7.00%) high severe
serialize_compress/["bincode"]["brotli"][Best]
                        time:   [10.763 µs 10.837 µs 10.921 µs]
Found 9 outliers among 100 measurements (9.00%)
  4 (4.00%) high mild
  5 (5.00%) high severe
serialize_compress/["bincode"]["bz"][Default]
                        time:   [10.878 µs 10.940 µs 11.012 µs]
Found 14 outliers among 100 measurements (14.00%)
  6 (6.00%) high mild
  8 (8.00%) high severe
serialize_compress/["bincode"]["bz"][Fastest]
                        time:   [10.902 µs 10.983 µs 11.083 µs]
Found 12 outliers among 100 measurements (12.00%)
  10 (10.00%) high mild
  2 (2.00%) high severe
serialize_compress/["bincode"]["bz"][Best]
                        time:   [10.813 µs 10.887 µs 10.974 µs]
Found 17 outliers among 100 measurements (17.00%)
  8 (8.00%) high mild
  9 (9.00%) high severe
serialize_compress/["bincode"]["lzma"][Default]
                        time:   [10.842 µs 10.917 µs 11.009 µs]
Found 7 outliers among 100 measurements (7.00%)
  5 (5.00%) high mild
  2 (2.00%) high severe
serialize_compress/["bincode"]["lzma"][Fastest]
                        time:   [10.876 µs 10.931 µs 10.994 µs]
Found 19 outliers among 100 measurements (19.00%)
  1 (1.00%) low mild
  4 (4.00%) high mild
  14 (14.00%) high severe
serialize_compress/["bincode"]["lzma"][Best]
                        time:   [11.035 µs 11.161 µs 11.317 µs]
Found 16 outliers among 100 measurements (16.00%)
  6 (6.00%) high mild
  10 (10.00%) high severe
serialize_compress/["bincode"]["deflate"][Default]
                        time:   [11.081 µs 11.164 µs 11.262 µs]
Found 10 outliers among 100 measurements (10.00%)
  5 (5.00%) high mild
  5 (5.00%) high severe
serialize_compress/["bincode"]["deflate"][Fastest]
                        time:   [10.928 µs 10.978 µs 11.039 µs]
Found 14 outliers among 100 measurements (14.00%)
  6 (6.00%) high mild
  8 (8.00%) high severe
serialize_compress/["bincode"]["deflate"][Best]
                        time:   [10.988 µs 11.156 µs 11.363 µs]
Found 15 outliers among 100 measurements (15.00%)
  3 (3.00%) high mild
  12 (12.00%) high severe
serialize_compress/["bincode"]["zstd"][Default]
                        time:   [10.936 µs 11.002 µs 11.079 µs]
Found 8 outliers among 100 measurements (8.00%)
  4 (4.00%) high mild
  4 (4.00%) high severe
serialize_compress/["bincode"]["zstd"][Fastest]
                        time:   [10.917 µs 10.973 µs 11.042 µs]
Found 9 outliers among 100 measurements (9.00%)
  3 (3.00%) high mild
  6 (6.00%) high severe
serialize_compress/["bincode"]["zstd"][Best]
                        time:   [10.963 µs 11.033 µs 11.111 µs]
Found 9 outliers among 100 measurements (9.00%)
  3 (3.00%) high mild
  6 (6.00%) high severe
serialize_compress/["postcard"]["none"][Default]
                        time:   [15.770 µs 15.880 µs 16.007 µs]
Found 17 outliers among 100 measurements (17.00%)
  6 (6.00%) high mild
  11 (11.00%) high severe
serialize_compress/["postcard"]["none"][Fastest]
                        time:   [16.084 µs 16.195 µs 16.326 µs]
Found 9 outliers among 100 measurements (9.00%)
  5 (5.00%) high mild
  4 (4.00%) high severe
serialize_compress/["postcard"]["none"][Best]
                        time:   [16.107 µs 16.233 µs 16.380 µs]
Found 10 outliers among 100 measurements (10.00%)
  3 (3.00%) high mild
  7 (7.00%) high severe
serialize_compress/["postcard"]["zlib"][Default]
                        time:   [15.832 µs 15.917 µs 16.021 µs]
Found 14 outliers among 100 measurements (14.00%)
  5 (5.00%) high mild
  9 (9.00%) high severe
serialize_compress/["postcard"]["zlib"][Fastest]
                        time:   [15.712 µs 15.813 µs 15.942 µs]
Found 15 outliers among 100 measurements (15.00%)
  3 (3.00%) high mild
  12 (12.00%) high severe
serialize_compress/["postcard"]["zlib"][Best]
                        time:   [15.913 µs 16.044 µs 16.195 µs]
Found 6 outliers among 100 measurements (6.00%)
  5 (5.00%) high mild
  1 (1.00%) high severe
serialize_compress/["postcard"]["gzip"][Default]
                        time:   [20.523 µs 21.205 µs 22.025 µs]
Found 18 outliers among 100 measurements (18.00%)
  10 (10.00%) low mild
  6 (6.00%) high mild
  2 (2.00%) high severe
serialize_compress/["postcard"]["gzip"][Fastest]
                        time:   [17.081 µs 17.784 µs 18.816 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe
serialize_compress/["postcard"]["gzip"][Best]
                        time:   [15.599 µs 15.696 µs 15.808 µs]
Found 13 outliers among 100 measurements (13.00%)
  7 (7.00%) high mild
  6 (6.00%) high severe
serialize_compress/["postcard"]["brotli"][Default]
                        time:   [16.574 µs 16.917 µs 17.368 µs]
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe
serialize_compress/["postcard"]["brotli"][Fastest]
                        time:   [15.773 µs 15.882 µs 16.038 µs]
Found 16 outliers among 100 measurements (16.00%)
  4 (4.00%) high mild
  12 (12.00%) high severe
serialize_compress/["postcard"]["brotli"][Best]
                        time:   [15.839 µs 15.944 µs 16.079 µs]
Found 12 outliers among 100 measurements (12.00%)
  5 (5.00%) high mild
  7 (7.00%) high severe
serialize_compress/["postcard"]["bz"][Default]
                        time:   [15.824 µs 15.928 µs 16.052 µs]
Found 11 outliers among 100 measurements (11.00%)
  7 (7.00%) high mild
  4 (4.00%) high severe
serialize_compress/["postcard"]["bz"][Fastest]
                        time:   [16.289 µs 16.649 µs 17.120 µs]
Found 10 outliers among 100 measurements (10.00%)
  1 (1.00%) high mild
  9 (9.00%) high severe
serialize_compress/["postcard"]["bz"][Best]
                        time:   [15.923 µs 16.040 µs 16.176 µs]
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe
serialize_compress/["postcard"]["lzma"][Default]
                        time:   [15.624 µs 15.731 µs 15.859 µs]
Found 10 outliers among 100 measurements (10.00%)
  3 (3.00%) high mild
  7 (7.00%) high severe
serialize_compress/["postcard"]["lzma"][Fastest]
                        time:   [15.753 µs 15.857 µs 15.983 µs]
Found 16 outliers among 100 measurements (16.00%)
  4 (4.00%) high mild
  12 (12.00%) high severe
serialize_compress/["postcard"]["lzma"][Best]
                        time:   [15.759 µs 15.846 µs 15.948 µs]
Found 13 outliers among 100 measurements (13.00%)
  6 (6.00%) high mild
  7 (7.00%) high severe
serialize_compress/["postcard"]["deflate"][Default]
                        time:   [15.705 µs 15.867 µs 16.056 µs]
Found 7 outliers among 100 measurements (7.00%)
  5 (5.00%) high mild
  2 (2.00%) high severe
serialize_compress/["postcard"]["deflate"][Fastest]
                        time:   [15.886 µs 16.001 µs 16.141 µs]
Found 10 outliers among 100 measurements (10.00%)
  9 (9.00%) high mild
  1 (1.00%) high severe
serialize_compress/["postcard"]["deflate"][Best]
                        time:   [15.707 µs 15.813 µs 15.936 µs]
Found 13 outliers among 100 measurements (13.00%)
  7 (7.00%) high mild
  6 (6.00%) high severe
serialize_compress/["postcard"]["zstd"][Default]
                        time:   [15.768 µs 15.879 µs 16.015 µs]
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe
serialize_compress/["postcard"]["zstd"][Fastest]
                        time:   [15.701 µs 15.935 µs 16.201 µs]
Found 12 outliers among 100 measurements (12.00%)
  7 (7.00%) high mild
  5 (5.00%) high severe
serialize_compress/["postcard"]["zstd"][Best]
                        time:   [15.661 µs 15.769 µs 15.893 µs]
Found 12 outliers among 100 measurements (12.00%)
  3 (3.00%) high mild
  9 (9.00%) high severe
serialize_compress/["json"]["none"][Default]
                        time:   [300.97 µs 303.37 µs 306.26 µs]
Found 7 outliers among 100 measurements (7.00%)
  5 (5.00%) high mild
  2 (2.00%) high severe
serialize_compress/["json"]["none"][Fastest]
                        time:   [297.55 µs 299.10 µs 300.80 µs]
Found 8 outliers among 100 measurements (8.00%)
  5 (5.00%) high mild
  3 (3.00%) high severe
serialize_compress/["json"]["none"][Best]
                        time:   [297.02 µs 299.23 µs 302.26 µs]
Found 13 outliers among 100 measurements (13.00%)
  5 (5.00%) high mild
  8 (8.00%) high severe
serialize_compress/["json"]["zlib"][Default]
                        time:   [304.24 µs 318.16 µs 339.91 µs]
Found 10 outliers among 100 measurements (10.00%)
  7 (7.00%) high mild
  3 (3.00%) high severe
serialize_compress/["json"]["zlib"][Fastest]
                        time:   [298.59 µs 300.56 µs 302.79 µs]
Found 14 outliers among 100 measurements (14.00%)
  4 (4.00%) high mild
  10 (10.00%) high severe
serialize_compress/["json"]["zlib"][Best]
                        time:   [300.88 µs 303.03 µs 305.48 µs]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high severe
serialize_compress/["json"]["gzip"][Default]
                        time:   [297.65 µs 299.77 µs 302.28 µs]
Found 11 outliers among 100 measurements (11.00%)
  4 (4.00%) high mild
  7 (7.00%) high severe
serialize_compress/["json"]["gzip"][Fastest]
                        time:   [297.03 µs 298.67 µs 300.56 µs]
Found 13 outliers among 100 measurements (13.00%)
  4 (4.00%) high mild
  9 (9.00%) high severe
serialize_compress/["json"]["gzip"][Best]
                        time:   [300.22 µs 302.05 µs 304.19 µs]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild
serialize_compress/["json"]["brotli"][Default]
                        time:   [297.47 µs 299.26 µs 301.45 µs]
Found 12 outliers among 100 measurements (12.00%)
  6 (6.00%) high mild
  6 (6.00%) high severe
serialize_compress/["json"]["brotli"][Fastest]
                        time:   [299.26 µs 301.07 µs 303.10 µs]
Found 11 outliers among 100 measurements (11.00%)
  7 (7.00%) high mild
  4 (4.00%) high severe
serialize_compress/["json"]["brotli"][Best]
                        time:   [300.24 µs 302.26 µs 304.61 µs]
Found 8 outliers among 100 measurements (8.00%)
  8 (8.00%) high mild
serialize_compress/["json"]["bz"][Default]
                        time:   [298.42 µs 299.99 µs 301.80 µs]
Found 10 outliers among 100 measurements (10.00%)
  6 (6.00%) high mild
  4 (4.00%) high severe
serialize_compress/["json"]["bz"][Fastest]
                        time:   [359.36 µs 374.30 µs 390.86 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
serialize_compress/["json"]["bz"][Best]
                        time:   [364.75 µs 377.06 µs 390.34 µs]
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe
serialize_compress/["json"]["lzma"][Default]
                        time:   [591.61 µs 646.63 µs 703.45 µs]
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
serialize_compress/["json"]["lzma"][Fastest]
                        time:   [311.22 µs 318.60 µs 326.92 µs]
Found 6 outliers among 100 measurements (6.00%)
  6 (6.00%) high mild
serialize_compress/["json"]["lzma"][Best]
                        time:   [305.38 µs 313.31 µs 322.79 µs]
Found 8 outliers among 100 measurements (8.00%)
  5 (5.00%) high mild
  3 (3.00%) high severe
serialize_compress/["json"]["deflate"][Default]
                        time:   [299.56 µs 302.47 µs 306.02 µs]
Found 11 outliers among 100 measurements (11.00%)
  5 (5.00%) high mild
  6 (6.00%) high severe
serialize_compress/["json"]["deflate"][Fastest]
                        time:   [298.67 µs 300.77 µs 303.11 µs]
Found 9 outliers among 100 measurements (9.00%)
  4 (4.00%) high mild
  5 (5.00%) high severe
serialize_compress/["json"]["deflate"][Best]
                        time:   [298.31 µs 300.15 µs 302.29 µs]
Found 6 outliers among 100 measurements (6.00%)
  3 (3.00%) high mild
  3 (3.00%) high severe
serialize_compress/["json"]["zstd"][Default]
                        time:   [300.68 µs 303.64 µs 307.28 µs]
Found 13 outliers among 100 measurements (13.00%)
  6 (6.00%) high mild
  7 (7.00%) high severe
serialize_compress/["json"]["zstd"][Fastest]
                        time:   [303.81 µs 308.19 µs 313.34 µs]
Found 11 outliers among 100 measurements (11.00%)
  8 (8.00%) high mild
  3 (3.00%) high severe
serialize_compress/["json"]["zstd"][Best]
                        time:   [300.44 µs 302.89 µs 305.87 µs]
Found 7 outliers among 100 measurements (7.00%)
  3 (3.00%) high mild
  4 (4.00%) high severe

[0xterminator]

Keys

The above quoted benchmark tests have a few key items to be explained:

1. time: The average time taken for an operation (serialize, deserialize, etc.) over multiple iterations.
2. change: The percentage change in performance compared to a baseline or a previous run. Negative values indicate performance improvements, while positive values suggest performance degradation.
3. p-value: A statistical measure to determine the significance of the results. A p-value below 0.05 typically indicates that the results are statistically significant.
4. outliers: Measurements that fall far from the median and mean, which can indicate variability in performance due to external factors or inconsistencies in the process.

Analysis of Results

The results above have been analyzed in terms of averaged time and statistical significance.

serialize (no compression):

• json is the worst

• bincode is the fastest of all

serialize + compression:

• bincode + no compression seems to be fastest, but this is expected as compression takes time

• bincode + gzip/deflate/zstd/zlib/brotli/bz/lzma (def/fastest/best) have all similar performances irregardless of the level of compression selected. Of course, fastest level delivers the best times of all for each compression type benchmarked.

deserialize (no compression):

• json is the worst

• bincode and postcard are quite similar in terms of performance

decompress_deserialize:

Below are the significant results:

• json is the worst of all

• bincode and no compression delivers best perf, but this is expected as there is no compression applied

• bincode + zstd + fastest level delivers good performance

• bincode + zstd + fastest level delivers good performance

• postcard and no compression also delivers great perf., but this is expected as there is no compression applied

• postcard + zstd + fastest level delivers good performance

• postcard + zlib + fastest level delivers good performance

Key Takeaways:

Binary Formats are Generally Faster: Binary serialization formats like Bincode and Postcard tend to perform better in terms of both time and stability under various compression schemes.
JSON is More Costly: JSON, due to its textual nature, generally takes longer to process, which is evident from higher serialization and deserialization times.
Compression Can Affect Performance: Adding compression generally increases the time it takes to serialize and deserialize, as seen with the regressed performance in most compression scenarios. However, it reduces the size of the data, which is not directly shown in these results but can be critical for network transmission or storage.

Compression size analysis:

A few analysis were performed and the following compressed and serialized data sizes were compared in the benchmarks acc. to the serialization and compression schemes:

json 45700 bytes
postcard 11027 bytes
bincode 15710 bytes

It seems that postcard delivers the best data sizes, but that needs to be analyzed further varying the data types.

Conclusions

Bincode and postcard are the two main alternatives to be considered.

bincode shows the fastest serialization with average ~9.1 µs out of all. Applied compression does not really contribute that much to data reduction with bincode which is also due to the type of data we have with little entropy. Serialized data sizes are slightly higher compared to postcard. Seems like bincode with zstd (fastest level) brings the best performance.

Postcard on the other hand, shows ~13.5 µs average time. Here again, compression brings little value which again is due to the little entropy present in the data. Serialized data sizes tend to be smaller compared to bincode. Seems like postcard with zlib (fastest level) brings the best performance.

I would suggest using bincode(+ zstd + fastest) vs. postcard (+zlib + fastest) in cloud-based infrastructure with real-time data and comparing the overall performance based on the following criteria:

• overall data storage for N messages that go through the system
• publishing N messages (time)
• consuming N messages (time)