Skip to content

Commit

Permalink
Upgrade Snappy to 1.2.1
Browse files Browse the repository at this point in the history
  • Loading branch information
big-r81 committed Nov 20, 2024
1 parent e4ea6ef commit 8d619fd
Show file tree
Hide file tree
Showing 12 changed files with 2,679 additions and 1,101 deletions.
26 changes: 26 additions & 0 deletions c_src/snappy/COPYING
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,29 @@ DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

===

Some of the benchmark data in testdata/ is licensed differently:

- fireworks.jpeg is Copyright 2013 Steinar H. Gunderson, and
is licensed under the Creative Commons Attribution 3.0 license
(CC-BY-3.0). See https://creativecommons.org/licenses/by/3.0/
for more information.

- kppkn.gtb is taken from the Gaviota chess tablebase set, and
is licensed under the MIT License. See
https://sites.google.com/site/gaviotachessengine/Home/endgame-tablebases-1
for more information.

- paper-100k.pdf is an excerpt (bytes 92160 to 194560) from the paper
“Combinatorial Modeling of Chromatin Features Quantitatively Predicts DNA
Replication Timing in _Drosophila_” by Federico Comoglio and Renato Paro,
which is licensed under the CC-BY license. See
http://www.ploscompbiol.org/static/license for more ifnormation.

- alice29.txt, asyoulik.txt, plrabn12.txt and lcet10.txt are from Project
Gutenberg. The first three have expired copyrights and are in the public
domain; the latter does not have expired copyright, but is still in the
public domain according to the license information
(http://www.gutenberg.org/ebooks/53).
60 changes: 60 additions & 0 deletions c_src/snappy/NEWS
Original file line number Diff line number Diff line change
@@ -1,3 +1,63 @@
Snappy v1.1.10, Mar 8th 2023:

* Performance improvements

* Compilation fixes for various environments

Snappy v1.1.9, May 4th 2021:

* Performance improvements.

* Google Test and Google Benchmark are now bundled in third_party/.

Snappy v1.1.8, January 15th 2020:

* Small performance improvements.

* Removed snappy::string alias for std::string.

* Improved CMake configuration.

Snappy v1.1.7, August 24th 2017:

* Improved CMake build support for 64-bit Linux distributions.

* MSVC builds now use MSVC-specific intrinsics that map to clzll.

* ARM64 (AArch64) builds use the code paths optimized for 64-bit processors.

Snappy v1.1.6, July 12th 2017:

This is a re-release of v1.1.5 with proper SONAME / SOVERSION values.

Snappy v1.1.5, June 28th 2017:

This release has broken SONAME / SOVERSION values. Users of snappy as a shared
library should avoid 1.1.5 and use 1.1.6 instead. SONAME / SOVERSION errors will
manifest as the dynamic library loader complaining that it cannot find snappy's
shared library file (libsnappy.so / libsnappy.dylib), or that the library it
found does not have the required version. 1.1.6 has the same code as 1.1.5, but
carries build configuration fixes for the issues above.

* Add CMake build support. The autoconf build support is now deprecated, and
will be removed in the next release.

* Add AppVeyor configuration, for Windows CI coverage.

* Small performance improvement on little-endian PowerPC.

* Small performance improvement on LLVM with position-independent executables.

* Fix a few issues with various build environments.

Snappy v1.1.4, January 25th 2017:

* Fix a 1% performance regression when snappy is used in PIE executables.

* Improve compression performance by 5%.

* Improve decompression performance by 20%.

Snappy v1.1.3, July 6th 2015:

This is the first release to be done from GitHub, which means that
Expand Down
122 changes: 76 additions & 46 deletions c_src/snappy/README → c_src/snappy/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
Snappy, a fast compressor/decompressor.

[![Build Status](https://github.com/google/snappy/actions/workflows/build.yml/badge.svg)](https://github.com/google/snappy/actions/workflows/build.yml)

Introduction
============
Expand Down Expand Up @@ -29,12 +30,12 @@ and the like.

Performance
===========

Snappy is intended to be fast. On a single core of a Core i7 processor
in 64-bit mode, it compresses at about 250 MB/sec or more and decompresses at
about 500 MB/sec or more. (These numbers are for the slowest inputs in our
benchmark suite; others are much faster.) In our tests, Snappy usually
is faster than algorithms in the same class (e.g. LZO, LZF, FastLZ, QuickLZ,
is faster than algorithms in the same class (e.g. LZO, LZF, QuickLZ,
etc.) while achieving comparable compression ratios.

Typical compression ratios (based on the benchmark suite) are about 1.5-1.7x
Expand All @@ -51,8 +52,8 @@ In particular:

- Snappy uses 64-bit operations in several places to process more data at
once than would otherwise be possible.
- Snappy assumes unaligned 32- and 64-bit loads and stores are cheap.
On some platforms, these must be emulated with single-byte loads
- Snappy assumes unaligned 32 and 64-bit loads and stores are cheap.
On some platforms, these must be emulated with single-byte loads
and stores, which is much slower.
- Snappy assumes little-endian throughout, and needs to byte-swap data in
several places if running on a big-endian platform.
Expand All @@ -62,25 +63,41 @@ Performance optimizations, whether for 64-bit x86 or other platforms,
are of course most welcome; see "Contact", below.


Building
========

You need the CMake version specified in [CMakeLists.txt](./CMakeLists.txt)
or later to build:

```bash
git submodule update --init
mkdir build
cd build && cmake ../ && make
```

Usage
=====

Note that Snappy, both the implementation and the main interface,
is written in C++. However, several third-party bindings to other languages
are available; see the Google Code page at http://code.google.com/p/snappy/
for more information. Also, if you want to use Snappy from C code, you can
use the included C bindings in snappy-c.h.
are available; see the [home page](docs/README.md) for more information.
Also, if you want to use Snappy from C code, you can use the included C
bindings in snappy-c.h.

To use Snappy from your own C++ program, include the file "snappy.h" from
your calling file, and link against the compiled library.

There are many ways to call Snappy, but the simplest possible is

snappy::Compress(input.data(), input.size(), &output);
```c++
snappy::Compress(input.data(), input.size(), &output);
```
and similarly
snappy::Uncompress(input.data(), input.size(), &output);
```c++
snappy::Uncompress(input.data(), input.size(), &output);
```

where "input" and "output" are both instances of std::string.

Expand All @@ -92,44 +109,57 @@ information.
Tests and benchmarks
====================

When you compile Snappy, snappy_unittest is compiled in addition to the
library itself. You do not need it to use the compressor from your own library,
but it contains several useful components for Snappy development.

First of all, it contains unit tests, verifying correctness on your machine in
various scenarios. If you want to change or optimize Snappy, please run the
tests to verify you have not broken anything. Note that if you have the
Google Test library installed, unit test behavior (especially failures) will be
significantly more user-friendly. You can find Google Test at

http://code.google.com/p/googletest/

You probably also want the gflags library for handling of command-line flags;
you can find it at

http://code.google.com/p/google-gflags/

In addition to the unit tests, snappy contains microbenchmarks used to
tune compression and decompression performance. These are automatically run
before the unit tests, but you can disable them using the flag
--run_microbenchmarks=false if you have gflags installed (otherwise you will
need to edit the source).

Finally, snappy can benchmark Snappy against a few other compression libraries
(zlib, LZO, LZF, FastLZ and QuickLZ), if they were detected at configure time.
To benchmark using a given file, give the compression algorithm you want to test
Snappy against (e.g. --zlib) and then a list of one or more file names on the
command line. The testdata/ directory contains the files used by the
microbenchmark, which should provide a reasonably balanced starting point for
benchmarking. (Note that baddata[1-3].snappy are not intended as benchmarks; they
are used to verify correctness in the presence of corrupted data in the unit
test.)

When you compile Snappy, the following binaries are compiled in addition to the
library itself. You do not need them to use the compressor from your own
library, but they are useful for Snappy development.

* `snappy_benchmark` contains microbenchmarks used to tune compression and
decompression performance.
* `snappy_unittests` contains unit tests, verifying correctness on your machine
in various scenarios.
* `snappy_test_tool` can benchmark Snappy against a few other compression
libraries (zlib, LZO, LZF, and QuickLZ), if they were detected at configure
time. To benchmark using a given file, give the compression algorithm you want
to test Snappy against (e.g. --zlib) and then a list of one or more file names
on the command line.

If you want to change or optimize Snappy, please run the tests and benchmarks to
verify you have not broken anything.

The testdata/ directory contains the files used by the microbenchmarks, which
should provide a reasonably balanced starting point for benchmarking. (Note that
baddata[1-3].snappy are not intended as benchmarks; they are used to verify
correctness in the presence of corrupted data in the unit test.)

Contributing to the Snappy Project
==================================

In addition to the aims listed at the top of the [README](README.md) Snappy
explicitly supports the following:

1. C++11
2. Clang (gcc and MSVC are best-effort).
3. Low level optimizations (e.g. assembly or equivalent intrinsics) for:
- [x86](https://en.wikipedia.org/wiki/X86)
- [x86-64](https://en.wikipedia.org/wiki/X86-64)
- ARMv7 (32-bit)
- ARMv8 (AArch64)
4. Supports only the Snappy compression scheme as described in
[format_description.txt](format_description.txt).
5. CMake for building

Changes adding features or dependencies outside of the core area of focus listed
above might not be accepted. If in doubt post a message to the
[Snappy discussion mailing list](https://groups.google.com/g/snappy-compression).

We are unlikely to accept contributions to the build configuration files, such
as `CMakeLists.txt`. We are focused on maintaining a build configuration that
allows us to test that the project works in a few supported configurations
inside Google. We are not currently interested in supporting other requirements,
such as different operating systems, compilers, or build systems.

Contact
=======

Snappy is distributed through Google Code. For the latest version, a bug tracker,
and other information, see

http://code.google.com/p/snappy/
Snappy is distributed through GitHub. For the latest version and other
information, see https://github.com/google/snappy.
Loading

0 comments on commit 8d619fd

Please sign in to comment.