Snappy is a compression/decompression library.

Related tags

Utilities snappy

Snappy, a fast compressor/decompressor.

Build Status Build status


Snappy is a compression/decompression library. It does not aim for maximum compression, or compatibility with any other compression library; instead, it aims for very high speeds and reasonable compression. For instance, compared to the fastest mode of zlib, Snappy is an order of magnitude faster for most inputs, but the resulting compressed files are anywhere from 20% to 100% bigger. (For more information, see "Performance", below.)

Snappy has the following properties:

  • Fast: Compression speeds at 250 MB/sec and beyond, with no assembler code. See "Performance" below.
  • Stable: Over the last few years, Snappy has compressed and decompressed petabytes of data in Google's production environment. The Snappy bitstream format is stable and will not change between versions.
  • Robust: The Snappy decompressor is designed not to crash in the face of corrupted or malicious input.
  • Free and open source software: Snappy is licensed under a BSD-type license. For more information, see the included COPYING file.

Snappy has previously been called "Zippy" in some Google presentations and the like.


Snappy is intended to be fast. On a single core of a Core i7 processor in 64-bit mode, it compresses at about 250 MB/sec or more and decompresses at about 500 MB/sec or more. (These numbers are for the slowest inputs in our benchmark suite; others are much faster.) In our tests, Snappy usually is faster than algorithms in the same class (e.g. LZO, LZF, QuickLZ, etc.) while achieving comparable compression ratios.

Typical compression ratios (based on the benchmark suite) are about 1.5-1.7x for plain text, about 2-4x for HTML, and of course 1.0x for JPEGs, PNGs and other already-compressed data. Similar numbers for zlib in its fastest mode are 2.6-2.8x, 3-7x and 1.0x, respectively. More sophisticated algorithms are capable of achieving yet higher compression rates, although usually at the expense of speed. Of course, compression ratio will vary significantly with the input.

Although Snappy should be fairly portable, it is primarily optimized for 64-bit x86-compatible processors, and may run slower in other environments. In particular:

  • Snappy uses 64-bit operations in several places to process more data at once than would otherwise be possible.
  • Snappy assumes unaligned 32 and 64-bit loads and stores are cheap. On some platforms, these must be emulated with single-byte loads and stores, which is much slower.
  • Snappy assumes little-endian throughout, and needs to byte-swap data in several places if running on a big-endian platform.

Experience has shown that even heavily tuned code can be improved. Performance optimizations, whether for 64-bit x86 or other platforms, are of course most welcome; see "Contact", below.


You need the CMake version specified in CMakeLists.txt or later to build:

git submodule update --init
mkdir build
cd build && cmake ../ && make


Note that Snappy, both the implementation and the main interface, is written in C++. However, several third-party bindings to other languages are available; see the home page for more information. Also, if you want to use Snappy from C code, you can use the included C bindings in snappy-c.h.

To use Snappy from your own C++ program, include the file "snappy.h" from your calling file, and link against the compiled library.

There are many ways to call Snappy, but the simplest possible is

snappy::Compress(, input.size(), &output);

and similarly

snappy::Uncompress(, input.size(), &output);

where "input" and "output" are both instances of std::string.

There are other interfaces that are more flexible in various ways, including support for custom (non-array) input sources. See the header file for more information.

Tests and benchmarks

When you compile Snappy, the following binaries are compiled in addition to the library itself. You do not need them to use the compressor from your own library, but they are useful for Snappy development.

  • snappy_benchmark contains microbenchmarks used to tune compression and decompression performance.
  • snappy_unittests contains unit tests, verifying correctness on your machine in various scenarios.
  • snappy_test_tool can benchmark Snappy against a few other compression libraries (zlib, LZO, LZF, and QuickLZ), if they were detected at configure time. To benchmark using a given file, give the compression algorithm you want to test Snappy against (e.g. --zlib) and then a list of one or more file names on the command line.

If you want to change or optimize Snappy, please run the tests and benchmarks to verify you have not broken anything.

The testdata/ directory contains the files used by the microbenchmarks, which should provide a reasonably balanced starting point for benchmarking. (Note that baddata[1-3].snappy are not intended as benchmarks; they are used to verify correctness in the presence of corrupted data in the unit test.)


Snappy is distributed through GitHub. For the latest version and other information, see

  • Make CMake sanitizer agnostic

    Make CMake sanitizer agnostic

    Fixes a minor issue in #78 (see and another one (see

    cla: yes 
    opened by bshastry 26
  • Set both VERSION and SOVERSION for target snappy

    Set both VERSION and SOVERSION for target snappy

    Set proper values for properties VERSION and SOVERSION of target snappy, so that CMake produces the link to the shared library that is built. This link was available in 1.1.4 (with autotools) and it is still necessary for the run time linker to work. Its absence leads to run time errors like "error while loading shared libraries: cannot open shared object file: No such file or directory".

    opened by gdsotirov 26
  • Add libFuzzer harness and cmake option to build it

    Add libFuzzer harness and cmake option to build it


    I noticed that snappy is a security critical dependency of chromium and decided to create a fuzzer for it.

    This PR

    • adds a libFuzzer style harness for fuzzing snappy's compressor-decompressor
    • adds a CMake option (off by default) for build the said harness

    Once this PR is reviewed and approved, I plan to create a PR to oss-fuzz to upstream this fuzzer.

    CC @dor1s @inferno-chromium @kcc

    cla: yes 
    opened by bshastry 20
  • Error for missing README

    Error for missing README

    Should not this rule produce README, instead of README.tmp? Otherwise, I receive the following error:

    cat > README.tmp
     /usr/bin/mkdir -p '/usr/src/tmp/package-snappy/usr/doc/snappy-1.1.5'
     /usr/bin/ginstall -c -m 644 ChangeLog COPYING INSTALL NEWS ./README format_description.txt framing_format.txt '/usr/src/tmp/package-snappy/usr/doc/snappy-1.1.5'
    /usr/bin/ginstall: cannot stat './README': No such file or directory
    Makefile:767: recipe for target 'install-dist_docDATA' failed
    make[1]: *** [install-dist_docDATA] Error 1
    make[1]: Leaving directory '/usr/src/tmp/snappy-1.1.5'
    Makefile:1230: recipe for target 'install-am' failed
    make: *** [install-am] Error 2
    opened by gdsotirov 20
  • CMake improvements

    CMake improvements

    So i tried to get my CMake wishes into the previous PRs, some of it landed, this is the rest.

    Since the changed CMake config is pretty important for downstream use, i hope this can be included before the initial CMake release. is my take on one of CMake’s weak points, autotools allowed to do this before.

    opened by Optiligence 15
  • Revert

    Revert "Improve zippy decompression speed."

    This reverts commit 8bfb028b618747a6ac8af159c87e9196c729566f.

    The commit 8bfb028 (Improve zippy decompression speed) introduce a crash when snappy is compiled with _FORTIFY_SOURCE with musl libc. Backtrace reveals that it it comes from using memcpy with overlap. Since this may have security implications we better revert it for now.

    Bactrace from core dump created with make check:

    (gdb) bt
     #0  memcpy (__n=8, __os=0xb38c8367eea, __od=0xb38c8367eeb)
         at /usr/include/fortify/string.h:48
     #1  snappy::(anonymous namespace)::UnalignedCopy64 (src=0xb38c8367eea,
         dst=0xb38c8367eeb) at
     #2  0x00006fb4c7c31717 in snappy::(anonymous namespace)::IncrementalCopy
         buf_limit=0xb38c8380ee0 "", op_limit=<optimized out>, op=<optimized
         src=0xb38c8367eea " .\001") at
     #3  snappy::SnappyArrayWriter::AppendFromSelf (len=<optimized out>,
         offset=<optimized out>, this=<synthetic pointer>) at
         writer=<synthetic pointer>, this=0x7f1d26737050) at
     #5  snappy::InternalUncompressAllTags<snappy::SnappyArrayWriter> (
         uncompressed_len=<optimized out>, writer=<synthetic pointer>,
         decompressor=0x7f1d26737050) at
     #6  snappy::InternalUncompress<snappy::SnappyArrayWriter> (
         writer=<synthetic pointer>, r=0x7f1d26737000) at
     #7  snappy::RawUncompress (compressed=compressed@entry=0x7f1d267370c0,
         uncompressed=0xb38c8367ee0 "  content: .\001") at
     #8  0x00006fb4c7c3194d in snappy::RawUncompress (compressed=<optimized
         n=<optimized out>, uncompressed=<optimized out>) at
     #9  0x00000b38c5c6261a in snappy::BM_UFlat (iters=99, arg=<optimized
     #10 0x00000b38c5c69846 in snappy::Benchmark::Run (this=0xb38c8323c40)
     #11 0x00000b38c5c5f3fd in RunSpecifiedBenchmarks () at snappy-test.h:485
     #12 main (argc=1, argv=0x7f1d267374b8) at
    opened by ncopa 14
  • In gcc __powerpc64__ not __ppc64__ defines the PPC64 architecture

    In gcc __powerpc64__ not __ppc64__ defines the PPC64 architecture

    Corrects 18488d6212331fee647ecfded85353ab3ad91de8 and still maintains clang compatibility (like the #27 originally).

    Compiler tests:

    $ uname -m ppc64le $ cat /tmp/x.c

    $ gcc -c /tmp/x.c /tmp/x.c:3:2: warning: #warning powerpc64 exists [-Wcpp] #warning powerpc64 exists ^~~~~~~ /tmp/x.c:11:2: error: #error ppc64 not defined #error ppc64 not defined ^~~~~ $ clang /tmp/x.c /tmp/x.c:3:2: warning: powerpc64 exists [-W#warnings] ^ /tmp/x.c:9:2: warning: ppc64 exists [-W#warnings] ^ 2 warnings generated.

    I should be on the CLA list already.

    cla: yes 
    opened by grooverdan 12
  • Added pkg-config file and .gitignore

    Added pkg-config file and .gitignore

    This is a redo of, which had merge conflicts and seemed abandoned.

    I implemented the suggested fix of using the variables provided by GNUInstallDirs

    A notable difference is that the snappy.pc is now installed in <prefix>/lib/pkgconfig instead of <prefix>/share/pkgconfig, which seems to be more in line with the other packages installing libraries.

    Grabbed the absolute/relative logic from; when building in Nixos, it overrides GNUInstallDirs to absolute paths, which may not be in the prefix:

    Verification using a relative CMAKE_INSTALL_LIBDIR:

    $ cat libsnappy.pc
    Name: snappy
    Description: Fast compressor/decompressor library.
    Version: 1.1.7
    Libs: -L${prefix}/lib -lsnappy
    Cflags: -I${prefix}/include
    $ pkg-config libsnappy --libs
    -L/usr/local/lib -lsnappy
    $ pkg-config libsnappy --cflags

    With an absolute CMAKE_INSTALL_LIBDIR, building with nix:

    Name: snappy
    Description: Fast compressor/decompressor library.
    Version: 1.1.7
    Libs: -L/nix/store/hknzbw87dnmz7zq0lf99vjf7bf2l2hps-snappy-1.1.7/lib -lsnappy
    Cflags: -I/nix/store/z67qhiawswr2jq1p71r4f2987czx3slh-snappy-1.1.7-dev/include
    $ pkg-config libsnappy --libs
    -L/nix/store/hknzbw87dnmz7zq0lf99vjf7bf2l2hps-snappy-1.1.7/lib -lsnappy
    $ pkg-config libsnappy --cflags

    Installing the snappy gem (i.e. it properly detected it and didn't recompile snappy):

    $ time gem install snappy
    Fetching snappy-0.0.17.gem
    Building native extensions. This could take a while...
    Successfully installed snappy-0.0.17
    Parsing documentation for snappy-0.0.17
    Installing ri documentation for snappy-0.0.17
    Done installing documentation for snappy after 0 seconds
    1 gem installed
    gem install snappy  1.25s user 0.57s system 80% cpu 2.270 total
    wontfix cla: yes 
    opened by lavoiesl 11
  • Allow building with cmake (to ease building on windows)

    Allow building with cmake (to ease building on windows)

    The patch series allows for building on windows with the following steps:

    git clone git://
    mkdir build
    cd build
    cmake -G "NMake Makefiles" ..\snappy
    nmake all test

    (It should also build on unix systems by dropping the "-G NMake Makefiles" part)

    opened by trondn 10
  • fix cmake build error

    fix cmake build error

    I want to install libsnappy on linux mint 19. I already install GTest from github, but make complained undefined reference:

    [100%] Linking CXX executable snappy_unittest
    CMakeFiles/snappy_unittest.dir/ In function `snappy::Snappy_ZeroOffsetCopy_Test::TestBody()': undefined reference to `testing::Message::Message()'

    It seems like CMakeList not link0 to libgtest. Fix it by adding target_link_libraries(snappy_unittest gtest) in CMakelists.txt

    cla: yes 
    opened by LYKZZzz 9
  • Fix UBSan error (ptr + offset overflow)

    Fix UBSan error (ptr + offset overflow)

    As i + offset is promoted to a "negative" size_t, UBSan would complain when adding the resulting offset to dst:

    /tmp/RtmptDX1SS/file584e37df4e/snappy_ep-prefix/src/snappy_ep/ runtime error: addition of unsigned offset to 0x6120003c5ec1 overflowed to 0x6120003c5ec0
        #0 0x7f9ebd21769c in snappy::(anonymous namespace)::Copy64BytesWithPatternExtension(char*, unsigned long) /tmp/RtmptDX1SS/file584e37df4e/snappy_ep-prefix/src/snappy_ep/
        #1 0x7f9ebd21769c in std::__1::pair<unsigned char const*, long> snappy::DecompressBranchless<char*>(unsigned char const*, unsigned char const*, long, char*, long) /tmp/RtmptDX1SS/file584e37df4e/snappy_ep-prefix/src/snappy_ep/
    cla: yes 
    opened by pitrou 8
  • Add CIFuzz GitHub Action

    Add CIFuzz GitHub Action

    Add CIFuzz workflow action to have fuzzers build and run on each PR.

    This is a service offered by OSS-Fuzz where Snappy already runs ( CIFuzz can help detect regressions and catch fuzzing build issues early, and has a variety of features (see the URL above). In the current PR the fuzzers gets build on a pull request and will run for 300 seconds.

    Signed-off-by: David Korczynski [email protected]

    opened by DavidKorczynski 0
  • Add new framing chunk types without checksums

    Add new framing chunk types without checksums

    Adds two new chunk types to the Snappy framing format: compressed data without a checksum, and uncompressed data without a checksum. These types are identical to their existing counterparts except they do not contain a CRC-32C checksum. Essentially, this makes including checksums for each data chunk optional rather than required.

    In some use cases, computing the CRC-32C checksums for the data chunks in the Snappy framing format ends up dominating execution time. Eliminating the checksums provides massive 2.5x performance improvements in our uses of Snappy for compressing address trace data prior to storing to disk.

    Existing readers of the Snappy framing format would be expected to fail up front on an unknown chunk type when encountering the new types, until updated to handle them, which should be a simple coding change.

    opened by derekbruening 1
  • Allow compiling with MSVC-compatible compilers like clang-cl

    Allow compiling with MSVC-compatible compilers like clang-cl

    In order to be able to compile with clang-cl, test for MSVC rather than CMAKE_CXX_COMPILER_ID STREQUAL "MSVC". This is true for other MSVC command-line compatible compilers, i.e. clang-cl.

    opened by goedderz 0
  • 1.1.9(May 4, 2021)

  • 1.1.8(Jan 14, 2020)

  • 1.1.7(Aug 25, 2017)

    • Improved CMake build support for 64-bit Linux distributions.
    • MSVC builds now use MSVC-specific intrinsics that map to clzll.
    • ARM64 (AArch64) builds use the code paths optimized for 64-bit processors.
    Source code(tar.gz)
    Source code(zip)
  • 1.1.6(Jul 13, 2017)

  • 1.1.5(Jun 29, 2017)

    This release has broken SONAME / SOVERSION values. Users of snappy as a shared library should avoid 1.1.5 and use 1.1.6 instead. SONAME / SOVERSION errors will manifest as the dynamic library loader complaining that it cannot find snappy's shared library file ( / libsnappy.dylib), or that the library it found does not have the required version. 1.1.6 has the same code as 1.1.5, but carries build configuration fixes for the issues above.

    • Add CMake build support. The autoconf build support is now deprecated, and will be removed in the next release.
    • Add AppVeyor configuration, for Windows CI coverage.
    • Small performance improvement on little-endian PowerPC.
    • Small performance improvement on LLVM with position-independent executables.
    • Fix a few issues with various build environments.
    Source code(tar.gz)
    Source code(zip)
  • 1.1.4(Jan 27, 2017)

  • 1.1.3(Jul 7, 2015)

    This is the first release to be done from GitHub, which means that some minor things like the ChangeLog format has changed (git log format instead of svn log).

    • Add support for Uncompress() from a Source to a Sink.
    • Various minor changes to improve MSVC support; in particular, the unit tests now compile and run under MSVC.
    Source code(tar.gz)
    Source code(zip)
    snappy-1.1.3.tar.gz(1.43 MB)
Google ❤️ Open Source
archive-rs provides a generic way of dealing with multiple archive and compression formats in Rust

archive-rs A Rust crate that aims to provide a generic way of dealing with multiple archive and compression formats by providing a generic abstraction

S.J.R. van Schaik 2 Nov 21, 2021
A library to compile USDT probes into a Rust library

sonde sonde is a library to compile USDT probes into a Rust library, and to generate a friendly Rust idiomatic API around it. Userland Statically Defi

Ivan Enderlin 40 Jan 7, 2023
A Rust library for calculating sun positions

sun A rust port of the JS library suncalc. Install Add the following to your Cargo.toml [dependencies] sun = "0.2" Usage pub fn main() { let unixti

Markus Kohlhase 36 Dec 28, 2022
A cross-platform serial port library in Rust.

Introduction serialport-rs is a general-purpose cross-platform serial port library for Rust. It provides a blocking I/O interface and port enumeration

Bryant Mairs 143 Nov 5, 2021
The feature-rich, portable async channel library

The feature-rich, portable async channel library > > Why use Postage? Includes a rich set of channels. | barrier | broadcast | dispa

Austin Jones 221 Dec 26, 2022
A high level diffing library for rust based on diffs

Similar: A Diffing Library Similar is a dependency free crate for Rust that implements different diffing algorithms and high level interfaces for it.

Armin Ronacher 617 Dec 30, 2022
A reactive DOM library for Rust in WASM

maple A VDOM-less web library with fine-grained reactivity. Getting started The recommended build tool is Trunk. Start by adding maple-core to your Ca

Luke Chu 1.8k Jan 3, 2023
transmute-free Rust library to work with the Arrow format

Arrow2: Transmute-free Arrow This repository contains a Rust library to work with the Arrow format. It is a re-write of the official Arrow crate using

Jorge Leitao 708 Dec 30, 2022
Cross-platform Window library in Rust for Tauri. [WIP]

Cross-platform application window creation library in Rust that supports all major platforms like Windows, macOS, Linux, iOS and Android. Built for you, maintained for Tauri.

Tauri 899 Jan 1, 2023
Node/Electron library for global key listening.

GlobalKey Building cargo install nj-cli nj-cli build --release Calling from node npm i globalkey # or yarn add globalkey const globalkey = require(

Will 20 Dec 15, 2022
An opinionated, practical color management library for games and graphics.

colstodian An opinionated color management library built on top of kolor. Introduction colstodian is a practical color management library for games an

Gray Olson 27 Dec 7, 2022
A library in Rust for theorem proving with Intuitionistic Propositional Logic.

Prop Propositional logic with types in Rust. A library in Rust for theorem proving with Intuitionistic Propositional Logic. Supports theorem proving i

AdvancedResearch 48 Jan 3, 2023
A Rust library for constructing tilings of regular polygons

tiling tiling is a library for constructing tilings of regular polygons and their dual tilings. Resources Documentation Tilings by regular polygons Li

Jonas Michel 29 Sep 30, 2021
bevy_blender is a Bevy library that allows you to use assets created in Blender directly from the .blend file

bevy_blender bevy_blender is a Bevy library that allows you to use assets created in Blender directly from the .blend file.

Jerald Thomas 45 Jan 4, 2023
A low-level I/O ownership and borrowing library

This library introduces OwnedFd, BorrowedFd, and supporting types and traits, and corresponding features for Windows, which implement safe owning and

Dan Gohman 74 Jan 2, 2023
miette is a diagnostic library for Rust. It includes a series of traits/protocols that allow you to hook into its error reporting facilities, and even write your own error reports!

miette is a diagnostic library for Rust. It includes a series of traits/protocols that allow you to hook into its error reporting facilities, and even write your own error reports!

Kat Marchán 1.2k Jan 1, 2023
Generative arts library in Rust

Generative Generative (WIP) is 2D generational arts creation library written in Rust. Currently it is in nascent stage and is somewhat unstable. Examp

Gaurav Patel 22 May 13, 2022
Utility library to work with tuples.

Utility library to work with tuples.

René Kijewski 9 Nov 30, 2022
Membrane is an opinionated crate that generates a Dart package from a Rust library. Extremely fast performance with strict typing and zero copy returns over the FFI boundary via bincode.

Membrane is an opinionated crate that generates a Dart package from a Rust library. Extremely fast performance with strict typing and zero copy returns over the FFI boundary via bincode.

Jerel Unruh 70 Dec 13, 2022