Zip implementation in Rust

Related tags

Compression zip
Overview

zip-rs

Build Status Crates.io version

Documentation

Info

A zip library for rust which supports reading and writing of simple ZIP files.

Supported compression formats:

  • stored (i.e. none)
  • deflate
  • bzip2

Currently unsupported zip extensions:

  • Encryption
  • Multi-disk

Usage

With all default features:

[dependencies]
zip = "0.5"

Without the default features:

[dependencies]
zip = { version = "0.5", default-features = false }

The features available are:

  • deflate: Enables the deflate compression algorithm, which is the default for zipfiles
  • bzip2: Enables the BZip2 compression algorithm.
  • time: Enables features using the time crate.

All of these are enabled by default.

MSRV

Our current Minimum Supported Rust Version is 1.34.0. When adding features, we will follow these guidelines:

  • We will always support the latest four minor Rust versions. This gives you a 6 month window to upgrade your compiler.
  • Any change to the MSRV will be accompanied with a minor version bump
    • While the crate is pre-1.0, this will be a change to the PATCH version.

Examples

See the examples directory for:

  • How to write a file to a zip.
  • How to write a directory of files to a zip (using walkdir).
  • How to extract a zip file.
  • How to extract a single file from a zip.
  • How to read a zip from the standard input.
Comments
  • Move to a GitHub organization or add collaborators to this repository?

    Move to a GitHub organization or add collaborators to this repository?

    As the README says:

    Unfortunately, due to a lack of time and loss of interest, this project will no longer be actively maintained.

    Since zip-rs crate is widely used(crates.io reverse dependencies show 120), how about moving this repository to a GitHub organization or add collaborators to this repository and let the community maintain this crate?

    opened by messense 20
  • Add new APIs that allow copying zip file entries between zip files

    Add new APIs that allow copying zip file entries between zip files

    The copy is done directly using the raw compressed data, avoiding decompression and recompression.

    In summary, the changes:

    • Add a private API for ZipFile to get the raw stream that doesn't do any CRC check or decompression.
    • Add the copy_file and copy_file_rename to ZipWriter that use that raw stream to copy the file.

    I get nearly 95% faster Zip file modifications when I copy nearly all files, and just ignore the files I am removing from the Zip.

    closes #95

    opened by robmv 20
  • Async support

    Async support

    This is now in a good state. ~~To be clear, this only implements async zip reading, writing is not added in this PR~~.

    Fixes https://github.com/zip-rs/zip/issues/124

    opened by zacps 19
  • concurrent reads

    concurrent reads

    Hello Maintainer,

    Firstly, thank you for creating and maintaining this library!

    Secondly, are you interested in adapting this library to support concurrent reads?

    For my application, I need to read entries (at random) from a zip file over an extended period of time. The reads might be concurrent, and might be interleaved (read a few bytes from zip entry A, then a few bytes from zip entry B, then A again, etc).

    The current API shape makes this use-case impossible without creating multiple ZipArchives.

    I've hacked the code to accomplish my goals, and thought I'd share it here in case you or any library users are interested in doing something similar.

    I'd rather not maintain a separate fork of this library just to suit my use-case if it can be avoided, so if there is interest in incorporating any kind of support for concurrent reads, I'm happy to help out.

    If there is no interest, please don't hesitate to close this issue.

    opened by mark-buer 17
  • Save multi `ZipFile` of a same `ZipArchive`, and use it later, will panic

    Save multi `ZipFile` of a same `ZipArchive`, and use it later, will panic

        let zip_file = fs::File::open(PATH)?;
        let zip = ZipArchive::new(&zip_file)?;
        let mut files = vec![zip.clone(); zip.len()];
        let mut files: Vec<_> = files
            .iter_mut()
            .enumerate()
            .map(|(i, file)| file.by_index(i).unwrap())
            .collect();
        files.iter_mut().for_each(|x| {
            let mut s = String::new();
            x.read_to_string(&mut s).unwrap();
            println!("{}", s);
        });
    
    `Err` value: Custom { kind: Other, error: "Invalid checksum" }
    

    panicked at x.read_to_string(&mut s).unwrap();

    opened by rise0chen 15
  • Support AES decryption

    Support AES decryption

    This PR adds support for decrypting AES encrypted zip archives (compression method 99). It implements the special non-NIST variant of the CTR mode used by WinZip, but otherwise uses off-the-shelf crypto crates.

    The feature itself is hidden behind an aes feature flag. We will leave it up to upstream to enable or disable it and/or mention it in the docs.

    Encryption can be added if desired, as the decryption methods can be used to encryption as well.

    opened by mbr 15
  • Cannot read zip from byte stream

    Cannot read zip from byte stream

    This is more of a question than it is a bug, I'm sure. Given the following code:

    fn index(req: &HttpRequest) -> FutureResponse<HttpResponse> {
      // Get the zip file in the form of bytes
      return req
        .body()
        .limit(512000)
        .from_err()
        .and_then(move |b: Bytes| {
          println!("{:?}", b);
    
          let mut archive = zip::read::read_zipfile_from_stream(&mut b).unwrap(); // <- error here.
    
          Ok(HttpResponse::NoContent().into())
        })
        .responder()
    }
    

    Obviously I do not have an actual stream, but I do have the bytes. How can I create an archive purely from bytes? I've looked into stdin, but that doesn't quite fit the bill. I'm trying to read an archive that's being sent in the form of a curl request. Is there some way I can achieve this with my current code as the framework?

    opened by jcoleman-techempower 14
  • Wrong minimum version of (likely) thiserror

    Wrong minimum version of (likely) thiserror

    If you run

    cargo +nightly update -Z minimal-versions && cargo +nightly check
    

    on this project (and require at least [email protected], since that crate has a similar bug), you should see the build fail with multiple errors like this one:

    error[E0277]: `?` couldn't convert the error to `ZipError`
       --> […]\.cargo\registry\src\github.com-1ecc6299db9ec823\zip-0.5.9\src\write.rs:713:35
        |
    668 | fn write_central_directory_header<T: Write>(writer: &mut T, file: &ZipFileData) -> ZipResult<()> {
        |                                                                                    ------------- expected `ZipError` because of this
    ...
    713 |     writer.write_all(&extra_field)?;
        |                                   ^ the trait `From<std::io::Error>` is not implemented for `ZipError`
        |
        = note: the question mark operation (`?`) implicitly performs a conversion on the error value using the `From` trait
        = note: required by `from`
    
    error: aborting due to 137 previous errors
    

    It seems that thiserror introduced a new feature between versions 1.0.6 and 1.0.7 without increasing their minor version.

    I suggest specifying

    thiserror = "1.0.7"
    

    and, ideally, yanking all affected versions.

    bug 
    opened by Tamschi 12
  • Broken files after compression

    Broken files after compression

    Hi. When using compression (CompressionMethod::Bzip2 / CompressionMethod::Deflated), some files in the archive are broken. (examples of files in the attachment)

    If you use CompressionMethod::Stored, files are saved correctly

    example.zip broken_archive.zip

    use std::fs::File;
    use zip::CompressionMethod;
    use zip::write::FileOptions;
    use zip::write::ZipWriter;
    
    let files = vec!["file1", "file2"...];
    let arc_opt = FileOptions::default().compression_method(CompressionMethod::Deflated);
    
    let cursor = Cursor::new(Vec::new());
    let mut archive = ZipWriter::new(cursor);
    
    files.iter().for_each(|file| {
        archive.start_file(&file, arc_opt.clone()).unwrap();
    
        let mut file_buf = vec![];
    
        File::open(file).unwrap().read_to_end(&mut file_buf).unwrap();
    
        archive.write(&file_buf).unwrap();
    }
    
    let data = archive.finish().unwrap();
    
    File::create(&archive_path).unwrap().write_all(data.into_inner().as_ref()).unwrap();
    

    Features flags do not affect the result:

    zip = "0.5"
    
    zip = { version = "0.5", features = [ "deflate-zlib", "deflate-miniz" ] }
    

    Am I missing something? Thanks for the any help.

    bug 
    opened by 1271 12
  • Broken ZIP files created by commit a1da686

    Broken ZIP files created by commit a1da686

    PR #290 (rev a1da6860ea423e999b731fca866ed54505e0096e) appears to have broken writing, at least header writing. Creating test.zip by my example code in #287, tools report:

    7-Zip:

    Error: Headers Error
    Unconfirmed start of archive
    
    Warning: Headers Error
    There are some data after the end of the payload data
    

    zip -T test.zip (zip 3.0-11build1 on Ubuntu 20.04):

      error:  invalid compressed data to inflate testfile
    test of test.zip FAILED
    
    zip error: Zip file invalid, could not spawn unzip, or wrong unzip (original files unmodified)
    

    Neither 7-Zip nor unzip can extract the archive created by this library.

    The same code, when using rev 574bb3df1748a89c8cb1dbed65e625526e4496ee, the immediately preceding commit, creates a working ZIP file.

    bug 
    opened by AlyoshaVasilieva 11
  • Add ZipCrypto reading support

    Add ZipCrypto reading support

    Added two new functions to the API ( resolves #64 ):

    ZipArchive::by_name_decrypt
    ZipArchive::by_index_decrypt
    

    These functions behave identically to their already existing counterparts without _decrypt, but take a password in the form of &[u8]. The password is not a &str because the ZipCrypto standard does not define which encoding is used, so UTF-8 cannot represent all possible passwords (for details, please read the comments in zipcrypto.rs).

    The functions integrate seamlessly with the existing API. Substantial restructuring was necessary to insert the CryptoReader between the other readers. The code has been modified to facilitate the addition of further crypto or decompression routines.

    Note that the API for read_zipfile_from_stream does not yet support ZipCrypto reading. However, this is not hard to add. Also, ZipCrypto writing support has not been added at this point in time. I want to make sure that my work is in line with what the authors and the users expect before I continue working on this.

    In addition to that, a ZipCrypto writer will need cryptographically secure randomness (even though the crypto is weak). Where this notoriously hard to get randomness comes from is up for discussion.

    opened by BenjaminRi 10
  • Deletion of zip entries in-place

    Deletion of zip entries in-place

    fixes #283

    This brings the changes made by @rushiiMachine to support in-place deletion.

    I tried my best to keep original authorship, but I fixed a couple tests, documentation and formatting. I'm also adding a test for file deletion on top of @rushiiMachine 's work.

    opened by baloo 3
  • Disable `time` features

    Disable `time` features

    I left these enabled in dev-dependencies, so that the tests still work, but to be honest, I don't think those tests are really useful.

    The reason to disable this is the proc macro stuff has a significant build time impact, and it's not really something zip users need.

    opened by lnicola 0
  • Files inside extracted zips don't have the correct `last_modified` timestamps

    Files inside extracted zips don't have the correct `last_modified` timestamps

    Hi there,

    I'm using this crate in something that uses zips as a backup medium, where I'm looking to compare timestamps of files between zips etc.

    So I can use this crate to save the timestamp into the zip correctly using options.last_modified_time & other apps (e.g. Finder) will extract the files out with the metadata attached -- but when I'm using archive.extract all the files are given a last_modified timestamp of whenever the extract happened.

    Not sure if there's an option on extract that I'm missing?

    Thanks

    opened by neil-morrison44 0
  • to_owned or OwnedZipFile method?

    to_owned or OwnedZipFile method?

    I think it could be beneficial to have some to_owned method here and possibly elsewhere? Or is there perhaps a more fundament way of doing this?

    https://docs.rs/zip/latest/zip/read/struct.ZipArchive.html#method.by_index

    fn by_index2(self, file_number: usize) -> ZipResult<OwnedZipFile>
    
    opened by cottrell 0
  • Bzip2 level = 0 panic

    Bzip2 level = 0 panic

    When using bzip2 with compression level 0, it panics with this error:

    thread 'main' panicked at 'assertion failed: `(left == right)`
      left: `-2`,
     right: `0`', /home/test/.cargo/registry/src/github.com-1ecc6299db9ec823/bzip2-0.4.3/src/mem.rs:123:13
    

    According to docs, 0 is allowed for bzip2: https://docs.rs/zip/0.6.3/zip/write/struct.FileOptions.html#method.compression_method

    bug 
    opened by LumaRay 2
Owner
null
An extremely fast alternative to zip which is written in rust.

Zap Compress and/or encrypt folders fast. Like, really fast. or as some say, blazingly fast. Installation To install Zap, run the following command fr

null 39 Dec 23, 2022
An extremely fast alternative to zip which is written in rust.

Zap Compress and/or encrypt folders fast. Like, really fast. or as some say, blazingly fast. Installation To install Zap, run the following command fr

null 37 Nov 9, 2022
A utility that can download JavaScript and TypeScript module graphs and store them locally in a special zip file.

eszip A utility that can download JavaScript and TypeScript module graphs and store them locally in a special zip file. To create a new archive: > esz

Deno Land 162 Dec 24, 2022
A library to create zip files on a non-seekable writer

A library to create zip files on a non-seekable writer

nyantec GmbH 2 Mar 17, 2022
A Brotli implementation in pure and safe Rust

Brotli-rs - Brotli decompression in pure, safe Rust Documentation Compression provides a <Read>-struct to wrap a Brotli-compressed stream. A consumer

Thomas Pickert 59 Oct 7, 2022
A Rust implementation of the Zopfli compression algorithm.

Zopfli in Rust This is a reimplementation of the Zopfli compression tool in Rust. I have totally ignored zopflipng. More info about why and how I did

Carol (Nichols || Goulding) 76 Oct 20, 2022
Ribzip2 - A bzip2 implementation in pure Rust.

ribzip2 - a comprehensible bzip2 implementation ribzip2 is command line utility providing bzip2 compression and decompression written in pure Rust. It

null 16 Oct 24, 2022
Brotli compressor and decompressor written in rust that optionally avoids the stdlib

rust-brotli What's new in 3.2 into_inner conversions for both Reader and Writer classes What's new in 3.0 A fully compatible FFI for drop-in compatibi

Dropbox 659 Dec 29, 2022
libbz2 (bzip2 compression) bindings for Rust

bzip2 Documentation A streaming compression/decompression library for rust with bindings to libbz2. # Cargo.toml [dependencies] bzip2 = "0.4" License

Alex Crichton 67 Dec 27, 2022
DEFLATE, gzip, and zlib bindings for Rust

flate2 A streaming compression/decompression library DEFLATE-based streams in Rust. This crate by default uses the miniz_oxide crate, a port of miniz.

The Rust Programming Language 619 Jan 8, 2023
Snappy bindings for Rust

Snappy [ Originally forked from https://github.com/thestinger/rust-snappy ] Documentation Usage Add this to your Cargo.toml: [dependencies] snappy = "

Jeff Belgum 14 Jan 21, 2022
Tar file reading/writing for Rust

tar-rs Documentation A tar archive reading/writing library for Rust. # Cargo.toml [dependencies] tar = "0.4" Reading an archive extern crate tar; use

Alex Crichton 490 Dec 30, 2022
Pure Rust bzip2 decoder

bzip2-rs Pure Rust 100% safe bzip2 decompressor. Features Default features: Rust >= 1.34.2 is supported rustc_1_37: bump MSRV to 1.37, enable more opt

Paolo Barbolini 36 Jan 6, 2023
Like pigz, but rust - a cross platform, fast, compression and decompression tool.

?? crabz Like pigz, but rust. A cross platform, fast, compression and decompression tool. Synopsis This is currently a proof of concept CLI tool using

Seth 232 Jan 2, 2023
A reimplementation of the Zopfli compression tool in Rust.

Zopfli in Rust This is a reimplementation of the Zopfli compression tool in Rust. Carol Nichols started the Rust implementation as an experiment in in

null 11 Dec 26, 2022
A Rust application that compress files and folders

Quick Storer This is a Rust application that compress files and folders. Usage Download or build the binary and place it on your desktop, or any other

AL68 & co. 1 Feb 2, 2022
lzlib (lzip compression) bindings for Rust

lzip Documentation A streaming compression/decompression library for rust with bindings to lzlib. # Cargo.toml [dependencies] lzip = "0.1" License Lic

Firas Khalil Khana 8 Sep 20, 2022
banzai: pure rust bzip2 encoder

banzai banzai is a bzip2 encoder with linear-time complexity, written entirely in safe Rust. It is currently alpha software, which means that it is no

Jack Byrne 27 Oct 24, 2022
Simple NoNG songs manager for GD, written in Rust.

nong-manager Simple NoNG songs manager for GD, written in Rust. Powered by Song File Hub (https://songfilehub.com/home) How to use Enter song ID that

Alexander Simonov 4 May 13, 2023