The example code works fine for me; if I write the file using:

let f = std::fs::File::create("/tmp/test.xz").unwrap();
let mut f = lzma::LzmaWriter::new_compressor(f, 6).unwrap();
write!(f, "Hello LZMA World!").unwrap();
f.finish().unwrap();

(I'm new to Rust, so forgive me if my code has a weird accent.)

Then I can read it back in with:

let file = std::fs::File::open("/tmp/test.xz").unwrap();
let mut file = lzma::LzmaReader::new_decompressor(file).unwrap();
let mut contents = String::new();
file.read_to_string(&mut contents).expect("read failed");
println!("{}", contents);

However, if I create a file on the command line:

$ echo 'Hello LZMA World!' | lzma > /tmp/test.lzma
$ lzmadec /tmp/test.lzma 
Hello LZMA World!

Then the above reading code (with the file name changed, of course) fails:

thread 'main' panicked at 'read failed: Custom { kind: Other, error: Format }', libcore/result.rs:1009:5

Indeed there is a difference between these two files:

$ file /tmp/test.xz
/tmp/test.xz: XZ compressed data
$ file /tmp/test.lzma 
/tmp/test.lzma: LZMA compressed data, streamed

Conversely, lzmadec can't read the .xz file:

$ lzmadec /tmp/test.xz
lzmadec: /tmp/test.xz: File format not recognized

So is the library mis-named? Can it actually read LZMA, as opposed to XZ? The compression algorithm is the same, but the container format is different.

It's worth noting that both my compiled Rust binary and /usr/bin/lzma are linked with /usr/lib/liblzma.so.5, so the functionality should be in there somewhere.

I've noticed that the LzmaWriter write can return zero bytes consumed, but then if called again and again, it will eventually accept more bytes.

This means we cannot use write_all, since it will stop with error ErrorKind::WriteZero as soon as the write above returns zero.

The workaround is ugly,

let todo = buffer.len();
let mut index = 0;
while {
    let bytes = decompressor.write(&buffer[index..]).unwrap();
    index += bytes;

    index < todo
} {}

when it could have been just:

decompressor.write_all(&buffer).unwrap();

I have implemented drop for the lzmawriter so that it is not mandatory to run finish().

I have unfortunately run into situations where it was no longer possible for me to execute finish(). For this purpose I have implemented what is executed in finish() in Drop. With finish(), however, the return value must then be adjusted (see pull request).

With Drop you don't get any feedback in case of an error - but you wouldn't if finish() wasn't executed. I see no disadvantage here at the moment.

For some reason, the current version of the documentation on docs.rs shows "rust-lzma-0.5.1 doesn't have any documentation."

https://docs.rs/crate/rust-lzma/0.5.1

If the write-method reuturns zero bytes written it is considered an error. Since it is possible, that liblzma needs to write out some internal buffers first before consuming more input, write could return 0 in some cases. Since this seems to be normal behaviour of liblzma, this patch makes sure, that all internal buffers are written out and at least some of the input is consumed.

Fixes #13

Error: `thread 'main' panicked at 'Could not find liblzma using pkg-config',

I had to download a Ubuntu lzma package to make it work there. I downloaded a Windows bin - how do I make this compile?

When creating a LzmaWriter object, the enclosed writer is moved. In order to get it back, .finish() returns it. This required the removal of the Drop implementation. But since .finish() must be called, all cleanups can be done there.

Or am I mistaken?

This fixes #14

I've tried using rust-lzma on my Raspberry Pi (Debian Jessie armhf), but unfortunately, it doesn't work, it always fails when writing with "Unsupported compression options". My example:

extern crate lzma;

use lzma::LzmaWriter;
use std::io::prelude::*;
use std::fs::File;

fn main() {
    let f = File::create("foo.xz").unwrap();
    let mut f = LzmaWriter::new_compressor(f, 6).unwrap();

    match write!(f, "It's a small world!") {
        Ok(_) => println!("ok"),
        Err(err) => panic!("error writing: {}", err)
    }
    f.finish().unwrap();
}

When run, I get:

thread '<main>' panicked at 'error writing: Unsupported compression options', examples/compress.rs:13
stack backtrace:
   1: 0x54afc747 - sys::backtrace::tracing::imp::write::hf670d8b6386d22deJcv
   2: 0x54afed67 - panicking::default_handler::_$u7b$$u7b$closure$u7d$$u7d$::closure.44482
   3: 0x54afe8f3 - panicking::default_handler::h8779de026b877d91YTz
   4: 0x54af5dc3 - sys_common::unwind::begin_unwind_inner::h3b511d4750e909fd20t
   5: 0x54af602b - sys_common::unwind::begin_unwind_fmt::hfb02698582574ecb8Zt
   6: 0x54aeeab3 - main::h476f27c94c33d42diaa
                at /home/user/src/rust-lzma/<std macros>:8
   7: 0x54afe4bb - sys_common::unwind::try::try_fn::h5358980013828492550
   8: 0x54afbd23 - __rust_try
   9: 0x54afbd23 - __rust_try
...

Also, all tests fail on the same error:

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Error { repr: Custom(Custom { kind: Other, error: Options }) }', ../src/libcore/result.rs:746
...

I've tried using 0, 6, and 9 as compression preset, no difference. I've looked into your code, but I couldn't find any clue what could be wrong. It works on Debian Jessie x86_64 though. Thanks

Basically, this PR just forces LzmaWriter to call finish before ending the stream. This prevents any accidental corruptions and makes it much easier to write any structs that might contain it, and makes it behave a bit more like compressors in the flate2 and bzip2 libraries.

I can appreciate that perhaps the LzmaWriter was designed to write to files on disk, where the compressed data isn't immediately re-read from disk in the same process. However if we want to compress to memory, where the stream we write to is backed by memory, this becomes a pain.

We currently have to hack around the stream being moved into the LzmaWriter:

let mut backing: Vec<u8> = Vec::with_capacity(backing_size);
{
    let mut compressor = lzma::LzmaWriter::new_compressor(&mut backing, 6).unwrap();
    // ... perform writes ...
    compressor.finish().unwrap();
}
// ... now use compressed data in backing.

If the finish() method of the LzmaWriter returned the inner io::Write, it would reduce to:

let mut compressor = lzma::LzmaWriter::new_compressor(Vec::with_capacity(backing_size), 6).unwrap();
// ... perform writes ...
let backing = compressor.finish().unwrap();
// ... now use compressed data in backing.

The flate2 library does this and makes for some clean code.

Using LzmaWriter after calling finish is not a currently supported feature, so have finish consume self instead of &self.

Credits to krdln for the feedback.

NOTE: liblzma supports compressing/decompressing multiple xz streams. In the future the LzmaWriter API could be enhanced with a finish_and_continue method which finishes the current stream and then starts handling the next stream. Or something like that.

Right now, finish needs to be called on an LzmaWriter to get it to finish compressing/decompressing. This differs from the way a normal Write-like object works, so it's easy to forget and cause weird bugs.

Investigate the suggestion of using Drop to either call finish or panic if finish hasn't been called. BufWriter implements Drop like this: https://static.rust-lang.org/doc/master/src/std/io/buffered.rs.html#501-509

Credits to krdln for the feedback.

LZMAReader and LZMAWriter duplicate a lot of code between each other. It would be nice to eliminate that, as well as the duplication of documentation. It's necessary to keep distinct Reader and Writer types, so they can't just be condensed into a single Struct.

Some parts of the library return Result with an LZMAError, which is good, but the parts of the library that implement the Read and Write traits have to return io::Result. They take any LZMAError errors that arise and squish them into an io::Error. This makes handling those errors outside of the library challenging.

Need to find a better solution.