Given an internally-tagged enum with NewType variants, serializing produces a valid value but deserializing fails. For example:

#[derive(Serialize, Deserialize)]
#[serde(tag = "type")]
enum TestEnum {
    Foo(TestType),
    Bar(TestType),
}

#[derive(Serialize, Deserialize)]
struct TestType {
  inner: u32,
}

let test_enum1 = TestEnum::Foo(TestType{inner: 123});
let serialized = rmp_serde::to_vec(&test_enum1).unwrap();
let test_enum2 = rmp_serde::from_slice(&serialized).unwrap();

The serializer produces [146, 163, 70, 111, 111, 123], equivalent to the JSON value ["Foo", 123]. That's the value I was expecting.

The deserializer fails with the error Syntax("invalid type: sequence, expected any value").

Everything works if I remove the line #[serde(tag = "type")], but then the serializer produces [146, 0, 145, 145, 123], equivalent to the JSON value [0, [[123]]]. If possible I'd prefer to use the internally tagged representation.

When playing around with afl, I found the following example:

extern crate rmpv;

use std::io::Cursor;

fn main() {
    let data = [219, 175, 142, 142, 201, 219, 128, 0, 50, 175, 142, 196, 100, 212, 185];
    let mut cursor = Cursor::new(data);
    let decoded = rmpv::decode::value::read_value(&mut cursor);
    println!("Done: {:?}", decoded);
}

It takes almost 200ms in release mode and 2min 47s in debug mode to run.

We recently updated from 0.14.0 to 0.14.1 (and to serde version 1.0.104) and the following minimal test case

 use std::net::{IpAddr, Ipv4Addr};

use rmp_serde::{from_slice, to_vec};

#[test]
fn roundtrip_ip_addr() {
    let addr = IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1));
    let addr1: IpAddr = from_slice(&to_vec(&addr).unwrap()).unwrap();

    assert_eq!(addr1, addr);
}

started to fail with

Syntax("invalid type: integer `0`, expected `V4` or `V6`")'

Hi, I want to deserialize stream of MessagePack structures with zero-copy.

But when I create ReadRefReader with:

    let contents = std::fs::read(&path).unwrap();
    let mut deserializer = Deserializer::from_read_ref(&contents);

... and then deserialize a first struct:

    let msg = Message::deserialize(&mut deserializer)?;

I have no way of knowing where did the deserializer end.

However, if I would add this into the impl<'de, R> Deserializer<ReadRefReader<'de, R>>:

    pub fn get_buf(&self) -> &'de [u8] {
        self.rd.buf
    }

... and used it like this:

    let mut buf = &contents[..];

    let mut result = Vec::new();

    loop {
        let mut deserializer = Deserializer::from_read_ref(&buf[..]);

        if let Ok(event) = Message::deserialize(&mut deserializer) {
            result.push(event);
        }
        else {
            break;
        }

        buf = deserializer.get_buf();
    }

It would work.

So the question is - am I missing something? Is there a way how to deserialize streaming MessagePack structs from buffer reference? If not - would it be possible to add the get_buf() method? And if not... could you think of a better way how to implement this?

From this comment:

For c-style enums this serializes quite poorly.

enum Foo {
   A,
   B,
   C
}

Resulting msgpack is {"A": None} where I expected "A". Can this be fixed before release or does this need another option, were I to open a PR?

It generally seems like rmp's output cannot really be consumed from other languages.

serde_json does this optimization, and it seems pretty easy: https://github.com/serde-rs/json/blob/master/src/ser.rs#L883-L890. If I recall correctly, we'll need to modify the deserializer to accept this kind of enum serialization too - but that shouldn't be too bad.

For consistency, I think we should do this both when writing string variant names and when writing integer variant indices.

I'm not sure how breaking this change would be with our policies. On an upgrade, it would cause the serializer to produce different output for the same enum, output which would not be accepted by older versions of this library. We could also just add this as an option, and make it the default in the next major version?

I've recently encountered compilation breakage when upgrading from 0.14.0 to 0.14.2:

error[E0107]: wrong number of type arguments: expected 2, found 1
  --> ykpack/src/encode.rs:11:10
   |
11 |     ser: Serializer<&'a mut dyn Write>,
   |          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected 2 type arguments

This is because Serializer had a new type parameter C added.

Shouldn't 0.14.x have been bumped to 0.15.0 at the time this was introduced?

The previous version of the msgpack spec (v4.0) does not have the concept of UTF-8 strings - only a 'raw' type which admits any binary data. In the v5.0 revision the markers used by msgpack to indicate 'raw' data in v4.0 were repurposed (yes really) to mean UTF-8 strings, and a new set of markers for binary data were introduced; consequently, if you attempt to parse msgpack data from a 4.0 compliant implementation in strict accordance with the 5.0 spec, it is possible to receive Utf8Error:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Utf8Error(Utf8Error { valid_up_to: 5 })'

Could we get a field attribute that allows string fields to be interpreted as binary so that old msgpack files can be parsed successfully?

Edit: As of 2019-06-20, I've made some edits to this PR, and rewritten the main description. Original description is collapsed below.

This adds VariantStringSerializer and VariantIntegerSerializer to modify the encoding of enum variants similar to how the current StructMapSerializer and StructTupleSerializer behave.

The serializer code itself should be pretty straightforward. Each new serializer forwards all of its methods to the inner one, except those necessary to encode variants. For those, it will encode the variant, then forward to the corresponding non-variant method so that chaining StructMapSerializer with VariantStringSerializer works together correctly.

For wrapping StructMapSerializer with VariantStringSerializer, I had to introduce a new trait. Since both serializers modify the Serializer::serialize_struct_variant method, they could not really forward to eachother properly using just the Serializer trait. SerializeParts is a new trait for doing sub-part serialization, and in particular it has a method for serializing just the ident of an enum variant.

For deserializing this, I removed the code in VariantAccess which limited variants to be only integers. Since serde naturally allows for using strings to identify enums, nothing more was necessary.

Fixes #172.

Supersedes and closes #154.

Some low- and high-level functions are signal unsafe. That means if Read or Write given returns an Err with EINTR there are no way to continue decoding/encoding properly.

• [x] Investigate which functions and traits are signal unsafe by marking it in a documentation.
• [x] Implement signal-safe alternatives if possible.

https://github.com/3Hren/msgpack-rust/issues/129

Changed encoding of struct A(B) from [B] to B. This PR is for newtype struct only. Variants and units are harder, so I'd like to discuss it first.

I'm (slowly) working on #40, and I'm running into a problem with representing heterogenous lists (for arguments). I've resorting to using Vec<Value> as a struct field. However, since Value isn't encodable, it's difficult to serialize it as part of a larger struct.

https://github.com/3Hren/msgpack-rust/blob/941441f484c325464c9746fdca9d11c1ca6a2d78/rmp-serde/src/config.rs#L58

It seems it doesn't actually write enum variants by index, it uses idents as strings to prefix enum variant.

Impl: https://github.com/3Hren/msgpack-rust/blob/941441f484c325464c9746fdca9d11c1ca6a2d78/rmp-serde/src/config.rs#L87-L97

hello, I would like to ask if any of you could help me write a serializer for HashMap<Vec<u8,rmpv::Value>? because when I use .to_vec() it stores keys as a list of ints and this is not good or backwards compatible

Hi! Looks like rmp_serde::to_vec works not properly for rmpv::Value with map inside it. I'm expected that result is encoded message pack vector, but result is a map. For example:

#[test]
fn serialize_value_as_map() {
    let value = rmpv::Value::Map(vec![
        (rmpv::Value::from("a"), rmpv::Value::from(1)),
        (rmpv::Value::from("b"), rmpv::Value::from(2)),
    ]);
    let encoded = rmp_serde::to_vec(&value).unwrap();

    assert_eq!(vec![130, 161, 97, 1, 161, 98, 2], encoded);
}

Hello,

I noticed that between versions 0.15.4 and 1.0.0 the enumerations are serialized by name rather than index.

Is it possible by configuration to achieve back the old behavior?

use rmp_serde::Serializer;
use serde::Serialize;

extern crate rmp_serde;
extern crate serde;
#[macro_use]
extern crate serde_derive;

#[derive(Debug, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
struct  A {
    b: B,
}

#[derive(Debug, Serialize, Deserialize)]
#[serde(tag = "kind", content = "data")]
#[serde(rename_all = "snake_case")]
pub enum B {
    First(i32),
    Second(Messagedata)
}

#[derive(Debug, Serialize, Deserialize)]
#[serde(tag = "someKey")]
pub struct Messagedata {
    status: String,
    message: String,
}

fn main() {
    let a = A {
        b: B::Second (Messagedata { status: "Failure".to_string(), message: "You are not getting the things".to_string() }),
    };
    //SERDE
    let string_data = serde_json::to_string(&a).unwrap();
    println!("data: {:?}", string_data);
    let data = serde_json::from_str::<A>(&string_data).unwrap();
    println!("A: {:?}", data);
    println!("-------------------------------------------------------------------------------");

    //MESSAGEPACK
    let mut buf = Vec::new();
    a.serialize(&mut Serializer::new(&mut buf)).unwrap();
    // println!("a: {:?}", buf);
    let a = rmp_serde::from_slice::<A>(&buf).unwrap();
    println!("a: {:?}", a);
}

Here is error I am getting:

data: "{\"b\":{\"kind\":\"second\",\"data\":{\"someKey\":\"Messagedata\",\"status\":\"Failure\",\"message\":\"You are not getting the things\"}}}"
A: A { b: Second(Messagedata { status: "Failure", message: "You are not getting the things" }) }
-------------------------------------------------------------------------------
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: LengthMismatch(2)', src/main.rs:45:46
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

There is no problem with serializing and deserializing with serde. With rmp_serde there is no problem in serializing a struct with a given tag but while deserializing it is not able to take tag provided as key and giving length error.

I can't deserialize an entire file because the Deserializer does not implement into_iter as other serde libraries do.

How can I get around this?

Code thus far is:

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let file_path = "./src/foo.msgpack";
    let reader = BufReader::new(File::open(file_path).unwrap());
    let writer = BufWriter::new(File::create("./src/results.json").unwrap());

    let mut deserializer = rmp_serde::Deserializer::from_read(reader);

    // let mut serializer = serde_json::Serializer::new(io::stdout());
    let mut serializer = serde_json::Serializer::pretty(writer);

    serde_transcode::transcode(&mut deserializer, &mut serializer).unwrap();
    serializer.into_inner().flush().unwrap();

    Ok(())
}