Prost is a Protocol Buffers implementation for the Rust Language.

Overview

continuous integration Documentation Crate Dependency Status

PROST!

prost is a Protocol Buffers implementation for the Rust Language. prost generates simple, idiomatic Rust code from proto2 and proto3 files.

Compared to other Protocol Buffers implementations, prost

  • Generates simple, idiomatic, and readable Rust types by taking advantage of Rust derive attributes.
  • Retains comments from .proto files in generated Rust code.
  • Allows existing Rust types (not generated from a .proto) to be serialized and deserialized by adding attributes.
  • Uses the bytes::{Buf, BufMut} abstractions for serialization instead of std::io::{Read, Write}.
  • Respects the Protobuf package specifier when organizing generated code into Rust modules.
  • Preserves unknown enum values during deserialization.
  • Does not include support for runtime reflection or message descriptors.

Using prost in a Cargo Project

First, add prost and its public dependencies to your Cargo.toml:

[dependencies]
prost = "0.10"
# Only necessary if using Protobuf well-known types:
prost-types = "0.10"

The recommended way to add .proto compilation to a Cargo project is to use the prost-build library. See the prost-build documentation for more details and examples.

See the snazzy repository for a simple start-to-finish example.

Generated Code

prost generates Rust code from source .proto files using the proto2 or proto3 syntax. prost's goal is to make the generated code as simple as possible.

protoc

It's recommended to install protoc locally in your path to improve build times. Prost uses protoc to parse protobuf files and will attempt to compile protobuf from source requiring a C++ toolchain. For more info checkout the prost-build docs.

Packages

Prost can now generate code for .proto files that don't have a package spec. prost will translate the Protobuf package into a Rust module. For example, given the package specifier:

package foo.bar;

All Rust types generated from the file will be in the foo::bar module.

Messages

Given a simple message declaration:

// Sample message.
message Foo {
}

prost will generate the following Rust struct:

/// Sample message.
#[derive(Clone, Debug, PartialEq, Message)]
pub struct Foo {
}

Fields

Fields in Protobuf messages are translated into Rust as public struct fields of the corresponding type.

Scalar Values

Scalar value types are converted as follows:

Protobuf Type Rust Type
double f64
float f32
int32 i32
int64 i64
uint32 u32
uint64 u64
sint32 i32
sint64 i64
fixed32 u32
fixed64 u64
sfixed32 i32
sfixed64 i64
bool bool
string String
bytes Vec

Enumerations

All .proto enumeration types convert to the Rust i32 type. Additionally, each enumeration type gets a corresponding Rust enum type. For example, this proto enum:

enum PhoneType {
  MOBILE = 0;
  HOME = 1;
  WORK = 2;
}

gets this corresponding Rust enum [1]:

pub enum PhoneType {
    Mobile = 0,
    Home = 1,
    Work = 2,
}

You can convert a PhoneType value to an i32 by doing:

PhoneType::Mobile as i32

The #[derive(::prost::Enumeration)] annotation added to the generated PhoneType adds these associated functions to the type:

impl PhoneType {
    pub fn is_valid(value: i32) -> bool { ... }
    pub fn from_i32(value: i32) -> Option { ... }
}

so you can convert an i32 to its corresponding PhoneType value by doing, for example:

let phone_type = 2i32;

match PhoneType::from_i32(phone_type) {
    Some(PhoneType::Mobile) => ...,
    Some(PhoneType::Home) => ...,
    Some(PhoneType::Work) => ...,
    None => ...,
}

Additionally, wherever a proto enum is used as a field in a Message, the message will have 'accessor' methods to get/set the value of the field as the Rust enum type. For instance, this proto PhoneNumber message that has a field named type of type PhoneType:

message PhoneNumber {
  string number = 1;
  PhoneType type = 2;
}

will become the following Rust type [1] with methods type and set_type:

pub struct PhoneNumber {
    pub number: String,
    pub r#type: i32, // the `r#` is needed because `type` is a Rust keyword
}

impl PhoneNumber {
    pub fn r#type(&self) -> PhoneType { ... }
    pub fn set_type(&mut self, value: PhoneType) { ... }
}

Note that the getter methods will return the Rust enum's default value if the field has an invalid i32 value.

The enum type isn't used directly as a field, because the Protobuf spec mandates that enumerations values are 'open', and decoding unrecognized enumeration values must be possible.

[1] Annotations have been elided for clarity. See below for a full example.

Field Modifiers

Protobuf scalar value and enumeration message fields can have a modifier depending on the Protobuf version. Modifiers change the corresponding type of the Rust field:

.proto Version Modifier Rust Type
proto2 optional Option
proto2 required T
proto3 default T for scalar types, Option otherwise
proto3 optional Option
proto2/proto3 repeated Vec

Note that in proto3 the default representation for all user-defined message types is Option, and for scalar types just T (during decoding, a missing value is populated by T::default()). If you need a witness of the presence of a scalar type T, use the optional modifier to enforce an Option representation in the generated Rust struct.

Map Fields

Map fields are converted to a Rust HashMap with key and value type converted from the Protobuf key and value types.

Message Fields

Message fields are converted to the corresponding struct type. The table of field modifiers above applies to message fields, except that proto3 message fields without a modifier (the default) will be wrapped in an Option. Typically message fields are unboxed. prost will automatically box a message field if the field type and the parent type are recursively nested in order to avoid an infinite sized struct.

Oneof Fields

Oneof fields convert to a Rust enum. Protobuf oneofs types are not named, so prost uses the name of the oneof field for the resulting Rust enum, and defines the enum in a module under the struct. For example, a proto3 message such as:

message Foo {
  oneof widget {
    int32 quux = 1;
    string bar = 2;
  }
}

generates the following Rust[1]:

pub struct Foo {
    pub widget: Option<foo::Widget>,
}
pub mod foo {
    pub enum Widget {
        Quux(i32),
        Bar(String),
    }
}

oneof fields are always wrapped in an Option.

[1] Annotations have been elided for clarity. See below for a full example.

Services

prost-build allows a custom code-generator to be used for processing service definitions. This can be used to output Rust traits according to an application's specific needs.

Generated Code Example

Example .proto file:

syntax = "proto3";
package tutorial;

message Person {
  string name = 1;
  int32 id = 2;  // Unique ID number for this person.
  string email = 3;

  enum PhoneType {
    MOBILE = 0;
    HOME = 1;
    WORK = 2;
  }

  message PhoneNumber {
    string number = 1;
    PhoneType type = 2;
  }

  repeated PhoneNumber phones = 4;
}

// Our address book file is just one of these.
message AddressBook {
  repeated Person people = 1;
}

and the generated Rust code (tutorial.rs):

, } /// Nested message and enum types in `Person`. pub mod person { #[derive(Clone, PartialEq, ::prost::Message)] pub struct PhoneNumber { #[prost(string, tag="1")] pub number: ::prost::alloc::string::String, #[prost(enumeration="PhoneType", tag="2")] pub r#type: i32, } #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord, ::prost::Enumeration)] #[repr(i32)] pub enum PhoneType { Mobile = 0, Home = 1, Work = 2, } } /// Our address book file is just one of these. #[derive(Clone, PartialEq, ::prost::Message)] pub struct AddressBook { #[prost(message, repeated, tag="1")] pub people: ::prost::alloc::vec::Vec, }">
#[derive(Clone, PartialEq, ::prost::Message)]
pub struct Person {
    #[prost(string, tag="1")]
    pub name: ::prost::alloc::string::String,
    /// Unique ID number for this person.
    #[prost(int32, tag="2")]
    pub id: i32,
    #[prost(string, tag="3")]
    pub email: ::prost::alloc::string::String,
    #[prost(message, repeated, tag="4")]
    pub phones: ::prost::alloc::vec::Vec<person::PhoneNumber>,
}
/// Nested message and enum types in `Person`.
pub mod person {
    #[derive(Clone, PartialEq, ::prost::Message)]
    pub struct PhoneNumber {
        #[prost(string, tag="1")]
        pub number: ::prost::alloc::string::String,
        #[prost(enumeration="PhoneType", tag="2")]
        pub r#type: i32,
    }
    #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord, ::prost::Enumeration)]
    #[repr(i32)]
    pub enum PhoneType {
        Mobile = 0,
        Home = 1,
        Work = 2,
    }
}
/// Our address book file is just one of these.
#[derive(Clone, PartialEq, ::prost::Message)]
pub struct AddressBook {
    #[prost(message, repeated, tag="1")]
    pub people: ::prost::alloc::vec::Vec<Person>,
}

Accessing the protoc FileDescriptorSet

The prost_build::Config::file_descriptor_set_path option can be used to emit a file descriptor set during the build & code generation step. When used in conjunction with the std::include_bytes macro and the prost_types::FileDescriptorSet type, applications and libraries using Prost can implement introspection capabilities requiring details from the original .proto files.

Using prost in a no_std Crate

prost is compatible with no_std crates. To enable no_std support, disable the std features in prost and prost-types:

[dependencies]
prost = { version = "0.6", default-features = false, features = ["prost-derive"] }
# Only necessary if using Protobuf well-known types:
prost-types = { version = "0.6", default-features = false }

Additionally, configure prost-build to output BTreeMaps instead of HashMaps for all Protobuf map fields in your build.rs:

let mut config = prost_build::Config::new();
config.btree_map(&["."]);

When using edition 2015, it may be necessary to add an extern crate core; directive to the crate which includes prost-generated code.

Serializing Existing Types

prost uses a custom derive macro to handle encoding and decoding types, which means that if your existing Rust type is compatible with Protobuf types, you can serialize and deserialize it by adding the appropriate derive and field annotations.

Currently the best documentation on adding annotations is to look at the generated code examples above.

Tag Inference for Existing Types

Prost automatically infers tags for the struct.

Fields are tagged sequentially in the order they are specified, starting with 1.

You may skip tags which have been reserved, or where there are gaps between sequentially occurring tag values by specifying the tag number to skip to with the tag attribute on the first field after the gap. The following fields will be tagged sequentially starting from the next number.

use prost;
use prost::{Enumeration, Message};

#[derive(Clone, PartialEq, Message)]
struct Person {
    #[prost(string, tag = "1")]
    pub id: String, // tag=1
    // NOTE: Old "name" field has been removed
    // pub name: String, // tag=2 (Removed)
    #[prost(string, tag = "6")]
    pub given_name: String, // tag=6
    #[prost(string)]
    pub family_name: String, // tag=7
    #[prost(string)]
    pub formatted_name: String, // tag=8
    #[prost(uint32, tag = "3")]
    pub age: u32, // tag=3
    #[prost(uint32)]
    pub height: u32, // tag=4
    #[prost(enumeration = "Gender")]
    pub gender: i32, // tag=5
    // NOTE: Skip to less commonly occurring fields
    #[prost(string, tag = "16")]
    pub name_prefix: String, // tag=16  (eg. mr/mrs/ms)
    #[prost(string)]
    pub name_suffix: String, // tag=17  (eg. jr/esq)
    #[prost(string)]
    pub maiden_name: String, // tag=18
}

#[derive(Clone, Copy, Debug, PartialEq, Eq, Enumeration)]
pub enum Gender {
    Unknown = 0,
    Female = 1,
    Male = 2,
}

FAQ

  1. Could prost be implemented as a serializer for Serde?

Probably not, however I would like to hear from a Serde expert on the matter. There are two complications with trying to serialize Protobuf messages with Serde:

  • Protobuf fields require a numbered tag, and currently there appears to be no mechanism suitable for this in serde.
  • The mapping of Protobuf type to Rust type is not 1-to-1. As a result, trait-based approaches to dispatching don't work very well. Example: six different Protobuf field types correspond to a Rust Vec: repeated int32, repeated sint32, repeated sfixed32, and their packed counterparts.

But it is possible to place serde derive tags onto the generated types, so the same structure can support both prost and Serde.

  1. I get errors when trying to run cargo test on MacOS

If the errors are about missing autoreconf or similar, you can probably fix them by running

brew install automake
brew install libtool

License

prost is distributed under the terms of the Apache License (Version 2.0).

See LICENSE for details.

Copyright 2022 Dan Burkert & Tokio Contributors

Comments
  • feat: Add protoc-gen-prost

    feat: Add protoc-gen-prost

    As the title says, this adds a new subcrate, protoc-gen-prost, a protoc plugin for code generation.

    Features

    Is supports all the options the regular prost-build does, with the addition of a full Cargo crate generation, with a Cargo.toml file and feature flags for each protobuf package, with proper feature dependency resolution.

    Motivation

    While the recommended way to use prost in Rust projects is with prost-build and running protoc through Cargo and build.rs, a protoc plugin allows to get a standard workflow in the protobuf ecosystem. Also, precompiling proto files to Rust code as files has some advantages:

    • easier to share compiled code across multiple projects, since they don't all need to setup the prost build
    • rust code can be stored in an easy to browse fashion
    • integrates well with standard protobuf/grpc/... tooling

    Notes

    The generation API is public, and not restricted to prost, as it is useful for other codegen tools for downstream RPC crates based off prost, like tonic (for which I have a similar PR incoming).

    Examples

    All examples assume protoc-gen-prost was installed and can be found by protoc, and the output directory was created:

     ~/prost$ cargo install protoc-gen-prost --path protoc-gen-prost --release
     ~/prost$ mkdir hello-rs
    

    Basic usage

    protoc --prost_out=hello-rs -I prost-build/third-party/protobuf/include -I prost-build/src hello.proto
    

    Results in:

    hello-rs/
    └── helloworld.rs
    

    Module generation

    This is suitable to generate a full module, typically directly in your crate source:

    protoc --prost_out=hello-rs --prost_opt=include_file=mod.rs \
        -I prost-build/third-party/protobuf/include -I prost-build/src hello.proto
    

    Results in

    hello-rs/
    ├── helloworld.rs
    └── mod.rs
    

    Generated mod.rs:

    pub mod helloworld {
        include!("helloworld.rs");
    }
    

    Crate generation

    This is suitable for larger proto codebases, as feature flags allow to turn individual package on and off during Rust compilation, greatly speeding up build times. It also allows for easier sharing of the proto across multiple projects by making it "just another dependency".

    It uses a Cargo.toml template:

    [package]
    name = "hello-rs"
    version = "1.0.0"
    authors = []
    edition = "2018"
    
    [dependencies]
    prost = "0.8"
    prost-types = "0.8"
    tonic = "0.5"
    
    [features]
    {{ features }}
    
    protoc --prost_out=hello-rs --prost_opt=gen_crate=hello-rs/Cargo.toml.tpl \
        -I prost-build/third-party/protobuf/include -I prost-build/src hello.proto
    

    Results in the full crate:

    hello-rs/
    ├── Cargo.toml
    ├── Cargo.toml.tpl
    ├── gen
    │   └── helloworld.rs
    └── src
        └── lib.rs
    

    With the following lib.rs:

    pub mod helloworld {
        #[cfg(feature = "helloworld")]
        include!("../gen/helloworld.rs");
    }
    

    This can be combined with the include_file option to name lib.rs something else, if some custom, non-generated code, needs to be added to the crate.

    opened by Tuetuopay 48
  • Make prost a `no_std` compatible library, and prost-build able to gen…

    Make prost a `no_std` compatible library, and prost-build able to gen…

    …erate no_std code

    The alternative is to get collections types from core and alloc. In the no_std mode in prost_build, we force it to always use BTreeMap since HashMap was not stabilized in alloc::collections library.

    The functionality is identical and the only incompatibilities in the interface are that we cannot use std::error::Error or std::io::Error in no_std mode because these types have not been moved to alloc and there is no alternative.

    opened by cbeck88 44
  • Don't invoke the bundled protoc binary on non-gnu Linux hosts

    Don't invoke the bundled protoc binary on non-gnu Linux hosts

    Revisiting the problem reported in #182. Unfortunately, it percolates to builds that use tower-grpc-build and troubleshooting clues are currently less than ideal.

    Is it possible to avoid running the bundled protoc on non-glibc (e.g. musl) Linux host environment and report that a binary needs to be supplied? Notably this is different from the build target.

    opened by mzabaluev 24
  • Add support for no package declaration

    Add support for no package declaration

    I added a test for this to make sure it works as intended. I've tested it on some internal proto files and it also works and they're pretty substantial.

    One thing I wasn't sure how to handle, but can be easily addressed is do we still need to error by default if there is no package defined? Could we agree on a default filename like __.default.rs or something else that is a non-valid rust module name?

    This also makes error handling worse because it won't give the filename of the proto that doesn't have a package spec. If we are still going to error if there is no package defined, then I can get the original error back into place.

    I'm happy to add any more tests that you think we need to make sure things are covered.

    opened by tylerhawkes 22
  • Suggestion for enumerations

    Suggestion for enumerations

    Hello

    I'm still a bit unhappy about the fact enumerations are i32 values, for two reasons:

    • Handling them is cumbersome. I can't just access the field, I have to convert it first…
    • They are interchangeable. Let's say I have enums enum Direction { Unspecified = 0, In = 1, Out = 2 } and enum Proto { Unspecified = 0, Udp = 1, Tcp = 2 }. I can do message.proto = Direction::Out as i32 and it will work. I can do Direction::from_i32(message.proto) and it'll work. And I like Rust's strict type system keeping an eye on me.

    So I was wondering if there could be more convenient and safer alternative. What about:

    • Create the enum as it is now, eg enum Direction { Unspecified = 0, In = 1, Out = 2 }
    • Also create a companion struct DirectionRaw(pub i32)

    This way, we can have conversion methods on both of them to each other, but not with i32 directly (type safety). The fact the conversion methods would be on the DirectionRaw would make it more discoverable.

    Furthermore, I think the conversion method would be something like fn enum(&self) -> Option<Direction> ‒ this way it is possible to both know if the input was valid/known and call unwrap_or_default() on it to get the appropriate default.

    Does that make sense? Or does it have some downsides I don't see?

    I would be willing to implement it, but I wanted to check first if it makes sense.

    opened by vorner 22
  • Experiment with using `bytes::Bytes` to back bytes and string fields

    Experiment with using `bytes::Bytes` to back bytes and string fields

    This PR uses bytes::Bytes as the type for bytes and a new type BytesString as the type for string (rather than Vec<u8> and String). BytesStringis a wrapper around Bytes in the same way that String is a wrapper around Vec<u8>. Obviously this is a severe breaking change.

    This idea has been discussed in #31 and #35.

    This PR improves performance by 25% (!) on Mac OS, 9% on Ubuntu, and 2% on CentOS. This was tested using a fairly realistic benchmark dervied from TiKV, however, the bytes and string values were all quite small. I expect the performance improvements would be smaller for larger values. I also tested a multi-threaded version of the benchmark on Ubuntu, there the average improvement was only 1.5%, however, the worst-performing thread improved by 14%, so the overall time to completion was significantly improved.

    If we should go forward with this PR, then I think there are few open questions, but I thought I'd post early to get feedback about whether it was possible to land at all, given the back-compat situation.

    If we do land, I think we can probably get rid of the BytesMutString type and just use BytesString - I didn't see any performance benefit or usability benefit in using the BytesMut version.

    One option might be to land the change to bytes but not to string. I feel like that is a less invasive change but will get a decent benefit for some users.

    @danburkert what do you think?

    opened by nrc 20
  • Include Message Descriptor information

    Include Message Descriptor information

    A new trait MessageDescriptor which includes the following info about a message:

    • package_name which is taken from the package field in the proto file;
    • message_name which is the message name itself;
    • type_url which is "type.googleapis.com/" + package_name + "." + message_name;

    The prefix "type.googleapis.com" is the default prefix for all protobuf types (see here).

    Adding the MessageDescriptor trait will help with the packing and unpacking of messages in the Any type, but may be helpful in other situations as well, and can easily be extended to include more proto information in future (e.g. source file etc).

    This pull request can address #277 completely as it will allow an external crate to provide all the functionalities without polluting the prost crate with additional dependencies. For an external crate that provides all the functionality described in #277, see prost-wkt.

    opened by fdeantoni 17
  • Conversion between Timestamp and chrono::DateTime

    Conversion between Timestamp and chrono::DateTime

    These changes enable idiomatic conversions between chrono::DateTime and the protobuf Timestamp.

    Such conversions were already provided with objects from std::time, but not with ones from the chrono crate. Since it adds a dependency, this code is conditionally used, only if feature chrono-conversions is set.

    opened by daladim 16
  • Prost should not require user to include a `bytes` dependency.

    Prost should not require user to include a `bytes` dependency.

    By doing extern crate bytes in the generated derive, Prost requires the user to add a bytes dependency.

    Instead, the prost crate should re-export bytes. I have done this in the past by having a hidden codegen module w/ the re-exports. Example.

    opened by carllerche 16
  • Provide enum -> string methods

    Provide enum -> string methods

    Hi,

    Google ProtoBuf implementations provide a way to get the string value out of PB enums. For example, the C++ generated code provides:

    const std::string& SomeEnum_Name(T enum_t_value) { ... }
    

    This is especially useful for things like metric label values and such. Would it be possible to provide this functionality with Prost?

    Would you accept a PR for this?

    opened by vriesk 14
  • Implement a recursion limit

    Implement a recursion limit

    This PR implements a recursion limit, which can be disabled by not setting a Cargo feature (which is on by default). This change is technically not backwards compatible, but in practice very few programs should hit the recursion limit.

    Although adding a context argument complicates the API somewhat, most clients would not implement or use the methods with the added argument and would instead only use decode, which is not changed.

    Fixes #176

    r? @danburkert

    opened by nrc 14
  • Generate .proto from existing structs

    Generate .proto from existing structs

    I was wondering if it is possible to generate proto messages from existing rust structs.

    I have the following struct:

    #[derive(Copy, Clone, Serialize, Message)]
    pub struct Node {
      pub node_id: i32,
      pub left_id: Option<i32>,
      pub right_id: Option<i32>,
      pub strength: f32,
      pub level: i32,
      pub right_parents: i32,
      pub left_parents: i32,
    }
    

    This struct is a core component of my application, and I want it to be the source of truth. Ideally, I would like to generate a proto based on it.

    At the end of the README it states to do something like this:

    #[derive(Copy, Clone, Serialize, Message)]
    pub struct Node {
      #[prost(int32)]
      pub node_id: i32,
      #[prost(int32, optional)]
      pub left_id: Option<i32>,
      #[prost(int32, optional)]
      pub right_id: Option<i32>,
      #[prost(float)]
      pub strength: f32,
      #[prost(int32)]
      pub level: i32,
      #[prost(int32)]
      pub right_parents: i32,
      #[prost(int32)]
      pub left_parents: i32,
    }
    

    But even when I do this I am still required to create a .proto file with:

    message Node {
      int32 node_id = 1;
      optional int32 left_id = 3;
      optional int32 right_id = 4;
      float strength = 5;
      int32 level = 6;
      int32 right_parents = 7;
      int32 left_parents = 8;
    }
    

    I really just want a way to use my struct as the source of truth and generate a proto from it without having to repeat the structure in a .proto file manually. The Node struct could change, and I don't want every developer to be responsible for making sure the structure changes in both the core of the application and the proto. Is this possible or is it not how Prost is meant to be used.

    Thanks a lot!

    opened by caamartin35 0
  • Remove protoc dependency?

    Remove protoc dependency?

    I would be interested in trying my hand at removing the need for protoc entirely. My understanding (from reading the docs not the code) is that protoc is being used to parse the proto files. Is there interest in having at least the option to use prost-build with a custom parser instead of protoc? I would be willing to work on this, but I want to make sure there's a chance it might actually get merged before digging in too far.

    opened by jhutchins 0
  • PartialOrd and Ord for Timestamp

    PartialOrd and Ord for Timestamp

    It seems like Timestamp could (and perhaps should) implement PartialOrd and Ord. This should be pretty straightforward given the constraints -- compare the seconds, and if equal compare the nanos. Any reason these shouldn't be added?

    (Originally filed https://github.com/fdeantoni/prost-wkt/issues/15)

    opened by bjchambers 0
  • Draft: Unknown fields

    Draft: Unknown fields

    A proof-of-concept implementation allowing users to opt in for preserving unknown fields. There is more cleanup to do on this implementation, but if the approach is promising, it would address the issue for adding support for unknown fields.

    This PR adds a prost-examples crate which illustrates the generate_unknown_fields build option.

    opened by jasonahills 1
  • Optimize varint encoding

    Optimize varint encoding

    This optimizes varint encoding by turning the loop into a for loop which causes the compiler to see that it can unroll the loop.

    Bench:

    varint/small/encode     time:   [61.577 ns 61.786 ns 62.016 ns]
                            change: [-18.793% -17.983% -17.024%] (p = 0.00 < 0.05)
                            Performance has improved.
    Found 5 outliers among 100 measurements (5.00%)
      1 (1.00%) high mild
      4 (4.00%) high severe
    
    varint/medium/encode    time:   [312.28 ns 313.01 ns 313.78 ns]
                            change: [-13.850% -13.002% -12.209%] (p = 0.00 < 0.05)
                            Performance has improved.
    Found 5 outliers among 100 measurements (5.00%)
      4 (4.00%) high mild
      1 (1.00%) high severe
    
    varint/large/encode     time:   [533.79 ns 535.11 ns 536.43 ns]
                            change: [-26.542% -25.921% -25.275%] (p = 0.00 < 0.05)
                            Performance has improved.
    Found 5 outliers among 100 measurements (5.00%)
      1 (1.00%) low mild
      1 (1.00%) high mild
      3 (3.00%) high severe
    
    varint/mixed/encode     time:   [308.34 ns 309.11 ns 309.87 ns]
                            change: [-23.418% -22.569% -21.718%] (p = 0.00 < 0.05)
                            Performance has improved.
    Found 6 outliers among 100 measurements (6.00%)
      2 (2.00%) high mild
      4 (4.00%) high severe
    
    opened by MelonShooter 0
  • Refactor to remove unsafe code usage when decoding varints

    Refactor to remove unsafe code usage when decoding varints

    This removes the unsafe code usage in the manually unrolled loop in decode_varint_slice by writing loops in a way where they can be unrolled by the compiler in an efficient manner. I had to change some of the tests slightly as well because I changed the function signature of decode_varint_slow when I moved its call into decode_varint_slice. Apparently, moving that function call into decode_varint_slice produced a huge speedup (see results below) in decoding and I'm not sure if it's an issue with the benchmark or if moving it allowed the compiler to perform an optimization it didn't see before.

    Benchmark results:

    varint/small/encode     time:   [76.518 ns 76.792 ns 77.090 ns]
                            change: [+1.1944% +2.1410% +3.3721%] (p = 0.00 < 0.05)
                            Performance has regressed.
    Found 5 outliers among 100 measurements (5.00%)
      1 (1.00%) low mild
      3 (3.00%) high mild
      1 (1.00%) high severe
    
    varint/small/decode     time:   [58.051 ns 58.233 ns 58.415 ns]
                            change: [-58.541% -58.225% -57.915%] (p = 0.00 < 0.05)
                            Performance has improved.
    Found 4 outliers among 100 measurements (4.00%)
      1 (1.00%) low mild
      1 (1.00%) high mild
      2 (2.00%) high severe
    
    varint/small/encoded_len
                            time:   [74.072 ns 74.267 ns 74.466 ns]
                            change: [+2.1763% +3.0063% +3.7575%] (p = 0.00 < 0.05)
                            Performance has regressed.
    Found 5 outliers among 100 measurements (5.00%)
      2 (2.00%) high mild
      3 (3.00%) high severe
    
    varint/medium/encode    time:   [351.15 ns 352.36 ns 353.53 ns]
                            change: [-2.8164% -1.9881% -1.1698%] (p = 0.00 < 0.05)
                            Performance has improved.
    Found 5 outliers among 100 measurements (5.00%)
      2 (2.00%) high mild
      3 (3.00%) high severe
    
    varint/medium/decode    time:   [145.97 ns 146.68 ns 147.59 ns]
                            change: [-20.539% -19.763% -18.999%] (p = 0.00 < 0.05)
                            Performance has improved.
    Found 7 outliers among 100 measurements (7.00%)
      1 (1.00%) high mild
      6 (6.00%) high severe
    
    varint/medium/encoded_len
                            time:   [73.548 ns 73.747 ns 73.979 ns]
                            change: [+0.8837% +1.8531% +2.7898%] (p = 0.00 < 0.05)
                            Change within noise threshold.
    Found 7 outliers among 100 measurements (7.00%)
      5 (5.00%) high mild
      2 (2.00%) high severe
    
    varint/large/encode     time:   [697.27 ns 699.23 ns 701.30 ns]
                            change: [-2.5895% -1.7824% -0.9898%] (p = 0.00 < 0.05)
                            Change within noise threshold.
    Found 6 outliers among 100 measurements (6.00%)
      3 (3.00%) high mild
      3 (3.00%) high severe
    
    varint/large/decode     time:   [238.21 ns 239.97 ns 241.99 ns]
                            change: [-19.652% -18.703% -17.849%] (p = 0.00 < 0.05)
                            Performance has improved.
    Found 7 outliers among 100 measurements (7.00%)
      1 (1.00%) high mild
      6 (6.00%) high severe
    
    varint/large/encoded_len
                            time:   [73.803 ns 73.983 ns 74.162 ns]
                            change: [+1.8433% +2.8766% +3.8957%] (p = 0.00 < 0.05)
                            Performance has regressed.
    Found 4 outliers among 100 measurements (4.00%)
      1 (1.00%) high mild
      3 (3.00%) high severe
    
    varint/mixed/encode     time:   [386.77 ns 388.05 ns 389.35 ns]
                            change: [-3.6711% -2.6998% -1.8145%] (p = 0.00 < 0.05)
                            Performance has improved.
    Found 6 outliers among 100 measurements (6.00%)
      3 (3.00%) high mild
      3 (3.00%) high severe
    
    varint/mixed/decode     time:   [150.70 ns 151.20 ns 151.70 ns]
                            change: [-26.334% -25.751% -25.144%] (p = 0.00 < 0.05)
                            Performance has improved.
    Found 7 outliers among 100 measurements (7.00%)
      5 (5.00%) high mild
      2 (2.00%) high severe
    
    varint/mixed/encoded_len
                            time:   [73.674 ns 73.881 ns 74.103 ns]
                            change: [+0.8260% +2.2875% +3.3646%] (p = 0.00 < 0.05)
                            Change within noise threshold.
    Found 7 outliers among 100 measurements (7.00%)
      1 (1.00%) low mild
      4 (4.00%) high mild
      2 (2.00%) high severe
    
    opened by MelonShooter 0
Releases(v0.11.3)
  • v0.11.3(Nov 29, 2022)

    PROST! is a Protocol Buffers implementation for the Rust Language. prost generates simple, idiomatic Rust code from proto2 and proto3 files.

    This patch updates brings a few new features and fixes:

    • build: Use log to print error logs.
    • build: Remove accidentally included prints.
    • build: Improve debian error message hint.
    Source code(tar.gz)
    Source code(zip)
  • v0.11.2(Nov 4, 2022)

    PROST! is a Protocol Buffers implementation for the Rust Language. prost generates simple, idiomatic Rust code from proto2 and proto3 files.

    This patch updates brings a few new features and fixes:

    • build: Improve protoc sourcing instructions in error messages.
    • build: Add Clone to Service types.
    • build: Sort modules to produce deterministic include files.
    • core: Tuple struct support.
    • build: Allow generated code derive path's to be changed.
    • build: Allow changing prost crate path.
    • build: Add code formatting support.
    Source code(tar.gz)
    Source code(zip)
  • prost-types-0.11.1(Aug 2, 2022)

  • prost-build-0.11.1(Aug 2, 2022)

  • v0.11.0(Jul 29, 2022)

    PROST! is a Protocol Buffers implementation for the Rust Language. prost generates simple, idiomatic Rust code from proto2 and proto3 files.

    Relase 0.11 brings a few breaking changes and changes to how prost sources protoc.

    • prost-build now requires protoc to be available in the path or set via the PROTOC env var.
    • prost-types now contains new Timestamp/Duration FromStr implementations.
    • MSRV bump to 1.56 and all crates have been moved to edition 2021

    Notible changes that are not breaking:

    • prost-build now has a cleanup-markdown feature for cleaining up code blocks from protobuf files so that they work under rustdoc tests.
    • prost-build now generates as_str_name for message types.
    Source code(tar.gz)
    Source code(zip)
  • v0.10.4(May 24, 2022)

  • v0.10.3(May 5, 2022)

    PROST! is a Protocol Buffers implementation for the Rust Language. prost generates simple, idiomatic Rust code from proto2 and proto3 files.

    This release reverts #605 and contains some typo fixes.

    (this release is actually the v0.10.2 release but prost-build had to be yanked because of a local publish issue)

    Source code(tar.gz)
    Source code(zip)
  • v0.10.2(May 5, 2022)

    PROST! is a Protocol Buffers implementation for the Rust Language. prost generates simple, idiomatic Rust code from proto2 and proto3 files.

    This release reverts #605 and contains some typo fixes.

    (this release was yanked due to missing third-party source, v0.10.3 is now the active release for this)

    Source code(tar.gz)
    Source code(zip)
  • v0.10.1(Apr 13, 2022)

  • v0.10.0(Mar 31, 2022)

    PROST! is a Protocol Buffers implementation for the Rust Language. prost generates simple, idiomatic Rust code from proto2 and proto3 files.

    Release 0.10 brings a few new ....

    • protoc is no longer bundled but is now compiled from bundled source
    • Minor performance improvements
    • Methods exposed to allow third party protobuf generation libraries
    Source code(tar.gz)
    Source code(zip)
  • v0.9.0(Oct 13, 2021)

    PROST! is a Protocol Buffers implementation for the Rust Language. prost generates simple, idiomatic Rust code from proto2 and proto3 files.

    Release 0.9.0 brings in many helpful changes and fixes, here are a few highlights:

    • Apple silicon support
    • Improve encode/decode varint performance
    • Support no package declaration
    • Support single include files
    • Fix multiple attribute support
    Source code(tar.gz)
    Source code(zip)
  • v0.8.0(Jul 8, 2021)

    PROST! is a Protocol Buffers implementation for the Rust Language. prost generates simple, idiomatic Rust code from proto2 and proto3 files.

    NOTE: This version contains a security fix for prost-types and is recommend that you upgrade to it from <0.7.

    prost 0.8.0 includes breaking changes:

    • Timestamp's From implementation for converting into SystemTime has been converted to a fallible TryFrom implementation.
    • prost-build's compile_protos now takes impl AsRef<Path> to allow each parameter to use its own generic type.
    • Bundled protoc version bumped to 3.15.8

    As well as many new (non-breaking) changes:

    • @pluth enabled zero-copy support for Bytes based fields.
    • @sfackler for fixing message optionals and oneofs in prost-build.
    • @rubdos for adding the ability to encode prost messages directly to a Vec<u8>.

    and numerous smaller fixes. Many thanks to the generous contributors who have helped out since 0.7:

    • @dfreese
    • @carols10cents
    • @pluth
    • @jfornoff
    • @sphw
    • @bpowers
    • @sfackler
    • @koushiro
    • @dbrgn
    • @argv-minus-one
    • @nagisa
    • @evanj
    • @aquarhead
    Source code(tar.gz)
    Source code(zip)
  • v0.7.0(Dec 27, 2020)

    PROST! is a Protocol Buffers implementation for the Rust Language. prost generates simple, idiomatic Rust code from proto2 and proto3 files.

    prost 0.7.0 includes breaking changes:

    • The minimum-supported Rust version (MSRV) is now 1.46.0.
    • The bytes public dependency version is now 1.0.
    • The bundled protoc version is now 3.14.0.

    As well as many new (non-breaking) features and improvements:

    • @hockeybuggy added support for deprecated field annotations.
    • @garbageslam and @dflemstr added no_std support.
    • @joseph-wakeling-frequenz fixed a bug in Timestamp <-> SystemTime conversions.
    • @rolftimmermans added support for generating Rust bytes::Bytes fields from protobuf bytes fields. When deserializing from a Bytes instance, this enables zero-copy deserialization for bytes fields!
    • @olix0r bumped the bytes dependency to 0.6, which included most of the heavy lifting for the subsequent move to bytes 1.0.
    • @danburkert added support for the experimental proto3 optional field presence feature.

    and numerous smaller fixes. Many thanks to the generous contributors who have helped out since 0.6.1:

    • [email protected] Dunlap
    • @Max-Meldrum
    • @Veetaha
    • @dan-fritchman
    • @danburkert
    • @dflemstr
    • @dfreese
    • @gabrielrussoc
    • @garbageslam
    • @hockeybuggy
    • @jen20
    • @joseph-wakeling-frequenz
    • @olix0r
    • @oll3
    • @repi
    • @rolftimmermans
    • @vorot93
    • @wchargin
    Source code(tar.gz)
    Source code(zip)
  • v0.6.1(Jan 16, 2020)

    PROST! is a Protocol Buffers implementation for the Rust Language. prost generates simple, idiomatic Rust code from proto2 and proto3 files.

    prost 0.6.1 fixes a bug in which decoding untrusted input could overflow the stack. The bug was reported by @dbrgn in #267 (with an attached repro!), and fixed by @danburkert. The fix extended the same recursion limits introduced in #186 by @nrc (released in 0.6.0) to the logic which skips unknown fields during decoding, which became recursive when support was added for decoding groups (also released in 0.6.0). The 0.6.0 release of the prost crates has been yanked from crates.io.

    Additionally, @koushiro updated the private dependencies of all prost crates to the latest versions.

    Many thanks to the generous contributors who have helped out since 0.6.0:

    • Dan Burkert
    • Danilo Bargen
    • Qinxuan Chen
    Source code(tar.gz)
    Source code(zip)
  • v0.6.0(Jan 13, 2020)

    PROST! is a Protocol Buffers implementation for the Rust Language. prost generates simple, idiomatic Rust code from proto2 and proto3 files.

    prost 0.6.0 includes breaking changes:

    • The minimum-supported Rust version (MSRV) is now 1.39.0.
    • The bytes public dependency version is now 0.5.
    • The prost-types Timestamp and Duration types now impl TryFrom for their std::time equivalents. The old inherent try_from methods have been removed.
    • Deeply nested messages now fail decoding with an error, instead of potentially resulting in a stack overflow. The limit is set to a nesting depth of 100, and the limit can be disabled using a new no-recursion-limit Cargo feature.

    As well as many new (non-breaking) features and improvements:

    • @hanya added support for protobuf Group types.
    • @danburkert improved the benchmark suite, including adding support for the full set of upstream message encoding and decoding benchmarks.
    • @nrc implemented a series of micro-optimizations which result in faster message encoding and decoding.
    • @dunmatt improved the method docs on generated types.
    • @lukaslueg removed the dependency on byteorder.
    • @parasyte added the ServiceGenerator::finalize_package, which is useful for advanced service generators.
    • @joelgallant improved the error message that occurs when attempting to compile .proto files without a package specifier.
    • @LucioFranco removed the direct dependency of generated types on the bytes crate, which means applications which use prost code generation are no longer required to declare a bytes dependency.
    • @ErichDonGubler and @hobofan bumped the syn, quote, and proc-macro2 to stable versions.
    • @Timmmm improved prost-build so that it no longer writes .rs files when they are unchanged, which improves working with tools like cargo watch in codebases with prost code generation.
    • @Hirevo replaced usage of failure with anyhow.
    • @danburkert bumped the packaged protoc version to 3.11.2.

    Many thanks to the generous contributors who have helped out since 0.5.0:

    • Dan Burkert
    • Erich Gubler
    • FujiApple
    • Hanya
    • Jay Oster
    • Joel Gallant
    • koushiro
    • Lucio Franco
    • Luffbee
    • lukaslueg
    • [email protected] Dunlap
    • Maximilian Goisser
    • Mikhail Zabaluev
    • Nick Cameron
    • Nicolas Polomack
    • Stephan Wolski
    • Tim Hutt
    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(Mar 10, 2019)

    PROST! is a Protocol Buffers implementation for the Rust Language. prost generates simple, idiomatic Rust code from proto2 and proto3 files.

    prost 0.5.0 should be mostly source-compatible with version 0.4.0, with one exception: prost 0.4.0 escapes field and message names which are Rust keyword by appending a suffix underscore; prost 0.5.0 does not escape field or message names, instead using the new raw-identifier feature. In addition, the minimum rustc version has been bumped to 1.32.0.

    0.5.0 ships with new features and bug fixes:

    • @adeschamps fixed a bug which caused enum fields with a default value whose prefix is stripped to be incorrectly generated.
    • @danburkert added an option for overriding the code-generation output directory with a new prost_build::Config::out_dir() method.
    • @danburkert added a re-export of prost-derive types in prost. As a result, it should no longer be necessary for applications to depend directly on the prost-derive crate.
    • @ebkalderon, @trha, and @danburkert added support to the code generator for referencing prost generated types in external crates with a new prost_build::Config::extern_path() method.
    • @jeffparsons made code generation more robust by always fully-qualifying standard library types.
    • @nrc added support for generating edition-2018 compatible code.

    Many thanks to the generous contributors who have helped out since 0.4.0:

    • Anthony Deschamps
    • Dan Burkert
    • Eliza Weisman
    • Eyal Kalderon
    • Jeff Parsons
    • Nick Cameron
    • Sebastian Hahn
    • Timothy Hobbs
    • Trinh Hoang Anh
    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(Jun 13, 2018)

    PROST! is a Protocol Buffers implementation for the Rust Language. prost generates simple, idiomatic Rust code from proto2 and proto3 files.

    prost 0.4.0 contains new features, bug-fixes, and one breaking change since 0.3.0:

    • @briansmith cleaned up the project dependencies and build scripts, resulting in faster and more robust builds for all prost users.
    • @kestred added tag inference on fields in types which derive Message. Tag inference reduces the number annotations necessary when manually writing message structs.
    • @warrenfalk added logic to strip the enum name from enum variants which include the enum name as a prefix. This style is common with Protobuf (since it's heavily tied to C++ name resolution), however it's non-idiomatic in Rust. To restore the previous behavior a new Config option is provided: retain_enum_prefix. This is a breaking change.
    • @danburkert added helper methods for working with length delimiters independently of message encoding and decoding.
    • And many more bug fixes.

    Many thanks to the generous contributors who have helped out since 0.3.0:

    • Brian Smith
    • Dan Burkert
    • Gergely Nagy
    • Ivan Mironov
    • Kevin Stenerson
    • Warren Falk
    • YetAnotherMinion
    • hcpl
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Jan 29, 2018)

    PROST! is a Protocol Buffers implementation for the Rust Language. prost generates simple, idiomatic Rust code from proto2 and proto3 files.

    prost 0.3.0 contains new features, bug-fixes, and one breaking change since 0.2.0:

    • @vorner added an important new feature to prost-build which allows custom attributes to be added to generated types. This allows applications to derive traits for generated types, such as SerDe support.
    • @vorner changed the Debug implementation for enumeration fields to be more helpful.
    • The prost_build::ServiceGenerator and prost_build::Config APIs have been changed to take &mut self instead of &self in order to simplify writing stateful service generators.

    Many thanks to the wonderful contributors who have pitched-in since 0.2.0:

    • Dan Burkert
    • Fabrice Desré
    • Michal 'vorner' Vaner
    • Oliver Gould
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Sep 10, 2017)

    PROST! is a Protocol Buffers implementation for the Rust Language. prost generates simple, idiomatic Rust code from proto2 and proto3 files.

    prost 0.2.0 includes many improvements over the previous version, including:

    • The prost-codegen crate has been removed, and its functionality has been combined into prost-build.
    • A new crate, prost-types, containing Protobuf well-known types has been added. The code generator will automatically use prost-types when well-known types are encountered during .proto.rs code generation.
    • Field accessors for proto2 optional fields and enum fields have been made more ergonomic.
    • prost-build has a new builder API (prost_build::Config), which can be used to specify custom code generation options. Current options include:
      • Using BTreeMap instead of HashMap for map fields.
      • Custom code generation for protobuf service definitions.
    • New prost-specific error types, prost::EncodeError and prost::DecodeError for encoding and decoding results.
    • The prost::Message trait has been made more flexible, and more ergonomic.
    • varint encoding and decoding has been optimized, see the following commit messages for results:
      • a655f2b00c29c7
      • 107857f443e4d6
      • 8a8c5ddbb646a8
    • Default values are now preserved during code generation (bugfix).
    • New Message::clear method.
    Source code(tar.gz)
    Source code(zip)
Owner
Tokio
Rust's asynchronous runtime.
Tokio
Official Rust implementation of the Nimiq protocol

Nimiq Core implementation in Rust (core-rs) Rust implementation of the Nimiq Blockchain Core Nimiq is a frictionless payment protocol for the web. Thi

Nimiq 72 Sep 23, 2022
Rust implementation of Zcash protocol

The Parity Zcash client. Gitter Blog: Parity teams up with Zcash Foundation for Parity Zcash client Installing from source Installing the snap Running

Parity Technologies 183 Sep 8, 2022
A prototype implementation of the Host Identity Protocol v2 for bare-metal systems, written in pure-rust.

Host Identity Protocol for bare-metal systems, using Rust I've been evaluating TLS replacements in constrained environments for a while now. Embedded

null 30 Nov 17, 2022
A Rust implementation of the Message Layer Security group messaging protocol

Molasses An extremely early implementation of the Message Layer Security group messaging protocol. This repo is based on draft 4 of the MLS protocol s

Trail of Bits 108 Oct 26, 2022
Ecoball Node is the Official Rust implementation of the Ecoball protocol.

Ecoball Node is the Official Rust implementation of the Ecoball protocol. It is a fork of OpenEthereum - https://github.com/openethereum/

Ecoball Chain 2 Jun 9, 2022
A rust implementation of the ABCI protocol for tendermint core

?? DEPRECATED ?? This repo has been deprecated. Development work continues as the "abci" crate of informalsystems/tendermint-rs. Please reference that

Tendermint 117 Oct 28, 2022
Open source Rust implementation of the Witnet decentralized oracle protocol, including full node and wallet backend 👁️🦀

witnet-rust is an open source implementation of the Witnet Decentralized Oracle Network protocol written in Rust. Components witnet-rust implements ma

The Witnet Project 155 Nov 21, 2022
The Rust implementation of Conflux protocol.

Conflux-Rust Conflux-rust is a Rust-based implementation of the Conflux protocol. It is fast and reliable. Please follow the Conflux Documentation to

Conflux 560 Nov 20, 2022
In addition to encryption library, pure RUST implementation of SSH-2.0 client protocol

In addition to encryption library, pure RUST implementation of SSH-2.0 client protocol

陈年旧事。 54 Nov 19, 2022
Next-generation implementation of Ethereum protocol ("client") written in Rust, based on Erigon architecture.

?? Martinez ?? Next-generation implementation of Ethereum protocol ("client") written in Rust, based on Erigon architecture. Why run Martinez? Look at

Arthur·Thomas 23 Jul 3, 2022
Rust implementation of the Inter-Blockchain Communication (IBC) protocol.

ibc-rs Rust implementation of the Inter-Blockchain Communication (IBC) protocol. This project hosts the ibc rust crate which defines the main data str

COSMOS 18 Nov 18, 2022
Pure Rust implementation of components of the Secure Shell (SSH) protocol

RustCrypto: SSH Pure Rust implementation of components of the Secure Shell (SSH) protocol. Crates Name crates.io Docs Description ssh—encoding Decoder

Rust Crypto 24 Nov 25, 2022
Minimal implementation of the Mimblewimble protocol.

Grin Grin is an in-progress implementation of the Mimblewimble protocol. Many characteristics are still undefined but the following constitutes a firs

null 5k Nov 22, 2022
An implementation of the OPAQUE password-authenticated key exchange protocol

The OPAQUE key exchange protocol OPAQUE is an asymmetric password-authenticated key exchange protocol. It allows a client to authenticate to a server

Novi 165 Dec 1, 2022
A safe implementation of the secure remote password authentication and key-exchange protocol (SRP), SRP6a and legacy are as features available.

Secure Remote Password (SRP 6 / 6a) A safe implementation of the secure remote password authentication and key-exchange protocol (SRP version 6a). Ver

Sven Assmann 10 Nov 3, 2022
Webb Protocol implementation in Ink!

Webb Protocol Ink! ??️ Webb Protocol Ink! ⧫ ⚠️ Beta Software ⚠️ Dependencies A prerequisite for compiling smart contracts is to have Rust and Cargo in

webb 4 Aug 5, 2022
Reference client for NEAR Protocol

Reference implementation of NEAR Protocol About NEAR NEAR's purpose is to enable community-driven innovation to benefit people around the world. To ac

NEAR 1.9k Dec 3, 2022
Kerberos protocol attacker

Cerbero Kerberos protocol attacker. Tool to perform several tasks related with Kerberos protocol in an Active Directory pentest. Installation From cra

Eloy 80 Oct 12, 2022
A substrate pallet that enables Manta's decentialized anynonymous payment (DAP) protocol.

This is a pallet that enables decentialized anynonymous payment (DAP) protocol. The best way to use this repo is to invoke it with a manta-runtime, available from either manta-node or cumulus.

Manta Network 27 Nov 23, 2022