Cap'n Proto for Rust

Last update: Jun 24, 2022

Cap'n Proto for Rust

Build Status




Cap'n Proto is a type system for distributed systems.

With Cap'n Proto, you describe your data and interfaces in a schema file, like this:


struct Point {
    x @0 :Float32;
    y @1 :Float32;

interface PointTracker {
    addPoint @0 (p :Point) -> (totalPoints :UInt64);

You can then use the capnp tool to generate code in a variety of programming languages. The generated code lets you produce and consume values of the types you've defined in your schema.

Values are encoded in a format that is suitable not only for transmission over a network and persistence to disk, but also for zero-copy in-memory traversal. That is, you can completely skip serialization and deserialization! It's in this sense that Cap'n Proto is "infinity times faster" than alternatives like Protocol Buffers.

In Rust, the generated code for the example above includes a point::Reader<'a> struct with get_x() and get_y() methods, and a point::Builder<'a> struct with set_x() and set_y() methods. The lifetime parameter 'a is a formal reminder that point::Reader<'a> and point::Builder<'a> contain borrowed references to the raw buffers that contain the encoded messages. Those underlying buffers are never actually copied into separate data structures.

The generated code for the example above also includes a point_tracker::Server trait with an add_point() method, and a point_tracker::Client struct with an add_point_request() method. The former can be implemented to create a network-accessible object, and the latter can be used to invoke a possibly-remote instance of a PointTracker.



capnp Runtime library for dealing with Cap'n Proto messages.
capnpc Rust code generator plugin, including support for hooking into a file in a cargo build.
capnp-futures Support for asynchronous reading and writing of Cap'n Proto messages.
capnp-rpc Object-capability remote procedure call system.


addressbook serialization, RPC

Who is using capnproto-rust?

Unimplemented / Future Work

  • 1. Remove need for `reborrow()`

    I looked at the 2014 blog post, which says:

    The Builder types of capnproto-rust also need to provide an exclusivity guarantee. Recall that if Foo is a struct defined in a Cap'n Proto schema, then a foo::Builder<'a> provides access to a writable location in arena-allocated memory that contains a Foo in Cap'n Proto format. To protect access to that memory, a foo::Builder<'a> ought to behave as if it were a &'a mut Foo, even though the Foo type cannot directly exist in Rust (because Cap'n Proto struct layout differs from Rust struct layout).

    The solution it comes up with is to use reborrow() everywhere. But that's not necessary, since it can just take &mut self in the accessors instead of taking an owned value; that both enforces exclusive access and allows reusing the builder later without needing a reborrow() call.

    Reviewed by jyn514 at 2021-12-13 22:02
  • 2. Sync ReadLimiter

    As mentioned in and, capnp::message::Reader<capnp::serialize::OwnedSegments> isn't Sync because of the Cell<u64> in the ReadLimiter.

    This solution will only work with platforms that have AtomicU64 and only works from 1.34 onward. This modification could be put behind a feature if this affects any user of the crate.

    Let me know !

    Reviewed by appaquet at 2020-09-11 13:57
  • 3. Support for 2018 edition

    Currently capnpc generates incompatible code with rust 2018.

    Seems to be an issue with using :: to refer to the crate root. This has been changed to crate:: in 2018.

    2015 Edition:

    pub fn get_currency(self) -> ::std::result::Result<::account_capnp::Currency,::capnp::NotInSchema> {

    2018 Edition:

    pub fn get_currency(self) -> ::std::result::Result<crate::account_capnp::Currency,::capnp::NotInSchema> {
    Reviewed by FallingSnow at 2018-08-22 21:32
  • 4. Add no_std feature

    Hey there! I've been wanting to have capnproto-rust available on no_std for awhile (per #71), and I've given it a few shots in the past but I think now I'm actually pretty close to nailing it. I wanted to open up a PR to get some feedback on my approach and ask some questions about things I've gotten hung up on.

    Things I've done:

    • Added a no_std feature flag to the capnp crate. This feature is used in a few #[cfg(...)] conditional compilations around the codebase.
    • Included the core_io crate as an optional dependency to be included when the no_std feature flag is set. This crate is mostly autogenerated in that it applies patches against std::io to take out all std usages and replace them using the alloc crate.
    • Replaced all references to std:: within the capnp crate to use core:: instead. In capnp's file, there is a conditional import based on #[cfg(feature = "no_std")]. If no_std is enabled, then the name core:: references core_io:: items. If no_std is not enabled, then core:: references std:: items. A similar strategy handles deciding between e.g. std::string and alloc::string and between std::str and alloc::str.

    Problems I'm having now:

    It seems that everything in the capnp crate is now building properly both when no_std is enabled and disabled. However, when I add the capnpc::CompilerCommand::new().file("schema.capnp").run().unwrap(); buildscript to a no_std project of mine, there's an interesting compilation problem. capnp needs to be compiled as no_std in order to comply with my no_std application, but since that is the case, the std-compiled capnpc seems to now be linking against the no_std-compiled capnp, whereas I think it should build a separate instance of capnp (std-compiled) to link against. The problem that this creates is that e.g. in capnpc/src/ let mut p = command.spawn()?;, the spawn() call returns a Result in which the error type is std::io::Error. Typically, when capnp is std-compiled, it would implement From<std::io::Error> for capnp::Error. However, since it is linking against the no_std-compiled version of capnp, the implementation that is actually implemented is From<core_io::Error> for capnp::Error. It seems that I can get around this by simply mapping the error each time capnpc deals with an io::Result, e.g. like this:

    let mut p = command.spawn().map_err(|err| capnp::Error::failed(err.to_string()))?;

    There are only perhaps five or so instances of this problem, but I figured I should ask if anybody has any suggestions on a strategy to take. I don't think it makes sense to force the no_std requirements to leak up into capnpc, but I also don't necessarily want to destroy ergonomics by forcing the client to use map_err everywhere. I think I'll probably work on adding a commit that does do that, just in order to get a working solution together, but any alternative suggestions are certainly welcome!


    I forgot to mention, since the core_io crate is built by applying patches against certain builds of std::io, you'll need a fixed version of nightly rust to build it using the no_std feature. You should be able to satisfy it using this override which corresponds to the last patch of the core_io crate:

    rustup override set nightly-2019-07-01
    Reviewed by nicholastmosher at 2019-07-28 00:55
  • 5. implement FlatArrayMessageReader

    Hi! In you say that reading from a Cap'n Proto message is comparable to a struct field access, or at least I've got that impression from your talk. Now, testing it with a simple benchmark

    #[bench] fn capnp_unpack (bencher: &mut Bencher) {
      bencher.bytes = 3;
      let encoded: Vec<u8> = {
        let mut builder = MallocMessageBuilder::new_default();
        { let mut cache_value = builder.init_root::<cache_value::Builder>();
          cache_value.set_value (b"foo"); }
        let mut buf = Vec::with_capacity (128);
        capnp::serialize::write_message (&mut buf, &builder) .unwrap();
      bencher.iter (|&:| {
        let mut bytes = encoded.as_slice();
        let reader = capnp::serialize::new_reader (&mut bytes, capnp::ReaderOptions::new()) .unwrap();
        let value = reader.get_root::<cache_value::Reader>();
        assert! (value.get_value() == b"foo");

    which uses the following schema

    struct CacheValue {
      expires @0 :UInt64;
      value @1 :Data;
      tags @2 :List(Text);

    shows that reading from a Cap'n Proto message is much more expensive than a struct access. I get 3931 ns/iter from it. (In fact, the LMDB database where I keep this value is faster at returning it from the database (616 ns/iter!) than Cap'n Proto is at decoding it).

    Theoretically I'd imagine that the decoding should involve just some memory accesses, but examining the new_reader code I see a couple of Vec allocations there.

    So, Am I doing something wrong? Is it a state of the art Cap'n Proto performance or will it be improved?

    Reviewed by ArtemGr at 2015-02-06 08:23
  • 6. async message reader with explicit continuations

    This is a sketch of a possible solution to #38. The strategy taken here is to return a enum type of Complete or Continue whenever reading or writing a message could block. Complete signals that the operation completed successfully, while Continue signals that the operation must be tried again, and carries along with it any necessary continuation state.

    The initial commit only includes a read_message implementation with continuations. The next step will be to implement a read_message_continue which takes the continuation and Read object and continues reading the message. I believe write_message can be implemented in much the same way, and perhaps with less impact to the code. Finally, I think packed serialization could be extended in much the same way.

    The motivation to use continuations instead of a higher level abstraction like Futures or Promises is that Rust doesn't have a standard implementation of these abstractions, so adapting the read/write machinery to one specific implementation would not benefit the others. My goal is that this continuation API is a stepping stone to implementing async message reading and writing in any higher level async abstraction.

    I'd love to get feedback on this strategy. I will continue filling it out in the meantime.

    Reviewed by danburkert at 2015-05-03 22:11
  • 7. `write_message` now takes ownership of the `Write` argument

    A recent commit changed write_message in from

    pub fn write_message<W, A>(write: &mut W, message: &message::Builder<A>) -> ::std::io::Result<()>


    pub fn write_message<W, A>(mut write: W, message: &message::Builder<A>) -> Result<()>

    I am wondering why this changed.

    It means you can no longer call write_message in a loop with a reference to a Write. E.g.

    pub fn transcode<I,O>(input: I, output: &mut O, packed: bool)
        -> Result<(),Box<dyn Error>>
        where I: Read, O: Write
        let write_message = if packed { serialize_packed::write_message } else { serialize::write_message };
        let input = BufReader::new(input);
        for line in input.lines() {
            let line = line?;
            let e: Event = serde_json::from_str(&line)?;
            let bin = e.to_capnp();
            write_message(output, &bin)?

    Now gives an error:

    17 | pub fn transcode<I,O>(input: I, output: &mut O, packed: bool)
       |                                             ------ move occurs because `output` has type `&mut O`, which does not implement the `Copy` trait
    28 |         write_message(output, &bin)?
       |                       ^^^^^^ value moved here, in previous iteration of loop
    Reviewed by eliaslevy at 2020-07-06 20:07
  • 8. support for no_std environments

    I came across this protocol and it looks absolutely perfect for microcontrollers! I understand some features support dynamically sized types, so cannot be supported on the stack, but was wondering if a subset of this library could support no_std?

    If it is possible, but would have to be a separate crate then that would also be useful to know.


    Reviewed by vitiral at 2017-05-27 17:23
  • 9. Using capnproto with non-blocking input/output streams

    It's prohibitively difficult to use capnproto-rust with non-blocking input/output streams (Read/Write instances that can return WouldBlock. I've been looking at the codebase the last few days to see what it would take to add support for resumable serialization/deserialization. I believe by adding a WouldBlock variant to the capnproto Error type which holds intermediate state it would be possible. So for instance:

    enum WouldBlockState {
    enum Error {
        Decode { .. },

    with additional read/write methods to resume reading a message after a WouldBlock error:

    pub fn resume_read_message(WouldBlockState, &mut InputStream, ReaderOptions) -> Result<..>;
    pub fn resume_read_message_packed(WouldBlockState, &mut BufferedInputStream, ReaderOptions) -> Result<..>;
    pub fn resume_write_message(WouldBlockState, &mut OutputStream, ReaderOptions) -> Result<..>;
    pub fn resume_write_message_packed(WouldBlockState, &mut BufferedOutputStream, ReaderOptions) -> Result<..>;

    I want to get your thoughts on this before diving too far into the implementation.

    Reviewed by danburkert at 2015-04-17 22:28
  • 10. simple function for loading messages?

    I think this could be great for python (and other) FFI so I have been trying to figure out how to load &[u8] into an actual object. This is what I have so far but I think it's probably misguided.

    pub extern fn test_capnproto(thing: *const u8, length: size_t) {
        let array: &[u8] = unsafe{ slice::from_raw_parts(thing, length as usize) };
        let mut message = MallocMessageBuilder::new_default();
        let mut test_date = message.init_root::<date::Builder>();
        { // scope for builder 
            let b = test_date.borrow();
            for (i, elem) in array.iter().enumerate() {
                b.set_data_field(i, *elem);
        println!("{:?}", test_date);

    Any tips? Perhaps something to do with read_message? I couldn't figure it out from the examples

    Reviewed by waynenilsen at 2015-07-09 14:48
  • 11. Consider using #![allow(clippy)] at top of generated files

    Clippy is giving a lot of warnings in the generated rust files. Using #![allow(clippy)] will disable clippy according to

    Reviewed by j-t-d at 2018-03-01 20:38
  • 12. How to tick an rpc server?


    I have been browsing the RPC examples in the capnproto crate. The examples are quite basic, in the sense that they never keep a RPC server alive. They run their RPC calls one after the other and shutdown the server afterwards.

    So I was wondering about this, how do I run an RPC server continuously? That allows clients to keep sending messages.

    My RPC server is setup like this (Using a very simple Handshake message schema now):

    pub struct Handshake {
        pub client: handshake::Client,
    impl Handshake {
        pub async fn setup(stream: tokio::net::TcpStream) -> Self {
                .run_until(async move {
                    let (reader, writer) =
                    let rpc_network = Box::new(twoparty::VatNetwork::new(
                    let mut rpc_system = RpcSystem::new(rpc_network, None);
                    let client: handshake::Client =
                    tokio::task::spawn_local(Box::pin(|_| ())));
                    Self { client: client }
        pub async fn send(&self, msg: &str) {
                .run_until(async move {
                    let mut req = self.client.say_hello_request();
                    tokio::task::spawn_local(async {
                        let reply = req.send().promise.await.unwrap();
                            "received: {}",

    But after this, what do I call to let the server keep processing incoming messages? I do not see any loop / processing calls in there.

    If I, after this setup, connect with a client and try to send a request using the send function above then this line let reply = req.send().promise.await.unwrap(); just hangs forever. Even more, when I debug the server, I never enter my HandshakeImpl. So somehow this send is failing silently.

    impl handshake::Server for HandshakeImpl {
        fn say_hello(
            &mut self,
            params: handshake::SayHelloParams,
            mut results: handshake::SayHelloResults,
        ) -> Promise<(), ::capnp::Error> {
            let request = params.get().unwrap().get_request().unwrap();
            let name = request.get_name().unwrap();
            let message = format!("Hello, {}!", name);

    This is probably a beginners mistake, but I cannot work it out :) I hope someone here recognizes some common pitfalls.

    Reviewed by Bvsemmer at 2022-05-30 16:13
  • 13. WIP: Improve documentation

    I found it very hard to get started with capnproto-rust. In particular, it was unclear to me how to send complex RPCs, and how to process RPCs on the server. I think I've got a handle on it now. But, I'd like to recommend adding more documentation to capnp showing idiomatic use of the interfaces (i.e., based on capnp-rpc, which is how most people will use the library, IIUC).

    Unfortunately, adding examples to capnp that use capnpc is a bit tricky. If we wanted capnp to generate a test schema, we'd need to add capnpc as a build dependency, but that would create a cycle in the dependency graph. As such, I added a new crate, capn-testdata, which compiles a schema and exports that.

    Let me know if you think this work is useful, and you agree with the approach.

    Reviewed by nwalfield at 2022-05-24 12:49
  • 14. capnpc: add option to build `capnp` exe from src

    Related to #182.

    This is one potential way we could implement option 2.) while keeping in mind @zenhack's concern about pushing users away from their package manager. The idea is that we check for the existence of capnp at build time. If it does not exist, we throw an error letting the user know to either install it OR provide feature = "build-capnp" to the crate so that we can build it for them.

    I think that it is good to at least provide this option to the user since many may find it desirable to have a plug-and-play solution without the need for prerequisites when building their crate.

    You may find adding the capnproto submodule undesirable, in which case we could potentially clone the repo at build time.

    Reviewed by rileylyman at 2022-05-15 22:41
  • 15. Small codegen alteration question/request regarding generic types with nested nodes

    Currently, nested capnp struct definitions translate into rust codegen builders/readers /etc that contain ALL possible generic types, even if they do not use them. For example:

    struct RootStruct(T) {
      tVal @0 :T;
      struct SubStruct(R) {
          rVal @0 :R;

    The SubStruct mod's structs contain both T and R generic types, even though they only use R.


    Here are some examples of some awkward usage this can potentially cause:

    let builder = TypedBuilder::<sub_struct::Owned<XXXX_unused_XXXXX, some_struct::Owned>>::new_default();
    fn write_to_builder(builder: sub_struct::Builder<XXXX_unused_XXXXX, some_struct::Owned>);

    Given the changes in this recent commit, is there any architectural reason it would not be possible to expand the concept of only including used generic types in struct/interface/group definitions instead of only unions? From some of my own (albeit limited) tests, it seems like it would be ok.

    Reviewed by aikalant at 2022-02-24 18:55
  • 16. Is there a way for a server to accept multiple type of clients?

    Hi there,

    I have two interfaces:

    interface Margin {
        struct Item {
            symbolId @0 : Int32;
            price @1 : Float64;
            amount @2 : Int32
        get @0 () -> (margin: List(Item));
    interface StockWatchList {
        struct Stock {
            symbolId @0 : Int32;
            exId @1 : Int16; 
        struct Delay {
            symbolId @0 : Int32; 
            exId @1 : Int16; 
            spiderDelay @2 : Int32; 
            source @3 : Int16;
        getWatchList @0 () -> (stock_list: List(Stock));
        insertDelay @1(delay: Delay);

    I'm trying to implement a server accepting either a margin::Client or a stock_watch_list::Client, but when constructing the rpc system, I have to specify the type of the client:

    struct RpcClient;
    impl margin::Server for RpcClient {...}
    impl stock_watch_list::Server for RpcClient {...}
    let client: margin::Client = capnp_rpc::new_client(RpcClient);
    // or 
    // let client: stock_watch_list::Client = capnp_rpc::new_client(RpcClient);
    let rpc_system = RpcSystem::new(network, Some(client.clone().client));

    Is there a way to enable this system to accept multiple types of clients?

    Thanks a lot.

    Reviewed by neodes-h at 2022-02-24 02:20
  • 17. How to return a reader?

    I want to store a Reader inside of a struct but it only uses a reference a does not implement Clone.

    use super::capnp_error::CapnpError;
    use capnp::{message::ReaderOptions, serialize_packed};
    use std::io::Cursor;
    pub struct ReaderWrapper<'a> {
        reader: &'a Reader<'a>,
    impl<'a> TryFrom<Vec<u8>> for ReaderWrapper<'a> {
        type Error = CapnpError;
        fn try_from(buffer: Vec<u8>) -> Result<ReaderWrapper<'a>, CapnpError> {
            let reader =
                &serialize_packed::read_message(&mut Cursor::new(buffer), ReaderOptions::new())?;
            let typed_reader = reader.get_root::<Reader>()?;
            Ok(ReaderWrapper {
                reader: &typed_reader,
    Reviewed by vbakc at 2022-02-18 14:04
MessagePack implementation for Rust /[Rust]

RMP - Rust MessagePack RMP is a pure Rust MessagePack implementation. This repository consists of three separate crates: the RMP core and two implemen

Jun 14, 2022
A Rust ASN.1 (DER) serializer.

rust-asn1 This is a Rust library for parsing and generating ASN.1 data (DER only). Installation Add asn1 to the [dependencies] section of your Cargo.t

May 4, 2022
Implementation of Bencode encoding written in rust

Rust Bencode Implementation of Bencode encoding written in rust. Project Status Not in active developement due to lack of time and other priorities. I

Jan 19, 2022
Encoding and decoding support for BSON in Rust

bson-rs Encoding and decoding support for BSON in Rust Index Overview of BSON Format Usage BSON Values BSON Documents Modeling BSON with strongly type

Jun 16, 2022
Rust library for reading/writing numbers in big-endian and little-endian.

byteorder This crate provides convenience methods for encoding and decoding numbers in either big-endian or little-endian order. Dual-licensed under M

Jun 16, 2022
A Gecko-oriented implementation of the Encoding Standard in Rust

encoding_rs encoding_rs an implementation of the (non-JavaScript parts of) the Encoding Standard written in Rust and used in Gecko (starting with Fire

Jun 16, 2022
Character encoding support for Rust

Encoding 0.3.0-dev Character encoding support for Rust. (also known as rust-encoding) It is based on WHATWG Encoding Standard, and also provides an ad

Jun 21, 2022
Rust implementation of CRC(16, 32, 64) with support of various standards

crc Rust implementation of CRC(16, 32, 64). MSRV is 1.46. Usage Add crc to Cargo.toml [dependencies] crc = "2.0" Compute CRC use crc::{Crc, Algorithm,

Jun 4, 2022
A CSV parser for Rust, with Serde support.

csv A fast and flexible CSV reader and writer for Rust, with support for Serde. Dual-licensed under MIT or the UNLICENSE. Documentation https://docs.r

Jun 17, 2022
A HTTP Archive format (HAR) serialization & deserialization library, written in Rust.

har-rs HTTP Archive format (HAR) serialization & deserialization library, written in Rust. Install Add the following to your Cargo.toml file: [depende

Jan 22, 2022
A HTML entity encoding library for Rust

A HTML entity encoding library for Rust Example usage All example assume a extern crate htmlescape; and use htmlescape::{relevant functions here}; is

Mar 29, 2022
pem-rs pem PEM jcreekmore/pem-rs [pem] — A Rust based way to parse and encode PEM-encoded data

pem A Rust library for parsing and encoding PEM-encoded data. Documentation Module documentation with examples Usage Add this to your Cargo.toml: [dep

May 8, 2022
PROST! a Protocol Buffers implementation for the Rust Language

PROST! prost is a Protocol Buffers implementation for the Rust Language. prost generates simple, idiomatic Rust code from proto2 and proto3 files. Com

Jun 15, 2022
Rust implementation of Google protocol buffers

rust-protobuf Protobuf implementation in Rust. Written in pure rust Generate rust code Has runtime library for generated code (Coded{Input|Output}Stre

Jun 23, 2022
tnetstring serialization library for rust.

TNetStrings: Tagged Netstrings This module implements bindings for the tnetstring serialization format. API let t = tnetstring::str("hello world"); le

Jul 14, 2019
A TOML encoding/decoding library for Rust

toml-rs A TOML decoder and encoder for Rust. This library is currently compliant with the v0.5.0 version of TOML. This library will also likely contin

Jun 23, 2022
A fast, performant implementation of skip list in Rust.
A fast, performant implementation of skip list in Rust.

Subway A fast, performant implementation of skip list in Rust. A skip list is probabilistic data structure that provides O(log N) search and insertion

Apr 5, 2022
A Rust PAC for the RP2040 Microcontroller

rp2040-pac - PAC for Raspberry Pi RP2040 microcontrollers This is a Peripheral Access Crate for the Raspberry Pi RP2040 dual-core Cortex-M0+ microcont

May 21, 2022
Pure Rust port of CRFsuite: a fast implementation of Conditional Random Fields (CRFs)

crfs-rs Pure Rust port of CRFsuite: a fast implementation of Conditional Random Fields (CRFs) Currently only support prediction, model training is not

Mar 22, 2022