A (de)serializer for RLP encoding in ETH

Overview

A (de)serializer for RLP encoding in ETH

Cargo.toml

serlp = "0.3.0"
serde = { version = "1.0", features = ['derive'] }

Example

This example shows how can we encode a real transcation on ETH mainnet with the help of serlp.

The transcation is the #0 transcation of https://api.etherscan.io/api?module=proxy&action=eth_getBlockByNumber&tag=0xa1a489&boolean=true&apikey=YourApiKeyToken. The encoded data is from README of https://github.com/zhangchiqing/merkle-patricia-trie.

, #[serde(with = "biguint")] v: BigUint, #[serde(with = "biguint")] r: BigUint, #[serde(with = "biguint")] s: BigUint } let mut to = [0; 20]; to.copy_from_slice(&hex::decode("a3bed4e1c75d00fa6f4e5e6922db7261b5e9acd2").unwrap()); let bn = |s| BigUint::from_bytes_be(&hex::decode(s).unwrap()); let tx = LegacyTx { nonce: 0xa5, gas_price: bn("2e90edd000"), gas_limit: 0x12bc2, to, value: bn("00"), data: hex::decode("a9059cbb0000000000000000000000008bda8b9823b8490e8cf220dc7b91d97da1c54e250000000000000000000000000000000000000000000000056bc75e2d63100000").unwrap(), v: bn("26"), r: bn("6c89b57113cf7da8aed7911310e03d49be5e40de0bd73af4c9c54726c478691b"), s: bn("56223f039fab98d47c71f84190cf285ce8fc7d9181d6769387e5efd0a970e2e9") }; let expected = "f8ab81a5852e90edd00083012bc294a3bed4e1c75d00fa6f4e5e6922db7261b5e9acd280b844a9059cbb0000000000000000000000008bda8b9823b8490e8cf220dc7b91d97da1c54e250000000000000000000000000000000000000000000000056bc75e2d6310000026a06c89b57113cf7da8aed7911310e03d49be5e40de0bd73af4c9c54726c478691ba056223f039fab98d47c71f84190cf285ce8fc7d9181d6769387e5efd0a970e2e9"; let encoded = to_bytes(&tx).unwrap(); let orig: LegacyTx = from_bytes(&encoded).unwrap(); assert_eq!(orig, tx); assert_eq!(hex::encode(encoded), expected); }">
fn test_bn() {
    #[derive(Serialize, Deserialize, PartialEq, Eq, Debug)]
    struct LegacyTx {
        nonce: u64,
        #[serde(with = "biguint")]
        gas_price: BigUint,
        gas_limit: u64,
        #[serde(with = "byte_array")]
        to: [u8; 20],
        #[serde(with = "biguint")]
        value: BigUint,
        #[serde(with = "serde_bytes")]
        data: Vec<u8>,
        #[serde(with = "biguint")]
        v: BigUint,
        #[serde(with = "biguint")]
        r: BigUint,
        #[serde(with = "biguint")]
        s: BigUint
    }

    let mut to = [0; 20];
    to.copy_from_slice(&hex::decode("a3bed4e1c75d00fa6f4e5e6922db7261b5e9acd2").unwrap());
    let bn = |s| BigUint::from_bytes_be(&hex::decode(s).unwrap());
    
    let tx = LegacyTx {
        nonce: 0xa5,
        gas_price: bn("2e90edd000"),
        gas_limit: 0x12bc2,
        to,
        value: bn("00"),
        data: hex::decode("a9059cbb0000000000000000000000008bda8b9823b8490e8cf220dc7b91d97da1c54e250000000000000000000000000000000000000000000000056bc75e2d63100000").unwrap(),
        v: bn("26"),
        r: bn("6c89b57113cf7da8aed7911310e03d49be5e40de0bd73af4c9c54726c478691b"),
        s: bn("56223f039fab98d47c71f84190cf285ce8fc7d9181d6769387e5efd0a970e2e9")
    };

    let expected = "f8ab81a5852e90edd00083012bc294a3bed4e1c75d00fa6f4e5e6922db7261b5e9acd280b844a9059cbb0000000000000000000000008bda8b9823b8490e8cf220dc7b91d97da1c54e250000000000000000000000000000000000000000000000056bc75e2d6310000026a06c89b57113cf7da8aed7911310e03d49be5e40de0bd73af4c9c54726c478691ba056223f039fab98d47c71f84190cf285ce8fc7d9181d6769387e5efd0a970e2e9";
    let encoded = to_bytes(&tx).unwrap();
    let orig: LegacyTx = from_bytes(&encoded).unwrap();

    assert_eq!(orig, tx);
    assert_eq!(hex::encode(encoded), expected);
}

Not Supported Types

  • bool
  • float numbers and signed integers
  • maps
  • enum (only deserialize)

We do not support enum when deserializing because we lost some information (i.e. variant index) about the original value when serializing. However, in some specific cases you can derive Deserialize trait for a enum with the help of RlpProxy, which will be discussed later.

We have to choose this approach because there is no enums in Golang while ETH is written in go. Treating enums as a transparent layer can make our furture implementation compatiable with ETH.

Design principle

Accroding to the ETH Yellow Paper, all supported data structure can be represented with either recursive list of byte arrays or byte arrays . So we can transform all Rust's compound types, for example, tuple, struct and list, into lists. And then encode them as exactly described in the paper

Features

RLP Proxy

We have a RlpProxy struct that implemented Deserialize trait, which just stores the original rlp encoded data after deserialization (no matter what type it is). You can gain more control over the deserialization process with it.

Here is an example:

for Classify { fn from(proxy: RlpProxy) -> Self { let raw = proxy.raw(); let mut tree = proxy.rlp_tree(); if tree.value_count() == 2 { return Classify::Ten(from_bytes(raw).unwrap()) } let val = tree.next().unwrap()[0]; match val { 0 => Classify::Zero(0), 1 => Classify::One(1), _ => panic!("Value Error.") } } }">
#[derive(Serialize, Deserialize, PartialEq, Eq, Debug)]
#[serde(from = "RlpProxy")]
enum Classify {
    Zero(u8),
    One(u8),
    Ten((u8, u8))
}

impl From 
   for 
   Classify {
    
   fn 
   from(proxy: RlpProxy) -> 
   Self {
        
   let raw 
   = proxy.
   raw();
        
   let 
   mut tree 
   = proxy.
   rlp_tree();
        
   if tree.
   value_count() 
   == 
   2 {
            
   return Classify
   ::
   Ten(
   from_bytes(raw).
   unwrap())
        }

        
   let val 
   = tree.
   next().
   unwrap()[
   0];
        
   match val {
            
   0 
   => Classify
   ::
   Zero(
   0),
            
   1 
   => Classify
   ::
   One(
   1),
            _ 
   => 
   panic!(
   "Value Error.")
        }
    }
}
  

(de)serializers for frequently used types

We provide two (de)serializers for frequently used types in blockchain.

  • biguint for num_bigint::BigUint
  • byte_array for [u8; N]

Put #[serde(with = "biguint")] or #[serde(with = "byte_array")] before your struct field to use them.

You might also like...
Yet Another Serializer/Deserializer

yaserde   Yet Another Serializer/Deserializer specialized for XML Goal This library will support XML de/ser-ializing with all specific features. Suppo

A Rust ASN.1 (DER) serializer.

rust-asn1 This is a Rust library for parsing and generating ASN.1 data (DER only). Installation Add asn1 to the [dependencies] section of your Cargo.t

Serializer and deserializer for the VCR Cassette format

vcr-cassette Serializer and deserializer for the VCR Cassette format API Docs | Releases | Contributing Examples Given the following .json VCR Cassett

A lightning-fast Sixel serializer/deserializer

sixel-image This is a (pretty fast!) sixel serializer/deserializer with cropping support. It accepts a sixel serialized string byte-by-byte, deseriali

An HCL serializer/deserializer for rust

hcl-rs This crate provides functionality to deserialize, serialize and manipulate HCL data. The main types are Deserializer for deserializing data, Se

Macro for fast implementing serialize methods in serde::Serializer trait

impl_serialize! This library provides a simple procedural macro for fast implementing serialize methods in serde::Serializer trait. [dependencies] imp

Procedural macro to derive Serde serializer-deserializer for Prost

prost-serde-derive prost-serde-derive is a procedural macro to generate Serde serializers and deserializers for Prost-generated structs. Rationale Cur

Token De/Serializer for testing De/Serialize implementations

serde_test   This crate provides a convenient concise way to write unit tests for implementations of Serialize and Deserialize. The Serialize impl for

I/O and binary data encoding for Rust

nue A collection of tools for working with binary data and POD structs in Rust. pod is an approach at building a safe interface for transmuting POD st

Implementation of Bencode encoding written in rust

Rust Bencode Implementation of Bencode encoding written in rust. Project Status Not in active developement due to lack of time and other priorities. I

Encoding and decoding support for BSON in Rust

bson-rs Encoding and decoding support for BSON in Rust Index Overview of BSON Format Usage BSON Values BSON Documents Modeling BSON with strongly type

A Gecko-oriented implementation of the Encoding Standard in Rust

encoding_rs encoding_rs an implementation of the (non-JavaScript parts of) the Encoding Standard written in Rust and used in Gecko (starting with Fire

Character encoding support for Rust

Encoding 0.3.0-dev Character encoding support for Rust. (also known as rust-encoding) It is based on WHATWG Encoding Standard, and also provides an ad

A HTML entity encoding library for Rust

A HTML entity encoding library for Rust Example usage All example assume a extern crate htmlescape; and use htmlescape::{relevant functions here}; is

A TOML encoding/decoding library for Rust

toml-rs A TOML decoder and encoder for Rust. This library is currently compliant with the v0.5.0 version of TOML. This library will also likely contin

Encoding and decoding images in Rust
Encoding and decoding images in Rust

Image Maintainers: @HeroicKatora, @fintelia How to contribute An Image Processing Library This crate provides basic image processing functions and met

Rust-tokenizer offers high-performance tokenizers for modern language models, including WordPiece, Byte-Pair Encoding (BPE) and Unigram (SentencePiece) models

rust-tokenizers Rust-tokenizer offers high-performance tokenizers for modern language models, including WordPiece, Byte-Pair Encoding (BPE) and Unigra

Google Encoded Polyline encoding & decoding in Rust.

polyline Google Encoded Polyline encoding & decoding in Rust. A Note on Coordinate Order This crate uses Coordinate and LineString types from the geo-

TIFF decoding and encoding library in pure Rust

image-tiff TIFF decoding and encoding library in pure Rust Supported Features Baseline spec (other than formats and tags listed below as not supported

Owner
M4tsuri
CTFer
M4tsuri
CBOR (binary JSON) for Rust with automatic type based decoding and encoding.

THIS PROJECT IS UNMAINTAINED. USE serde_cbor INSTEAD. This crate provides an implementation of RFC 7049, which specifies Concise Binary Object Represe

Andrew Gallant 121 Dec 27, 2022
Derive macro for encoding/decoding instructions and operands as bytecode

bytecoding Derive macro for encoding and decoding instructions and operands as bytecode. Documentation License Licensed under either of Apache License

Niklas Sauter 15 Mar 20, 2022
A rust bencode encoding/decoding implementation backed by serde.

Bende A rust bencode encoding/decoding implementation backed by serde. About This is one of a few bencode implementations available for rust. Though t

Du Toit 3 Dec 4, 2022
Easy c̵̰͠r̵̛̠ö̴̪s̶̩̒s̵̭̀-t̶̲͝h̶̯̚r̵̺͐e̷̖̽ḁ̴̍d̶̖̔ ȓ̵͙ė̶͎ḟ̴͙e̸̖͛r̶̖͗ë̶̱́ṉ̵̒ĉ̷̥e̷͚̍ s̷̹͌h̷̲̉a̵̭͋r̷̫̊ḭ̵̊n̷̬͂g̵̦̃ f̶̻̊ơ̵̜ṟ̸̈́ R̵̞̋ù̵̺s̷̖̅ţ̸͗!̸̼͋

Rust S̵̓i̸̓n̵̉ I̴n̴f̶e̸r̵n̷a̴l mutability! Howdy, friendly Rust developer! Ever had a value get m̵̯̅ð̶͊v̴̮̾ê̴̼͘d away right under your nose just when

null 294 Dec 23, 2022
A temporary repo for ETH connector to be used by EVM

ETH connector for Rainbow bridge Definitions bridgedETH - NEP-141 fungible-token representation of ETH inside Near. nETH - native ETH inside Near EVM.

Project Aurora 36 Jul 26, 2022
An Rust implementation of the Modified Merkle Patricia Tree described by ETH Yellow Paper

Merkle Patricia Tree的Rust实现 博客文章:https://dere.press/2022/01/24/eth-trie/ 本实现参考下列项目: https://ethereum.github.io/yellowpaper/paper.pdf https://github.co

M4tsuri 3 Dec 13, 2022
Rust project for working with ETH - Ethereum transactions with Rust on Ganache and also deploy smart contracts :)

Just a test project to work with Ethereum but using Rust. I'm using plain Rust here, not Foundry. In future we will use Foundry. Hope you're already f

Akhil Sharma 2 Dec 20, 2022
Ethereum key tool - Lightweight CLI tool to deal with ETH keys written in rust

ekt - Etherum Key Tool ekt is a lightweight tool to generate ethereum keys and addresses. Installation Either clone it and run it with cargo or instal

null 5 May 8, 2023
A tool to deserialize data from an input encoding, transform it and serialize it back into an output encoding.

dts A simple tool to deserialize data from an input encoding, transform it and serialize it back into an output encoding. Requires rust >= 1.56.0. Ins

null 11 Dec 14, 2022
Databento Binary Encoding (DBZ) - Fast message encoding and storage format for market data

dbz A library (dbz-lib) and CLI tool (dbz-cli) for working with Databento Binary Encoding (DBZ) files. Python bindings for dbz-lib are provided in the

Databento, Inc. 15 Nov 4, 2022