Fully typed access to the Erigon db in rust

Overview

Overview

Fully typed access to the Erigon database in rust.

use erigon_db::{Erigon, env_open};
use ethereum_types::Address;

fn main() -> eyre::Result<()> {
    let path = std::path::Path::new(env!("ERIGON_CHAINDATA"));

    // Open an mdbx environment and begin a read-only database transaction
    let env = env_open(path)?;
    let db = Erigon::begin(&env)?;

    // get the canonical head block header
    let head_hash = db.read_head_header_hash()?.unwrap();
    let head_num = db.read_header_number(head_hash)?.unwrap();
    let header = db.read_header((head_num, head_hash))?.unwrap();

    // get the current state of an account
    let contract: Address = "0x0d4c6c6605a729a379216c93e919711a081beba2".parse()?;
    let account = db.read_account(contract)?.unwrap();
    let bytecode = db.read_code(account.codehash)?.unwrap();

    // get all of the contract's populated storage slots
    // (incarnation is an Erigon-specific value related to removal/revival of contracts)
    for read in db.walk_storage(contract, account.incarnation)? {
        let (slot, value) = read?;
        println!("The value at slot {} is {}", slot, value);
    }

    // get the state of the account at block 100
    let old_account = db.read_account_hist(contract, 100)?.unwrap_or_default();

    Ok(())
}

Acknowledgements

Much of this code has been taken from the Akula Ethereum client in order to enable its use with the stable rust toolchain. In particular, it repurposes many of Akula's kv utilities and abstractions for working with libmdbx and Ethereum data. These abstractions are extremely high-quality in my opinion, so the primary modifications were increasing the strictness and expressiveness of the accessor types and tailoring to Erigon's data representations and database layout.

Resources

  • Erigon has an excellent doc walking through their database layout, though it may not match the current implementation in some places.
  • Erigon's core/rawdb/accessors_*.go contains many of their low-level database interactions.
  • Some comments from Erigon on the advisability of this endeavor.
  • For some brief info on mdbx, see doc/mdbx.md.
  • The Erigon database layout is defined in src/erigon/tables.rs, and you can see how these tables are read in src/erigon/mod.rs.
Comments
  • Compile error: no method named `open_db_with_flags`

    Compile error: no method named `open_db_with_flags`

    Error when compiling

    error[E0599]: no method named `open_db_with_flags` found for struct `libmdbx::Transaction<'env, M, NoWriteMap>` in the current scope
       --> /home/m4tsuri/.cargo/git/checkouts/erigon-db-118ee4f05d317cb2/f1a7261/src/kv/mod.rs:142:24
        |
    142 |             self.inner.open_db_with_flags(Some(Db::NAME), flags)?,
        |                        ^^^^^^^^^^^^^^^^^^ method not found in `libmdbx::Transaction<'env, M, NoWriteMap>`
        |
        = note: the method was found for
                - `libmdbx::Transaction<'env, RW, E>`
    
    • erigon_db: version: f1a7261
    • rust version: rustc 1.65.0-nightly (c07a8b4e0 2022-08-26)
    opened by M4tsuri 2
  • Compile failing due to ::fastrlp::RlpMacEncodedLen not implemented for trait bound `U256: MaxEncodedLenAssoc`

    Compile failing due to ::fastrlp::RlpMacEncodedLen not implemented for trait bound `U256: MaxEncodedLenAssoc`

    I'm running into a compiler issue when attempting to import erigon-db into a project I'm working on. I'm new to Rust, so I'm probably missing something simple.

    Cloning and running erigon-db from source with the following main.rs file leads to the code compiling and running as expected

    git clone erigon-db
    cd erigon-db
    cargo run --bin erigon-db
    
    // ./erigon-db/src/main.rs 
    use erigon_db::{Erigon, env_open};
    use ethereum_types::Address;
    use std::fs::File;
    use std::io::Write;
    use std::io::LineWriter;
    
    fn main() -> eyre::Result<()> {
        let path = std::path::Path::new(env!("ERIGON_CHAINDATA"));
    
        // Open an mdbx environment and begin a read-only database transaction
        let env = env_open(path)?;
        let db = Erigon::begin(&env)?;
    
        // get the current state of an account
        let contract: Address = "0x5C69bEe701ef814a2B6a3EDD4B1652CB9cc5aA6f".parse()?; //univ2factory
        let account = db.read_account(contract)?.unwrap();
    
        // write populated storage slots to file
        let file = File::create("output.txt")?;
        let mut file = LineWriter::new(file);
    
        for read in db.walk_storage(contract, account.incarnation, None)? {
            let (slot, value) = read?;
            file.write(format!("{}: {}\n", slot, value).as_bytes())?;
        }
    
        Ok(())
    }
    

    However, when I create a new cargo project and use erigon-db as a dependency:

    # Cargo.toml
    # --snip--
    [dependencies]
    erigon-db = {git = "https://github.com/gio256/erigon-db"}
    ethers = { git = "https://github.com/gakonst/ethers-rs", optional = true }
    

    I get the following error when running cargo run:

    error[E0277]: the trait bound `U256: MaxEncodedLenAssoc` is not satisfied
       --> /home/plotchy/.cargo/git/checkouts/erigon-db-118ee4f05d317cb2/9b1347a/src/erigon/macros.rs:306:13
        |
    306 |             ::fastrlp::RlpMaxEncodedLen,
        |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `MaxEncodedLenAssoc` is not implemented for `U256`
        |
       ::: /home/plotchy/.cargo/git/checkouts/erigon-db-118ee4f05d317cb2/9b1347a/src/erigon/models/transaction.rs:93:1
        |
    93  | decl_u256_wrapper!(VPackChainId);
        | -------------------------------- in this macro invocation
        |
        = help: the following other types implement trait `MaxEncodedLenAssoc`:
                  Bloom
                  H128
                  H160
                  H256
                  H512
                  H520
                  H64
                  VPackChainId
                and 12 others
        = note: this error originates in the derive macro `::fastrlp::RlpMaxEncodedLen` (in Nightly builds, run with -Z macro-backtrace for more info)
    

    It seems that the forked repo with the uint patches created in the develop branch of gio256/fastrlp is not being found?

    opened by plotchy 2
  • build on fresh checkout fails to compile

    build on fresh checkout fails to compile

    The project fails to build on fresh checkout with the following issues. It is very likely I'm just being dumb, but wanted to enquire anyhow.

    error: there is no argument named `expected`
    .cargo/git/checkouts/fastrlp-4d3cd332b9d5b08f/70f5c13/src/decode.rs:74:59
       |
    74 |                 write!(f, "list length mismatch: expected {expected}, got {got}")
       |                                                           ^^^^^^^^^^
    
    error: there is no argument named `got`
    .cargo/git/checkouts/fastrlp-4d3cd332b9d5b08f/70f5c13/src/decode.rs:74:75
       |
    74 |                 write!(f, "list length mismatch: expected {expected}, got {got}")
       |                                                                           ^^^^^
    
    error: there is no argument named `err`
    cargo/git/checkouts/fastrlp-4d3cd332b9d5b08f/70f5c13/src/decode.rs:76:52
       |
    76 |             DecodeError::Custom(err) => write!(f, "{err}"),
       |                                                    ^^^^^
    error: could not compile `fastrlp` due to 3 previous errors
    
    opened by totlsota 2
  • build.rs fails to compile -> ethers-rs has made ethers::solc a feature rather than default

    build.rs fails to compile -> ethers-rs has made ethers::solc a feature rather than default

    This pr for ethers-rs has made ethers-solc an optional feature: https://github.com/gakonst/ethers-rs/pull/1463

    build.rs depends on ethers::solc

    [build-dependencies]
    ethers = { git = "https://github.com/gakonst/ethers-rs" , features = ["ethers-solc"]}
    

    After making the change, build.rs still fails to compile. I'm not familiar with the logic in that file, otherwise I would've made the fix a PR. Hope this helps.

    error[E0061]: this function takes 2 arguments but 1 argument was supplied
       --> /home/plotchy/.cargo/git/checkouts/erigon-db-d1f243f9aeca17a7/d88f5bf/build.rs:29:31
        |
    29  |         let contract = output.find(name).ok_or_else(|| {
        |                               ^^^^------ an argument is missing
        |
    note: associated function defined here
       --> /home/plotchy/.cargo/git/checkouts/ethers-rs-c3a7c0a0ae0fe6be/8db1899/ethers-solc/src/compile/output/mod.rs:246:12
        |
    246 |     pub fn find(&self, path: impl AsRef<str>, contract: impl AsRef<str>) -> Option<&T::Artifact> {
        |            ^^^^
    help: provide the argument
        |
    29  |         let contract = output.find(name, {_}).ok_or_else(|| {
        |                               ~~~~~~~~~~~~~~~
    
    opened by plotchy 1
  • Error type in public API

    Error type in public API

    Currently this crate is heavily using eyre::Report as its error type, even in public APIs. However, according to eyre's docs, this is not recommended.

    I'm trying to add a public error type to this crate. If you are interested, I'll make a PR later. Thank you.

    opened by M4tsuri 0
Owner
Gio
Gio
A Rust library for working with Bitcoin SV

Rust-SV A library to build Bitcoin SV applications in Rust. Documentation Features P2P protocol messages (construction and serialization) Address enco

Brenton Gunning 51 Oct 13, 2022
Coinbase pro client for Rust

Coinbase pro client for Rust Supports SYNC/ASYNC/Websocket-feed data support Features private and public API sync and async support websocket-feed sup

null 126 Dec 30, 2022
Custom Ethereum vanity address generator made in Rust

ethaddrgen Custom Ethereum address generator Get a shiny ethereum address and stand out from the crowd! Disclaimer: Do not use the private key shown i

Jakub Hlusička 153 Dec 27, 2022
The new, performant, and simplified version of Holochain on Rust (sometimes called Holochain RSM for Refactored State Model)

Holochain License: This repository contains the core Holochain libraries and binaries. This is the most recent and well maintained version of Holochai

Holochain 737 Dec 28, 2022
DEPRECATED. The Holochain framework implemented in rust with a redux style internal state-model.

Holochain-rust Travis: Circle CI: Codecov: License: This code is loosely based on the previous Golang prototype. Code Status: This Rust version is alp

Holochain 1k Jan 3, 2023
IBC modules and relayer - Formal specifications and Rust implementation

ibc-rs Rust implementation of the Inter-Blockchain Communication (IBC) protocol. This project comprises primarily four crates: The ibc crate defines t

Informal Systems 296 Jan 4, 2023
A Rust implementation of BIP-0039

bip39-rs A Rust implementation of BIP0039 Changes See the changelog file, or the Github releases for specific tags. Documentation Add bip39 to your Ca

Infincia LLC 49 Dec 9, 2022
Rust Ethereum 2.0 Client

Lighthouse: Ethereum 2.0 An open-source Ethereum 2.0 client, written in Rust and maintained by Sigma Prime. Documentation Overview Lighthouse is: Read

Sigma Prime 2.1k Jan 1, 2023
Official Rust implementation of the Nimiq protocol

Nimiq Core implementation in Rust (core-rs) Rust implementation of the Nimiq Blockchain Core Nimiq is a frictionless payment protocol for the web. Thi

Nimiq 72 Sep 23, 2022
Rust implementation of Zcash protocol

The Parity Zcash client. Gitter Blog: Parity teams up with Zcash Foundation for Parity Zcash client Installing from source Installing the snap Running

Parity Technologies 183 Sep 8, 2022
rust client libraries to deal with the current cardano mainnet (byron / cardano-sl)

Rust implementation of Cardano primitives, helpers, and related applications Cardano Rust is a modular toolbox of Cardano’s cryptographic primitives,

Input Output 275 Oct 9, 2022
Tendermint in Rust!

tendermint.rs Tendermint in Rust with TLA+ specifications. Tendermint is a high-performance blockchain consensus engine for Byzantine fault tolerant a

Informal Systems 439 Jan 1, 2023
A Rust library for generating cryptocurrency wallets

Table of Contents 1. Overview 2. Build Guide 2.1 Install Rust 2.2a Build from Homebrew 2.2b Build from Crates.io 2.2c Build from Source Code 3. Usage

Aleo 554 Dec 31, 2022
Rust port of the Terry Davis' (RIP) "god says" program

RIP Terry A. Davis 1969-2018 god says Rust port of the programmer Terry Davis' "god says" (AKA GodSpeaks) program. Terrence Andrew Davis (December 15,

Orhun Parmaksız 54 Dec 26, 2022
Implementation of the Kademlia DHT protocol in Rust

kademlia-dht Simple implementation of the Kademlia DHT protocol in Rust with state dumping features for educational purposes (not production-ready). T

Leonardo Folgoni 18 Sep 24, 2022
Collection of Key Derivation Functions written in pure Rust

RustCrypto: Key Derivation Functions Collection of Key Derivation Functions (KDF) written in pure Rust. Supported Algorithms Algorithm Crate Crates.io

Rust Crypto 44 Dec 25, 2022
Cryptocurrencies trend-following trading bot sandbox written in Rust.

Trend trading bot Experiments repo about (crypto) trend trading. By "trend" I mean trading following the trend using technical indicators (vs other ki

Julien 6 Oct 2, 2022
Next-generation implementation of Ethereum protocol ("client") written in Rust, based on Erigon architecture.

?? Martinez ?? Next-generation implementation of Ethereum protocol ("client") written in Rust, based on Erigon architecture. Why run Martinez? Look at

Arthur·Thomas 23 Jul 3, 2022
Martinez is vNext Ethereum implementation written in pure Rust with Erigon architecture as design.

?? Martinez ?? Next-generation implementation of Ethereum protocol ("client") written in Rust, based on Erigon architecture. Why run Martinez? Look at

Arthur·Thomas 23 Jul 3, 2022
Fully typed SQL query builder for Rust [deprecated]

What is Deuterium? Deuterium is a fancy SQL builder for Rust. It's designed to provide a DSL to easily build SQL queries in safe and typed way. Like R

Stanislav Panferov 169 Nov 20, 2022