Substreams development kit for Ethereum chains, contains Firehose Block model and helpers as well as utilities for Ethereum ABI encoding/decoding.

Overview

Substreams Ethereum

Substreams development kit for Ethereum chains, contains Rust Firehose Block model and helpers as well as utilities for Ethereum ABI encoding/decoding.

Usage

[package]
name = "substreams-acme"
version = 0.1.2

[lib]
crate-type = ["cdylib"]

[dependencies]
substreams-ethereum = "0.1.0"

Development

We manually keep in sync the rendered Rust Firehose Block models with the actual Protocol Buffer definitions file found in sf-ethereum and we commit them to Git.

This means changes to Protobuf files must be manually re-generated and commit, see below for how to do it.

Regenerate Rust Firehose Block from Protobuf

./gen.sh

Release

Important Don't forget to replace ${version} by your real version like 0.1.3!

  • Ensure build and tests
  • Ensure you are in a clean and pushed Git state
  • Find & replace all occurrences of Regex ^version = "[^"]+" in all Cargo.toml files to version = "${version}"
  • Find & replace all occurrences of Regex ^substreams-ethereum(-[^ =]+)\s*=\s*\{\s*version\s*=\s*"[^"]+" in all Cargo.toml files to substreams-ethereum$1 = { version = "${version"
  • Update the CHANGELOG.md to update the ## Unreleased header to become `## v0.1.2
  • Commit everything with message Preparing release of ${version}.
  • ./bin/release.sh v${version} (Add -f before v${version} to doing the real non-dry mode)
  • If everything went well, git tag ${version}
  • Git push everything

You can then update the https://github.com/streamingfast/substreams-template with the latest

Community

Need any help? Reach out!

License

Apache 2.0

Comments
  • Add view iterators to improve ergonomics

    Add view iterators to improve ergonomics

    This adds a block_view module, which adds helper methods to Block to iterate over transactions, receipts, logs and events. These views hold a reference to the context, so for example the receipt view holds both the receipt itself and a 'backreference' to the respective transaction. Depends on #4.

    This ultimately allows writing a handler as easily as:

    #[substreams::handlers::map]
    fn map_pairs(block: eth::Block) -> Result<uniswap::Pairs, Error> {
        let pairs = block
            .events::<abi::factory::events::PairCreated>(&[&FACTORY])
            .map(|(event, log)| {
                log::info!("Pair created: 0x{}", Hex(&event.pair));
    
                uniswap::Pair {
                    address: event.pair,
                    token0: event.token0,
                    token1: event.token1,
                    ordinal: log.ordinal(),
                }
            })
            .collect();
        Ok(uniswap::Pairs { pairs })
    }
    
    opened by leoyvens 3
  • Rework crate structure

    Rework crate structure

    Right now there are three crates, the main substreams-ethereum, substreams-ethereum-abigen and substreams-ethereum-derive. substreams-ethereum depends on the other two, for re-exporting them and serving as a single entry point. However it also contains core types which other crates may want to depend on, but can't because it creates a circular dependency.

    My suggestion would be to switch to a diamond structure, keeping substreams-ethereum as the single entrypoint which depends on all others, but move all the core types to a substreams-ethereum-core which depends on no other crates in the workspace, so all crates can depend on substreams-ethereum-core and interoperate through its types.

    opened by leoyvens 3
  • Batch rpc calls

    Batch rpc calls

    Do you want to request a feature or report a bug? Feature request

    What is the current behavior? Separate rpc calls

    If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem. N/A

    What is the expected behavior? Instead of doing

    let fee0 = abi::pool::functions::FeeGrowthGlobal0X128
    fee0.call(...)
    let fee1 = abi::pool::functions::FeeGrowthGlobal1X128
    fee1.call(...)
    

    We could provide a helper to do both calls at the same time instead of doing 2 calls separately.

    opened by Eduard-Voiculescu 1
  • WIP Added first working version of ABI generation for functions

    WIP Added first working version of ABI generation for functions

    It's possible to:

    • Transform a pb::eth::v2::Call into ABI generated "function" (extracts decoded params of a call)
    • Decode the output of pb::eth::v2::Call (extracts decoded output params of a call)
    • Encode ABI generated "function" into Ethereum packed binary encoding
    • Perform RPC call for any function defined in the ABI with automatic encoding of input and decoding of output

    There is good chance the API will slightly change or be improved based on feedback.

    opened by maoueh 1
  • Properly converting an `int` by using `BigInt` lib

    Properly converting an `int` by using `BigInt` lib

    matching on int and overloading the decoding which took the int and converted it into a U256 from ethereum. Big int will give us the correct value for the int as we want it to overflow to a negative value

    opened by Eduard-Voiculescu 1
  • Fix and test FixedBytes size

    Fix and test FixedBytes size

    FixedBytes in ethabi have length equal to the actual length in bytes. Therefore, the (byte_count / 32) + 1 seemed like a bug to me and was causing my substream to bug on bytes32 fields in the events.

    I've removed the calculation and added an extra test as well.

    opened by electricddev 0
  • RUSTSEC-2022-0054: wee_alloc is Unmaintained

    RUSTSEC-2022-0054: wee_alloc is Unmaintained

    wee_alloc is Unmaintained

    | Details | | | ------------------- | ---------------------------------------------- | | Status | unmaintained | | Package | wee_alloc | | Version | 0.4.5 | | URL | https://github.com/rustwasm/wee_alloc/issues/107 | | Date | 2022-05-11 |

    Two of the maintainers have indicated that the crate may not be maintained.

    The crate has open issues including memory leaks and may not be suitable for production use.

    It may be best to switch to the default Rust standard allocator on wasm32 targets.

    Last release seems to have been three years ago.

    Possible Alternative(s)

    The below list has not been vetted in any way and may or may not contain alternatives;

    Honorable Mention(s)

    The below may serve to educate on potential future alternatives:

    See advisory page for additional details.

    opened by github-actions[bot] 0
  • RUSTSEC-2021-0139: ansi_term is Unmaintained

    RUSTSEC-2021-0139: ansi_term is Unmaintained

    ansi_term is Unmaintained

    | Details | | | ------------------- | ---------------------------------------------- | | Status | unmaintained | | Package | ansi_term | | Version | 0.12.1 | | URL | https://github.com/ogham/rust-ansi-term/issues/72 | | Date | 2021-08-18 |

    The maintainer has adviced this crate is deprecated and will not receive any maintenance.

    The crate does not seem to have much dependencies and may or may not be ok to use as-is.

    Last release seems to have been three years ago.

    Possible Alternative(s)

    The below list has not been vetted in any way and may or may not contain alternatives;

    See advisory page for additional details.

    opened by github-actions[bot] 0
Releases(v0.8.0)
  • v0.8.0(Nov 28, 2022)

    Changelog

    • Bump substreams crate
    • Changing into() call when converting to BigInt in abi to call the proper signed or unsiged encoding
    • Adding into() method from BigInt to use unsigned_bytes_be

    Commits

    • b69dc5a Preparing release of 0.8.0
    • f6a695a changing into() call on chained into_uint() and into_int() to use the proper signed or unsigned bytes and fixing tests
    Source code(tar.gz)
    Source code(zip)
  • v0.7.0(Nov 28, 2022)

  • v0.6.2(Oct 21, 2022)

    Changelog

    • Replacing EthBigInt and ethabi::Uint to only use substreams-rs scalar BigInt struct.

    Commits

    • 18cd6d7 Preparing next version
    • 9a3ccb9 Preparing release of v0.6.2
    • 9826661 moving release instructions to RELEASE.md file
    • 6c5f7ee replacing ethbigint with substreams scalar bigint and fixing tests
    • da47959 update substreams crate dependency
    • 0ce9ecf wip bye bye eth big int
    Source code(tar.gz)
    Source code(zip)
  • v0.6.1(Oct 19, 2022)

    Changelog

    • Made Windows target(s) able to run tests when depending on substreams-ethereum crate.

    • Adding allow attribute(s) to remove warnings for code generated by ABI generator (should be good now).

    Commits

    • a416f4a Fixed default compilation target to be wasm32-unknown-unknown and fix it
    • 5f638d8 Made Windows target(s) able to run tests when depending on substreams-ethereum crate.
    • 6dd75a3 Preparing release of 0.6.1
    • d1f9bbf Updated a bit release instructions
    Source code(tar.gz)
    Source code(zip)
  • v0.6.0(Oct 18, 2022)

    Changelog

    • aa4edb8 Preparing release of v0.6.0
    • c77b089 add unused imports to remove warnings
    • 535fdf2 bump substreams crate to 0.3.0
    • 726f4d9 fixing broken url for 0.5.0 release
    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(Oct 12, 2022)

  • v0.4.0(Oct 12, 2022)

    Changelog

    • 3447a92 Preparing release of v0.4.0
    • 14bb0b3 abigen: fix RPCDecodable implementation
    • 60d803a rpc: add RpcBatch and add required code to abigen
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Oct 5, 2022)

    Changelog

    • Breaking change
      • Replaced num_bigint with EthBigInt and BigInt from substreams-rs crate

    Commits

    • e48f469 Adding helper for timestamp_seconds for the block and setting bigint value for ethbigint as private
    • b3ddb52 Preparing next unrelease version
    • 13c5390 Preparing release of v0.3.0
    • 7f4bd0d adding local substreams-rs
    • 9e9cc27 adding scalar big int and big decimal
    • ff169e1 fixing changes to EthBigInt scalar
    Source code(tar.gz)
    Source code(zip)
  • v0.2.2-rc.2(Sep 2, 2022)

    Changelog

    • Fixed issue when type of field was Vec of Uint and the underlying passed in value wasn't cloned. The clone only occurs when it is a Vec and not when the type is only a Uint.

    Commits

    • 5e9c290 Preparing for release v0.2.2-rc.2
    • 53bb215 fixing to_token when recursively calling with array type and adding a clone if the underlying array type is an array of uint
    Source code(tar.gz)
    Source code(zip)
  • v0.2.2-rc.1(Sep 2, 2022)

    Changelog

    • Fixed issue with dependency with api generation. Semver reference for proper way of importing dependencies.

    Commits

    • 95d1dda Preparing next unreleased version
    • f30bec6 Preparing release of v0.2.2-rc.1
    Source code(tar.gz)
    Source code(zip)
  • v0.2.1-rc(Aug 31, 2022)

    Changelog

    • Adding ABI function generation first draft

    Commits

    • 1c90d1d Added first working version of ABI generation for functions
    • 49669eb Preparing next unreleased version
    • e5e6719 Preparing release of v0.2.1-rc
    Source code(tar.gz)
    Source code(zip)
  • v0.2.1(Aug 31, 2022)

    Changelog

    Adding tests for decoding of different value types

    Commits

    • 196ce5c Fix and test FixedBytes size
    • 8563a1f Fix: always 32 bytes length for FixedBytes
    • e9a72f4 Fixed decoding of event's data when dealing with fixed bytes and fixed array
    • e677937 Merge pull request #9 from electricddev/develop
    • d449416 Preparing release of v0.2.1
    • b872721 Updated CHANGELOG with a note about must_decode and README about some release instructions
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Aug 23, 2022)

    Changelog

    • Breaking Replaced substreams-ethereum/pb::eth::v1 to substreams-ethereum/pb::eth::v2 (perform a global replace of any references of substreams-ethereum/pb::eth::v1 to substreams-ethereum/pb::eth::v2 and don't forget to re-generate ABI bindings also which depends on substreams-ethereum/pb::eth::v1).

    • Breaking substreams-ethereum/pb::eth::rpc::RpcCall#method_signature is now named data.

    • Breaking Bumped prost to ^0.11.0 (will requires you to bump substreams = "~0.0.19").

    • Breaking Removed must_decode on ABI generated Event code. Instead, use if let Some(event) = <EventType>::match_and_decode(&log). For example, if you had the following code:

      if !abi::erc721::events::Transfer::match_log(log) {
        return None;
      }
      
      let transfer = abi::erc721::events::Transfer::must_decode(log);
      

      Replace it with

      if let Some(transfer) = abi::erc721::events::Transfer::match_and_decode(log) {
        // Do something
      }
      

    Commits

    • 71cdd7f Preparing for next release
    • 7cf46bc Updated to latest sf.ethereum.type.v2 and to latest substreams
    • 3791ec9 bump crate version to 0.0.18
    • f668e23 bump crate version to 0.1.9
    Source code(tar.gz)
    Source code(zip)
  • v0.1.7(Aug 6, 2022)

  • v0.1.6(Aug 4, 2022)

    Changelog

    • ba1b052 Preparing release of 0.1.6
    • 30e7f4a adding substreams-ethereum-core crate to publish
    • 8154638 adding target32 wasm dependency for build and release
    Source code(tar.gz)
    Source code(zip)
  • v0.1.5(Aug 4, 2022)

    Changelog

    • 46e0eb4 Document pattern for handling multiple event types
    • 12945e0 Introduce block_view helpers
    • 4e28206 Merge branch 'develop' of github.com:streamingfast/substreams-ethereum into feature/num_big_int_type
    • 04979b9 Merge pull request #6 from streamingfast/feature/num_big_int_type
    • 7e62b79 Preparing release of 0.1.5
    • 800627f Updated the readme a little bit
    • 9250d7c abigen: Remove must_decode
    • 764b64f change substreams dependency to develop branch instead of fixed version
    • 547467d fix CI cargo fmt check
    • 96e5f18 fixing failing tests
    • 7f5f1ee matching on int and overloading the decoding which took the int and converted it into a U256 from ethereum. Big int will give us the correct value for the int as we want it to overflow to a negative value
    • f960a0d remove TransactionView
    • 8dd8ddd rework crate structure
    • f26b9fe rework crate structure (#4)
    • 9f77b98 update changelog
    • 31432a8 views: Add events
    Source code(tar.gz)
    Source code(zip)
  • v0.1.4(Jun 22, 2022)

    Changelog

    • d6f57c6 Merge pull request #2 from fubhy/fix/event-overloads
    • e415967 Preparing release of 0.1.4
    • cee69d1 Refactored previous PR
    • 06150af Updated for next version
    • 6df07e7 add test fixture with multiple overloads
    • a11ac98 chore: apply suggestions from Matt
    • 2b90bdd fix event overloads
    Source code(tar.gz)
    Source code(zip)
  • v0.1.3(Jun 20, 2022)

    Changelog

    • 10a7c30 Added GitHub workflow
    • 388f381 Added a test that fails because of the ordering of indexed/unindexed fields
    • 3f6049e Added exact initial failing tests reported as an error
    • 9af5cf9 Added tests to ensure the abigen works properly
    • b51b924 Disable rustfmt check for now due to diff between my local rustfmt and CI's one
    • f8cde79 Fix bytes usage and fix all tests
    • 3006f1f Fixed rustfmt lint errors
    • 2eca9fd Improved generated code for ABI generator for events to Self wherever possible
    • 88aa781 Merge pull request #1 from fubhy/develop
    • dd39a6c Preparing release 0.1.3
    • 0f568a9 Refactored a bit the tests
    • a56f039 fix order of event param names
    Source code(tar.gz)
    Source code(zip)
  • v0.1.2(Jun 3, 2022)

    Changelog

    • 3d98aa4 Bumped substreams, init is now alwais compiled, improvement to doc
    • fbf6c29 Preparing for release 0.1.2
    • bc1df23 bumped to 1.0.1
    • 135aab1 fixing release script
    Source code(tar.gz)
    Source code(zip)
  • v0.1.1(Jun 2, 2022)

  • v0.1.0(May 31, 2022)

    Summary

    • Added codegen API for ABI bindings that can be used instead of the macro for those who prefer that.

    • Added ABI bindings macro use_contract (restricted to Events for now).

    • Added NULL_ADDRESS constant that can be re-used by dependent projects.

    • StreamingFast Firehose Block generated Rust code is now included in this library directly.

    Changelog

    • 118f677 Added EMPTY_ADDRESS const
    • d59598d Added RPC ethereum type
    • 7243fa1 Added codegen feature
    • 08b7ad6 Added initial changelog
    • 55418fa Added macro substreams_ethereum::init that registers required getrandom custom implementation
    • 8577083 Fix bytes implementation in proto file
    • f4a4d55 Fixed all tests
    • 79b4a10 Fixed compilation due to addition of getrandom override
    • 6f1ee19 Fixed compilation when doing the cargo publish
    • e35e39d Have a nicer wrapper directly in the crate.
    • 32d51ff Initial ABI code generation for ABI's events
    • cb56b42 Initial version
    • a440091 Preparing for release
    • 2440345 Renamed EMPTY_ADDRESS to NULL_ADDRESS to match convention
    • 4b223f2 Renamed NOTICE to NOTICE.md and fixed content
    • 6025233 Turned into a workspace to accomodate for future macro incoming
    • 85657b1 Updated documentation of init!
    • bec0ce0 Updated to substreams 0.0.8-beta
    Source code(tar.gz)
    Source code(zip)
Owner
StreamingFast
StreamingFast is a protocol infrastructure company that provides a massively scalable architecture for streaming blockchain data
StreamingFast
Helpers crate to simplify ink! chain extension development

OpenBrush Chain Extension library The library provides tools and primitives to simplify the development of chain extensions for ink! and for the subst

Supercolony 7 Dec 1, 2022
Complete Ethereum and Celo wallet implementation and utilities in Rust

ethers.rs Complete Ethereum and Celo wallet implementation and utilities in Rust Documentation Extensive documentation and examples are available here

Georgios Konstantopoulos 1.5k Jan 8, 2023
Examples and helpers to build NFT contracts on CosmWasm

CosmWasm NFTS This repo is the official repository to work on all NFT standard and examples in the CosmWasm ecosystem. cw721 and cw721-base were moved

CosmWasm 147 Jan 4, 2023
Foundry is a blazing fast, portable and modular toolkit for Ethereum application development written in Rust.

foundry Foundry is a blazing fast, portable and modular toolkit for Ethereum application development written in Rust. Foundry consists of: Forge: Ethe

Georgios Konstantopoulos 5.1k Jan 9, 2023
Ethereum (and Ethereum like) indexer using P2P message to fetch blocks and transactions

Ethereum P2P indexer This project is an indexer for Ethereum and Ethereum forks. It takes advantage of the ETH (Ethereum Wire Protocol) to fetch block

null 5 Nov 10, 2023
Faterium Substrate Node, Runtime, and Pallets. Contains "faterium-polls-pallet" with logic of Faterium Polls.

Faterium Substrate Node Faterium - a place where fates are forged. This is the official implementation of Faterium Crowdfunding Polls in Rust as Subst

Faterium 3 Dec 15, 2022
`llm-chain` is a powerful rust crate for building chains in large language models allowing you to summarise text and complete complex tasks

llm-chain ?? llm-chain is a collection of Rust crates designed to help you work with Large Language Models (LLMs) more effectively. Our primary focus

Sobel IO 36 Apr 6, 2023
Starknet Stack let's you easily create new Cairo Starknet chains with their own sequencers, provers and verifiers

Starknet Stack flowchart LR A("Client") ==>|"Starknet Transactions"| subGraph0["Sequencer"] subGraph0 -.->|"Blocks with txs"| 300319["Watcher prover

Lambdaclass 7 Jul 11, 2023
This monorepository contains the source code for the smart contracts implementing bAsset Protocol on the Terra blockchain.

Crll bAsset Contracts This monorepository contains the source code for the smart contracts implementing bAsset Protocol on the Terra blockchain. You c

null 3 Mar 29, 2024
A template to build smart contracts in Rust to run inside a Cosmos SDK module on all chains that enable it.

CosmWasm Starter Pack This is a template to build smart contracts in Rust to run inside a Cosmos SDK module on all chains that enable it. To understan

null 1 Mar 7, 2022
Smart Contract built in Rust to run inside Cosmos SDK module on all chains that enable it

CoinSwap is a Smart Contract that is built on the terra blockchain and can be used to swap cryptocurrencies such as LUNA, UST, TerraUSD, Anchor, Mirror Protocol, LUNI and other CW20 tokens. The Project also contains a smart contract which works as a analysis tool for the gas fees on the Terra Blockchain.

Prajjwal Chittori 9 Oct 11, 2022
Alternative client for Substrate-based chains.

Lightweight Substrate and Polkadot client. Introduction smoldot is an alternative client of Substrate-based chains, including Polkadot. There exists t

null 14 Feb 16, 2023
A framework for developing EVM-compatible chains

rt-evm A compact development framework for creating EVM-compatible runtimes/chains. Usage Check the example for details. Projects referenced trie, MPT

Rust Util Collections 4 Mar 15, 2023
Web App for exploring Substrate-based Chains using Subxt

Subxt Explorer Check it out at https://paritytech.github.io/subxt-explorer/. An SPA for exploring metadata of substrate based blockchains in the Brows

Parity Technologies 5 Oct 30, 2023
generate peerid from secret_ed25119 for chains made with polkadot-sdk

genpeerid genpeerid is a command-line tool designed to generate a PeerId from an ED25519 secret key, formatted specifically for Polkadot and Substrate

Rotko Networks 3 Apr 3, 2024
VSCode extension to quickly write and customize well tested Solana snippets.

Solana Snippets The Solana Snippets VSCode Extension allows you to quickly insert Solana snippets into your code. This snippets are well tested in a r

patriciobcs 7 Dec 15, 2022
A very bare-bone Block Chain demo that purely implemented by Rust and yew

Intro bloc is one of bare-bone blockchain demo written by Rust, Yew during my learning BlockChain, about years ago. see demo here It just demonstrate

null 4 Dec 16, 2022
Bindings for the etherscan API and other block explorers.

foundry-block-explorers Bindings for the etherscan.io web API and other block explorers. Examples use ethers_core::types::Chain; use foundry_block_exp

Foundry 7 Nov 3, 2023
🚣‍♀️ <1kloc, well-documented Raft consensus algorithm implementation

miniraft A <1kloc, well-documented Raft consensus algorithm implementation This crate is a minimal implementation of the Raft consensus protocol with

Jacky Zhao 25 Dec 1, 2022