Alternative client for Substrate-based chains.

Last update: Feb 16, 2023

Related tags

Overview

Lightweight Substrate and Polkadot client.

Introduction

smoldot is an alternative client of Substrate-based chains, including Polkadot.

There exists two clients: the full client and the wasm light node. The full client is currently a work in progress and doesn't support many features that the official client supports.

The main development focus is currently around the wasm light node. Using https://github.com/polkadot-js/api/ and https://github.com/paritytech/substrate-connect/ (which uses smoldot as an implementation detail), one can easily connect to a chain and interact in a fully trust-less way with it, from JavaScript.

The Wasm light node is published:

On NPM: https://www.npmjs.com/package/smoldot
On Deno.land/x: https://deno.land/x/smoldot2 (URL to import: https://deno.land/x/smoldot2/index-deno.js)

Objectives

There exists multiple objectives behind this repository:

Write a client implementation that is as comprehensive as possible, to make it easier to understand the various components of a Substrate/Polkadot client. A large emphasis is put on documentation.
Implement a client that is lighter than Substrate, in terms of memory consumption, number of threads, and code size, in order to compile it to WebAssembly and distribute it in web pages.
Experiment with a new code architecture, to maybe upstream some components to Substrate and Polkadot.

Trade-offs

In order to simplify the code, two main design decisions have been made compared to Substrate:

No native runtime. The execution time of the wasmtime library is satisfying enough that having a native runtime isn't critical anymore.
No pluggable architecture. smoldot supports a certain hard coded list of consensus algorithms, at the moment Babe, Aura, and GrandPa. Support for other algorithms can only be added by modifying the code of smoldot, and it is not possible to plug a custom algorithm from outside.

Building manually

Wasm light node

In order to run the wasm light node, you must have installed rustup.

The wasm light node can be tested with cd bin/wasm-node/javascript and npm install; npm start. This will compile the smoldot wasm light node and start a WebSocket server capable of answering JSON-RPC requests. This demo will print a list of URLs that you can navigate to in order to connect to a certain chain. For example you can navigate to https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1%3A9944%2Fwestend in order to interact with the Westend chain.

Note: The npm start command starts a small JavaScript shim, on top of the wasm light node, that hard codes the chain to Westend and starts the WebSocket server. The wasm light node itself can connect to a variety of different chains (not only Westend) and doesn't start any server.

Full client

The full client is a binary similar to the official Polkadot client, and can be tested with cargo run.

Note: The Cargo.toml contains a section [profile.dev] opt-level = 2, and as such cargo run alone should give performances close to the ones in release mode.

The following list is a best-effort list of packages that must be available on the system in order to compile the full node:

clang or gcc
pkg-config
sqlite

Comments

Update to a more recent version of wasmi

Work in progress. Blocked by https://github.com/paritytech/wasmi/issues/658 and https://github.com/paritytech/wasmi/issues/659.

cc https://github.com/smol-dot/smoldot/issues/17

opened by tomaka 22
`state_getKeys` wrongly returns an empty list
I've created this sandbox on stackblitz to reproduce and isolate the issue.

In short, the issue is that smoldot -while connected to the polkadot network- returns an empty list of keys as a response to the following RPC request (which, in this example, asks for the keys of the staking.validators storage entry):

{ "id":1, "jsonrpc":"2.0", "method":"state_getKeys", "params":["0x5f3e4907f716ac89b6347d15ececedca88dcde934c658227ee1dfafcd6e16903"] }

while the exact same RPC request against wss://apps-rpc.polkadot.io/ returns a very long list of keys.
opened by josepot 4
Zero the Wasm VM memory after a reset

This simple change hasn't been done before because it slows down the execution of the full node by a lot. However, correctness is more important than speed.

I've now added a test to make sure that this behavior stays, and we'll optimize the code later.

opened by tomaka 3
Properly reinitialize the instance after a reset in wasmtime

Make sure to recreate the Instance when into_prototype is called. This was the case for wasmi but not for wasmtime.

cc https://github.com/smol-dot/smoldot/issues/17

opened by tomaka 2
Bump terminal_size from 0.2.3 to 0.2.5
Bumps terminal_size from 0.2.3 to 0.2.5.

Release notes

Sourced from terminal_size's releases.

v0.2.5

What's Changed

Update windows-sys requirement from 0.42.0 to 0.45.0 by @dependabot in eminence/terminal-size#49

Full Changelog: https://github.com/eminence/terminal-size/compare/v0.2.3...v0.2.5

(Note v0.2.4 was published and then immediately yanked due to an operator error. So v0.2.5 is the next version after v0.2.3)

Commits

c394316 Version v0.2.5

de53628 Version v0.2.4

74e800a Merge pull request #49 from eminence/dependabot/cargo/windows-sys-0.45.0

2bafefb Update windows-sys requirement from 0.42.0 to 0.45.0

See full diff in compare view

You can trigger a rebase of this PR by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
opened by dependabot[bot] 2
Bump ws from 8.12.0 to 8.12.1 in /wasm-node/javascript
Bumps ws from 8.12.0 to 8.12.1.

Release notes

Sourced from ws's releases.

8.12.1

Bug fixes

Added browser condition to package.json (#2118).

Commits

a04578e [dist] 8.12.1

0d114ef [pkg] Add browser condition (#2118)

2862c2f [doc] Add error handlers to examples and code snippets

See full diff in compare view

You can trigger a rebase of this PR by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
opened by dependabot[bot] 2
Bump @types/node from 18.13.0 to 18.14.0 in /wasm-node/javascript
Bumps @types/node from 18.13.0 to 18.14.0.

Commits

See full diff in compare view

You can trigger a rebase of this PR by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
opened by dependabot[bot] 2
Move things around

This PR is relatively big but straight-forward.

It moves the src directory to lib/src, and all the subdirectories of bin to the root. The chain specs have been moved to a new directory called demo-chain-specs.

I think that this "flat" layout of the various components of the project is better for the comprehension of people who aren't familiar with the source code.

opened by tomaka 2
Add child tries support in host.rs

cc #166

This PR is the first step towards support for child tries in the VM.

While in Substrate child tries support is I believe handled by the "storage layer", in smoldot I prefer to make things clear and explicit. For example, if the Wasm code reads the storage key :child_storage:default:foo, I don't emit a ExternalStorageGet variant but instead a ExternalStorageRoot variant with "foo" as the trie. Similarly, if the Wasm code calls ext_storage_next_key_version_1 with :child_storage:default:foo, I emit a ExternalStorageNextChildTrie variant.

Only default child storages are supported, as this is the only kind of child tries that exist at the moment (and in the foreseeable future).

After this PR, child tries don't work yet. There is a lot of effort remaining to implement support for them especially in runtime_host.rs (which is the code that handles the storage diff while the call is in progress, including things like storage transactions). However, this PR does a big chunk of the work by designing an API for them.

opened by tomaka 2

Bump base64 from 0.13.1 to 0.21.0

Bumps base64 from 0.13.1 to 0.21.0.

Changelog

Sourced from base64's changelog.

0.21.0

Migration

Functions

< 0.20 function 0.21 equivalent

encode() engine::general_purpose::STANDARD.encode() or prelude::BASE64_STANDARD.encode()

encode_config() engine.encode()

encode_config_buf() engine.encode_string()

encode_config_slice() engine.encode_slice()

decode() engine::general_purpose::STANDARD.decode() or prelude::BASE64_STANDARD.decode()

decode_config() engine.decode()

decode_config_buf() engine.decode_vec()

decode_config_slice() engine.decode_slice()

The short-lived 0.20 functions were the 0.13 functions with config replaced with engine.

Padding

If applicable, use the preset engines engine::STANDARD, engine::STANDARD_NO_PAD, engine::URL_SAFE, or engine::URL_SAFE_NO_PAD. The NO_PAD ones require that padding is absent when decoding, and the others require that canonical padding is present .

If you need the < 0.20 behavior that did not care about padding, or want to recreate < 0.20.0's predefined Configs precisely, see the following table.

0.13.1 Config 0.20.0+ alphabet encode_padding decode_padding_mode

STANDARD STANDARD true Indifferent

STANDARD_NO_PAD STANDARD false Indifferent

URL_SAFE URL_SAFE true Indifferent

URL_SAFE_NO_PAD URL_SAFE false Indifferent

0.21.0-rc.1

Restore the ability to decode into a slice of precisely the correct length with Engine.decode_slice_unchecked.

Add Engine as a pub use in prelude.

0.21.0-beta.2

Breaking changes

Re-exports of preconfigured engines in engine are removed in favor of base64::prelude::... that are better suited to those who wish to use the entire path to a name.

0.21.0-beta.1

Breaking changes

< 0.20 function	0.21 equivalent
`encode()`	`engine::general_purpose::STANDARD.encode()` or `prelude::BASE64_STANDARD.encode()`
`encode_config()`	`engine.encode()`
`encode_config_buf()`	`engine.encode_string()`
`encode_config_slice()`	`engine.encode_slice()`
`decode()`	`engine::general_purpose::STANDARD.decode()` or `prelude::BASE64_STANDARD.decode()`
`decode_config()`	`engine.decode()`
`decode_config_buf()`	`engine.decode_vec()`
`decode_config_slice()`	`engine.decode_slice()`

0.13.1 Config	0.20.0+ alphabet	`encode_padding`	`decode_padding_mode`
STANDARD	STANDARD	true	Indifferent
STANDARD_NO_PAD	STANDARD	false	Indifferent
URL_SAFE	URL_SAFE	true	Indifferent
URL_SAFE_NO_PAD	URL_SAFE	false	Indifferent

... (truncated)

Commits

d7fb31c v0.21.0
8350376 Merge pull request #207 from marshallpierce/mp/api-rework
726f784 v0.21.0-rc.1
b29ab01 Add Engine in prelude
64bbcc0 Remove no longer needed test helpers
0f981bd Add decode_slice_unchecked to restore ability to decode into a precisely size...
a51e822 v0.21.0-beta.2
936569a Move re-exports from engine to prelude
53e1091 Fix release notes typo
b03eb5a v0.21.0-beta.1
Additional commits viewable in compare view

You can trigger a rebase of this PR by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR
@dependabot recreate will recreate this PR, overwriting any edits that have been made to it
@dependabot merge will merge this PR after your CI passes on it
@dependabot squash and merge will squash and merge this PR after your CI passes on it
@dependabot cancel merge will cancel a previously requested merge and block automerging
@dependabot reopen will reopen this PR if it is closed
@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

opened by dependabot[bot] 2

RUSTSEC-2022-0061: Crate `parity-wasm` deprecated by the author

Crate parity-wasm deprecated by the author

| Details | | | ------------------- | ---------------------------------------------- | | Status | unmaintained | | Package | parity-wasm | | Version | 0.42.2 | | URL | https://github.com/paritytech/parity-wasm/pull/334 | | Date | 2022-10-01 |

This PR explicitly deprecates parity-wasm. The author recommends switching to wasm-tools.

See advisory page for additional details.

opened by github-actions[bot] 2
Remove usage of the `tracing` crate

Close https://github.com/smol-dot/smoldot/issues/46

As explained in the issue, despite having tried it for a long time I don't actually see the point of this crate. Spans are confusing and quite useless, and there's no traces sink that works well apart from the same ones as log (stdout and writing to files).

opened by tomaka 1
Use the fuel system of wasmi and wasmtime to prevent infinite loops

We should add a guarantee that the VM gets regularly interrupted. How regularly doesn't actually matter that much, as we just want to be sure to not be trapped in an infinite loop.

cc https://github.com/smol-dot/smoldot/issues/110

opened by tomaka 0
Add tests to host.rs

This PR adds a bunch of tests to the host.rs module. The goal is to test whether the core functionalities work as expected and that all host functions work properly.

opened by tomaka 1
Bump rimraf from 3.0.2 to 4.1.2 in /wasm-node/javascript
Bumps rimraf from 3.0.2 to 4.1.2.

Changelog

Sourced from rimraf's changelog.

v4.1

Improved hybrid module with no need to look at the .default dangly bit. .default preserved as a reference to rimraf for compatibility with anyone who came to rely on it in v4.0.

Accept and ignore -rf and -fr arguments to the bin.

v4.0

Remove glob dependency entirely. This library now only accepts actual file and folder names to delete.

Accept array of paths or single path.

Windows performance and reliability improved.

All strategies separated into explicitly exported methods.

Drop support for Node.js below version 14

rewrite in TypeScript

ship CJS/ESM hybrid module

Error on unknown arguments to the bin. (Previously they were silently ignored.)

v3.0

Add --preserve-root option to executable (default true)

Drop support for Node.js below version 6

v2.7

Make glob an optional dependency

2.6

Retry on EBUSY on non-windows platforms as well

Make rimraf.sync 10000% more reliable on Windows

2.5

Handle Windows EPERM when lstat-ing read-only dirs

Add glob option to pass options to glob

2.4

Add EPERM to delay/retry loop

Add disableGlob option

2.3

Make maxBusyTries and emfileWait configurable

Handle weird SunOS unlink-dir issue

Glob the CLI arg for better Windows support

... (truncated)

Commits

df3d085 4.1.2

385f86f typecheck the preserveRoot option properly

dc2fd42 ignore unnecessary -rf or -fr

89b38cf test that optArg throws on bad arguments

1b95def 4.1.1

ad72d73 fix: bundling library using webpack

dac1929 4.1.0

842c6ae Improved hybrid module, no .default needed

b563d8b 4.0.7

a08200d List types separately in each dist type

Additional commits viewable in compare view

You can trigger a rebase of this PR by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
opened by dependabot[bot] 1

Owner

GitHub

Substrate blockchain generated with Substrate Startkit

Substrate Node Template A new FRAME-based Substrate node, ready for hacking ?? Getting Started This project contains some configuration files to help

1 Oct 19, 2021

Substrate blockchain generated with Substrate Startkit

Substrate Node Template A new FRAME-based Substrate node, ready for hacking ?? Getting Started This project contains some configuration files to help

1 Oct 19, 2021

Substrate blockchain generated with Substrate Startkit

Substrate Node Template A new FRAME-based Substrate node, ready for hacking ?? Getting Started This project contains some configuration files to help

0 Nov 6, 2021

A template to build smart contracts in Rust to run inside a Cosmos SDK module on all chains that enable it.

CosmWasm Starter Pack This is a template to build smart contracts in Rust to run inside a Cosmos SDK module on all chains that enable it. To understan

1 Mar 7, 2022

Smart Contract built in Rust to run inside Cosmos SDK module on all chains that enable it

CoinSwap is a Smart Contract that is built on the terra blockchain and can be used to swap cryptocurrencies such as LUNA, UST, TerraUSD, Anchor, Mirror Protocol, LUNI and other CW20 tokens. The Project also contains a smart contract which works as a analysis tool for the gas fees on the Terra Blockchain.

9 Oct 11, 2022

Substreams development kit for Ethereum chains, contains Firehose Block model and helpers as well as utilities for Ethereum ABI encoding/decoding.

Substreams Ethereum Substreams development kit for Ethereum chains, contains Rust Firehose Block model and helpers as well as utilities for Ethereum A

15 Oct 25, 2022

A framework for developing EVM-compatible chains

rt-evm A compact development framework for creating EVM-compatible runtimes/chains. Usage Check the example for details. Projects referenced trie, MPT

4 Mar 15, 2023

`llm-chain` is a powerful rust crate for building chains in large language models allowing you to summarise text and complete complex tasks

llm-chain ?? llm-chain is a collection of Rust crates designed to help you work with Large Language Models (LLMs) more effectively. Our primary focus

36 Apr 6, 2023

Starknet Stack let's you easily create new Cairo Starknet chains with their own sequencers, provers and verifiers

Starknet Stack flowchart LR A("Client") ==>|"Starknet Transactions"| subGraph0["Sequencer"] subGraph0 -.->|"Blocks with txs"| 300319["Watcher prover

7 Jul 11, 2023

generate peerid from secret_ed25119 for chains made with polkadot-sdk

genpeerid genpeerid is a command-line tool designed to generate a PeerId from an ED25519 secret key, formatted specifically for Polkadot and Substrate

3 Apr 3, 2024

A Substrate-based PoA node supporting dynamic addition/removal of authorities.

Substrate PoA A sample Substrate runtime for a PoA blockchain that allows: Dynamically add/remove authorities. Automatically remove authorities when t

10 Jun 16, 2022

Polkadex - An Orderbook-based Decentralized Exchange using the Substrate Blockchain Framework.

What is Polkadex? ?? Polkadex is a Open Source, Decentralized Exchange Platform made using Substrate Blockchain Framework that provides traders with t

243 Dec 16, 2022

xx network Substrate based blockchain node

xx network Substrate based blockchain node Rust Setup First, complete the basic Rust setup instructions. MacOS users: setup to compile for Linux Befor

11 Dec 20, 2022

The Data Highway Substrate-based blockchain node.

DataHighway-Parachain, a parachain on the Polkadot network. Planned features include a decentralized LPWAN roaming hub for LoRaWAN IoT devices and network operator roaming agreements, participative mining, an inter-chain data market, and DAO governance.

11 Dec 2, 2022

A Web3.0 forum implemented based on Substrate

Substrate Node Template A fresh FRAME-based Substrate node, ready for hacking ?? Getting Started Follow the steps below to get started with the Node T

5 Mar 24, 2022

A Web3.0 forum implemented based on Substrate

Substrate Node Template A fresh FRAME-based Substrate node, ready for hacking ?? Getting Started Follow the steps below to get started with the Node T

5 Mar 24, 2022

A rewriting of Duniter based on Substrate framework

Duniter v2s A rewriting of Duniter based on Substrate framework. duniter-v2s is under active development, only a test network called "ĞDev" is deploye

9 Dec 15, 2022

Selendra is a multichains interoperable nominated Proof-of-Stake network for developing and running Substrate-based and EVM compatible blockchain applications.

Selendra An interoperable nominated Proof-of-Stake network for developing and running Substrate-based and EVM compatible blockchain applications. Read

16 Nov 29, 2022

ARYA Network is a polkadot/substrate based chain for Non-fungible Token platform on which we can own sell and buy the NFT's on polkadot network.

ARYA Network ARYA Network is a polkadot/substrate based chain for Non-fungible Token platform on which we can own sell and buy the NFT's on polkadot n

6 Dec 20, 2022

Alternative client for Substrate-based chains.

Related tags

Overview

Introduction

Objectives

Trade-offs

Building manually

Wasm light node

Full client

Comments

v0.2.5

What's Changed

8.12.1

Bug fixes

0.21.0

Migration

Functions

Padding

0.21.0-rc.1

0.21.0-beta.2

Breaking changes

0.21.0-beta.1

Breaking changes

v4.1

v4.0

v3.0

v2.7

2.6

2.5

2.4

2.3

Owner

Substrate blockchain generated with Substrate Startkit

Substrate blockchain generated with Substrate Startkit

Substrate blockchain generated with Substrate Startkit

A template to build smart contracts in Rust to run inside a Cosmos SDK module on all chains that enable it.

Smart Contract built in Rust to run inside Cosmos SDK module on all chains that enable it

Substreams development kit for Ethereum chains, contains Firehose Block model and helpers as well as utilities for Ethereum ABI encoding/decoding.

A framework for developing EVM-compatible chains

`llm-chain` is a powerful rust crate for building chains in large language models allowing you to summarise text and complete complex tasks

Starknet Stack let's you easily create new Cairo Starknet chains with their own sequencers, provers and verifiers

generate peerid from secret_ed25119 for chains made with polkadot-sdk

A Substrate-based PoA node supporting dynamic addition/removal of authorities.

Polkadex - An Orderbook-based Decentralized Exchange using the Substrate Blockchain Framework.

xx network Substrate based blockchain node

The Data Highway Substrate-based blockchain node.

A Web3.0 forum implemented based on Substrate

A Web3.0 forum implemented based on Substrate

A rewriting of Duniter based on Substrate framework

Selendra is a multichains interoperable nominated Proof-of-Stake network for developing and running Substrate-based and EVM compatible blockchain applications.

ARYA Network is a polkadot/substrate based chain for Non-fungible Token platform on which we can own sell and buy the NFT's on polkadot network.