Signed-off-by: Yi [email protected]

Description

relates to PR closes: #469.

This PR switch the runtime to wasm only, instead of NativeElseWasmExecutor. It means in compiled binary, it'll never contain a native runtime. You can get more relative issue on #469, but the brief benifit i think is:

• since the existing of forkless upgrade, only some specific version can be speed up with native binary because it can be only compiled once.
• eliminate the abstruction complexity to support two runtime
• lower binary size and compiling time

Before we can merge this PR, please make sure that all the following items have been checked off. If any of the checklist items are not applicable, please leave them but write a little note why.

• [x] Linked to Github issue with discussion and accepted design OR have an explanation in the PR that describes this work.
• [x] Added one line describing your change in <branch>/CHANGELOG.md
• [x] Re-reviewed Files changed in the Github PR explorer.

Situational Notes:

• If adding functonality, write unit tests!
• If importing a new pallet, choose a proper module index for it, and allow it in BaseFilter. Ensure every extrinsic works from front-end. If there's corresponding tool, ensure both work for each other.
• If needed, update our Javascript/Typescript APIs. These APIs are officially used by exchanges or community developers.
• If modifying existing runtime storage items, make sure to implement storage migrations for the runtime and test them with try-runtime. This includes migrations inherited from upstream changes, and you can search the diffs for modifications of #[pallet::storage] items to check for any.
• If runtime changes, need to update the version numbers properly:
  • authoring_version: The version of the authorship interface. An authoring node will not attempt to author blocks unless this is equal to its native runtime.
  • spec_version: The version of the runtime specification. A full node will not attempt to use its native runtime in substitute for the on-chain Wasm runtime unless all of spec_name, spec_version, and authoring_version are the same between Wasm and native.
  • impl_version: The version of the implementation of the specification. Nodes are free to ignore this; it serves only as an indication that the code is different; as long as the other two versions are the same then while the actual code may be different, it is nonetheless required to do the same thing. Non-consensus-breaking optimizations are about the only changes that could be made which would result in only the impl_version changing.
  • transaction_version: The version of the extrinsics interface. This number must be updated in the following circumstances: extrinsic parameters (number, order, or types) have been changed; extrinsics or pallets have been removed; or the pallet order in the construct_runtime! macro or extrinsic order in a pallet has been changed. You can run the metadata_diff.yml workflow for help. If this number is updated, then the spec_version must also be updated
• Verify benchmarks & weights have been updated for any modified runtime logics

Signed-off-by: Apokalip [email protected]

Workflow/s to check for labels in branches and release branches

closes: #742 relates: #661

Before we can merge this PR, please make sure that all the following items have been checked off. If any of the checklist items are not applicable, please leave them but write a little note why.

• [x] Linked to Github issue with discussion and accepted design OR have an explanation in the PR that describes this work.
• [x] Added one label out of the L- group to this PR
• [x] Added one or more labels out of A- and C- groups to this PR
• [x] Re-reviewed Files changed in the Github PR explorer.

Situational Notes:

• If adding functionality, write unit tests!
• If importing a new pallet, choose a proper module index for it, and allow it in BaseFilter. Ensure every extrinsic works from front-end. If there's corresponding tool, ensure both work for each other.
• If needed, update our Javascript/Typescript APIs. These APIs are officially used by exchanges or community developers.
• If modifying existing runtime storage items, make sure to implement storage migrations for the runtime and test them with try-runtime. This includes migrations inherited from upstream changes, and you can search the diffs for modifications of #[pallet::storage] items to check for any.
• If runtime changes, need to update the version numbers properly:
  • authoring_version: The version of the authorship interface. An authoring node will not attempt to author blocks unless this is equal to its native runtime.
  • spec_version: The version of the runtime specification. A full node will not attempt to use its native runtime in substitute for the on-chain Wasm runtime unless all of spec_name, spec_version, and authoring_version are the same between Wasm and native.
  • impl_version: The version of the implementation of the specification. Nodes are free to ignore this; it serves only as an indication that the code is different; as long as the other two versions are the same then while the actual code may be different, it is nonetheless required to do the same thing. Non-consensus-breaking optimizations are about the only changes that could be made which would result in only the impl_version changing.
  • transaction_version: The version of the extrinsics interface. This number must be updated in the following circumstances: extrinsic parameters (number, order, or types) have been changed; extrinsics or pallets have been removed; or the pallet order in the construct_runtime! macro or extrinsic order in a pallet has been changed. You can run the metadata_diff.yml workflow for help. If this number is updated, then the spec_version must also be updated
• Verify benchmarks & weights have been updated for any modified runtime logics

Signed-off-by: Adam Reif [email protected]

Description

Refactors the publish_draft_release workflow so it does only what it's meant to do - publish a draft release.

Changes in functionality:

1. Only run dolphin integration test if the PR touches the dolphin RT ( A-dolphin on PR )
2. Only run calamari with its long integration test if the PR touches calamari ( A-calamari on PR )
3. Only run docker_image_test with calamari_integration_test
4. only run runtime_upgrade_test on release branches and merge to manta
5. we run everything and the kitchen sink when it's merged to manta

All testing happens in a runtime-specific file and is triggered only when the PR touches the corresponding runtime as defined by the PR label.

I think this saves

• developer sanity maintaining the thing
• aws compute usage
• calamari and dolphin test workflows can be easily diffed against each other

Please discuss and contribute!

Also removes the manta runtime docker build from releases as these images are not currently used anywhere

closes #608

Before we can merge this PR, please make sure that all the following items have been checked off. If any of the checklist items are not applicable, please leave them but write a little note why.

• [x] Linked to Github issue with discussion and accepted design OR have an explanation in the PR that describes this work.
• [x] Added one label out of the L- group to this PR
• [x] Added one or more labels out of A- and C- groups to this PR
• [x] Re-reviewed Files changed in the Github PR explorer.

Situational Notes:

• If adding functionality, write unit tests!
• If importing a new pallet, choose a proper module index for it, and allow it in BaseFilter. Ensure every extrinsic works from front-end. If there's corresponding tool, ensure both work for each other.
• If needed, update our Javascript/Typescript APIs. These APIs are officially used by exchanges or community developers.
• If modifying existing runtime storage items, make sure to implement storage migrations for the runtime and test them with try-runtime. This includes migrations inherited from upstream changes, and you can search the diffs for modifications of #[pallet::storage] items to check for any.
• If runtime changes, need to update the version numbers properly:
  • authoring_version: The version of the authorship interface. An authoring node will not attempt to author blocks unless this is equal to its native runtime.
  • spec_version: The version of the runtime specification. A full node will not attempt to use its native runtime in substitute for the on-chain Wasm runtime unless all of spec_name, spec_version, and authoring_version are the same between Wasm and native.
  • impl_version: The version of the implementation of the specification. Nodes are free to ignore this; it serves only as an indication that the code is different; as long as the other two versions are the same then while the actual code may be different, it is nonetheless required to do the same thing. Non-consensus-breaking optimizations are about the only changes that could be made which would result in only the impl_version changing.
  • transaction_version: The version of the extrinsics interface. This number must be updated in the following circumstances: extrinsic parameters (number, order, or types) have been changed; extrinsics or pallets have been removed; or the pallet order in the construct_runtime! macro or extrinsic order in a pallet has been changed. You can run the metadata_diff.yml workflow for help. If this number is updated, then the spec_version must also be updated
• Verify benchmarks & weights have been updated for any modified runtime logics

Description

closes: #XXXX

Before we can merge this PR, please make sure that all the following items have been checked off. If any of the checklist items are not applicable, please leave them but write a little note why.

• [ ] Targeted PR against correct branch (manta or manta-pc) with right title (start with [Manta] or [Manta-PC]),
• [ ] Linked to Github issue with discussion and accepted design OR have an explanation in the PR that describes this work.
• [ ] Wrote unit tests.
• [ ] Updated relevant documentation in the code.
• [ ] Re-reviewed Files changed in the Github PR explorer.
• [ ] If runtime changes, need to update the version numbers properly:
  • authoring_version: The version of the authorship interface. An authoring node will not attempt to author blocks unless this is equal to its native runtime.
  • spec_version: The version of the runtime specification. A full node will not attempt to use its native runtime in substitute for the on-chain Wasm runtime unless all of spec_name, spec_version, and authoring_version are the same between Wasm and native.
  • impl_version: The version of the implementation of the specification. Nodes are free to ignore this; it serves only as an indication that the code is different; as long as the other two versions are the same then while the actual code may be different, it is nonetheless required to do the same thing. Non-consensus-breaking optimizations are about the only changes that could be made which would result in only the impl_version changing.
  • transaction_version: The version of the extrinsics interface. This number must be updated in the following circumstances: extrinsic parameters (number, order, or types) have been changed; extrinsics or pallets have been removed; or the pallet order in the construct_runtime! macro or extrinsic order in a pallet has been changed. If this number is updated, then the spec_version must also be updated
• [ ] If needed, notify the committer this is a draft-release and a tag is needed after merging the PR.
• [ ] Verify benchmarks & weights have been updated for any modified runtime logics

Description

closes: #196

Polkadot Changes:

• paritytech/polkadot#3763
• https://github.com/paritytech/polkadot/pull/3865
• paritytech/polkadot#3672
• paritytech/polkadot#3676
• paritytech/polkadot#3684
• paritytech/polkadot#3761 Overview PR:
• https://github.com/paritytech/polkadot/issues/3760

Substrate Changes:

• https://github.com/paritytech/substrate/pull/9692
• https://github.com/paritytech/substrate/pull/9715
• https://github.com/paritytech/substrate/pull/9704
• https://github.com/paritytech/substrate/pull/9767
• https://github.com/paritytech/substrate/pull/9845
• No overview PR

Cumulus Changes:

• https://github.com/paritytech/cumulus/pull/561
• https://github.com/paritytech/cumulus/pull/570
• https://github.com/paritytech/cumulus/commit/fa116cfbfb470a8114fc02f9d2e3c0a5ab6c7193
• No overview PR

statemine diffs between:

• fd80849dde5c209c20a996cfcc5aaacd4666dcbe -> 0.9.9
• fa116cfbfb470a8114fc02f9d2e3c0a5ab6c7193 -> 0.9.10

Performed tests with local calamari-local and calamari-testnet networks:

• [x] Does key injection script work (only for calamari-testnet)
• [x] Check block production
• [x] Sudo system.setCode upgrade
• [x] Force transfer balances
• [x] Pass sudo to multi-sig account
• [x] Upgrade with authorizeUpgrade + enactAuthorized
• [x] Check average block time is ~12 seconds
• [x] Check dropping and reconnecting a collator

Before we can merge this PR, please make sure that all the following items have been checked off. If any of the checklist items are not applicable, please leave them but write a little note why.

• [x] Targeted PR against correct branch (manta or manta-pc) with right title (start with [Manta] or [Manta-PC]),
• [x] Linked to Github issue with discussion and accepted design OR have an explanation in the PR that describes this work.
• [ ] Wrote unit tests.
• [x] Updated relevant documentation in the code.
• [x] Re-reviewed Files changed in the Github PR explorer.
• [x] If runtime changes, need to update the version numbers properly:
  • authoring_version: The version of the authorship interface. An authoring node will not attempt to author blocks unless this is equal to its native runtime.
  • spec_version: The version of the runtime specification. A full node will not attempt to use its native runtime in substitute for the on-chain Wasm runtime unless all of spec_name, spec_version, and authoring_version are the same between Wasm and native.
  • impl_version: The version of the implementation of the specification. Nodes are free to ignore this; it serves only as an indication that the code is different; as long as the other two versions are the same then while the actual code may be different, it is nonetheless required to do the same thing. Non-consensus-breaking optimizations are about the only changes that could be made which would result in only the impl_version changing.
  • transaction_version: The version of the extrinsics interface. This number must be updated in the following circumstances: extrinsic parameters (number, order, or types) have been changed; extrinsics or pallets have been removed; or the pallet order in the construct_runtime! macro or extrinsic order in a pallet has been changed. If this number is updated, then the spec_version must also be updated
• [x] If needed, notify the committer this is a draft-release and a tag is needed after merging the PR.

Signed-off-by: Dengjianping [email protected]

Description

closes: #553

Before we can merge this PR, please make sure that all the following items have been checked off. If any of the checklist items are not applicable, please leave them but write a little note why.

• [x] Linked to Github issue with discussion and accepted design OR have an explanation in the PR that describes this work.
• [ ] Wrote unit tests.
• [x] Updated relevant documentation in the code.
• [x] Added one line describing your change in <branch>/CHANGELOG.md
• [x] Re-reviewed Files changed in the Github PR explorer.
• [x] If runtime changes, need to update the version numbers properly:
  • authoring_version: The version of the authorship interface. An authoring node will not attempt to author blocks unless this is equal to its native runtime.
  • spec_version: The version of the runtime specification. A full node will not attempt to use its native runtime in substitute for the on-chain Wasm runtime unless all of spec_name, spec_version, and authoring_version are the same between Wasm and native.
  • impl_version: The version of the implementation of the specification. Nodes are free to ignore this; it serves only as an indication that the code is different; as long as the other two versions are the same then while the actual code may be different, it is nonetheless required to do the same thing. Non-consensus-breaking optimizations are about the only changes that could be made which would result in only the impl_version changing.
  • transaction_version: The version of the extrinsics interface. This number must be updated in the following circumstances: extrinsic parameters (number, order, or types) have been changed; extrinsics or pallets have been removed; or the pallet order in the construct_runtime! macro or extrinsic order in a pallet has been changed. You can run the metadata_diff.yml workflow for help. If this number is updated, then the spec_version must also be updated
• [x] Verify benchmarks & weights have been updated for any modified runtime logics
• [x] If importing a new pallet, choose a proper module index for it, and allow it in BaseFilter. Ensure every extrinsic works from front-end. If there's corresponding tool, ensure both work for each other.
• [x] If needed, update our Javascript/Typescript APIs. These APIs are officially used by exchanges or community developers.
• [x] If modifying existing runtime storage items, make sure to implement storage migrations for the runtime and test them with try-runtime. This includes migrations inherited from upstream changes, and you can search the diffs for modifications of #[pallet::storage] items to check for any.

Description

Closes #357 Port of https://github.com/paritytech/polkadot/pull/4332

Allows building Manta/Calamari runtimes with session and democracy times set to much shorter 2 or 5 minutes instead of their production values and for customizable timings using environment variables. Env variables are ignored unless the RT is built with --features=fast-runtime in which case they overwrite the (shortened) defaults.

Build fast RTs with cargo build --release --features=fast-runtime

or customize, to e.g. 1 minute period interval with env variable MANTA_PERIOD=5 cargo build --release --features=fast-runtime

Variables are named as capitalized versions of their name in the parameter_types! macro prefixed with MANTA_ or CALAMARI_ respectively

Before we can merge this PR, please make sure that all the following items have been checked off. If any of the checklist items are not applicable, please leave them but write a little note why.

• [x] Targeted PR against correct branch (manta or dolphin) with right title (start with [Manta] or [Dolphin]),
• [x] Linked to Github issue with discussion and accepted design OR have an explanation in the PR that describes this work.
• [x] ~Wrote unit tests.~ N/A, buildtime changes only
• [x] ~Updated relevant documentation in the code.~ N/A, buildtime changes only
• [x] Re-reviewed Files changed in the Github PR explorer.
• [x] ~If runtime changes, need to update the version numbers properly~ N/A, buildtime changes only
• [x] ~If needed, notify the committer this is a draft-release and a tag is needed after merging the PR.~ N/A, is feature PR
• [x] ~Verify benchmarks & weights have been updated for any modified runtime logics~ N/A, buildtime changes only
• [x] ~If needed, bump version for every crate.~ N/A, buildtime changes only
• [x] ~If import a new pallet, choose a proper module index for it, and allow it in BaseFilter. Ensure every extrinsic works from front-end. If there's corresponding tool, ensure both work for each other.~ N/A, buildtime changes only
• [x] ~If needed, update our Javascript/Typescript APIs. These APIs are offcially used by exchanges or community developers.~ N/A, buildtime changes only
• [x] ~If we're going to issue a new release, freeze the code one week early(it depends, but usually it's one week), ensure we have enough time for related testing.~ N/A, Is feature PR

Signed-off-by: Dengjianping [email protected]

Description

v0.9.16 -> v0.9.17

Polkadot:

• Optimized weights: https://github.com/paritytech/substrate/pull/10692
• Add a new host function for reporting fatal errors: https://github.com/paritytech/substrate/pull/10741
• Revert connectivity fix 0.9.17: https://github.com/paritytech/polkadot/pull/4943
• Allow two parachains to swap: https://github.com/paritytech/polkadot/pull/4772
• Add proxy type Society for Kusama: https://github.com/paritytech/polkadot/pull/4851

Substrate:

• Optimized weights: https://github.com/paritytech/substrate/pull/10692
• Add a new host function for reporting fatal errors: https://github.com/paritytech/substrate/pull/10741
• Add maxencodelen to implement_per_thing!: https://github.com/paritytech/substrate/pull/10715
• https://github.com/paritytech/substrate/pull/10775
• Upgradable contracts using set_code function: https://github.com/paritytech/substrate/pull/10690

Cumulus:

• Repalce cli crate structopt with clap: https://github.com/paritytech/cumulus/pull/1035
• Introduce rpc client for relay chain full node: https://github.com/paritytech/cumulus/pull/963.
• Fix parachain block time degradation: https://github.com/paritytech/polkadot/pull/4943
• Add the ability to suspend or resume XCM execution on the XCMP queue: https://github.com/paritytech/cumulus/pull/896

v0.9.17 -> v0.9.18

Polkadot:

• Allow two parachains to swap: https://github.com/paritytech/polkadot/pull/4772
• Add proxy type for kappa sigma mu: https://github.com/paritytech/polkadot/pull/4851
• Add disputes pallet to polkadot: https://github.com/paritytech/polkadot/pull/4993
• Revert "collator-protocol: short-term fixes for connectivity: https://github.com/paritytech/polkadot/pull/4914
• Co #10897: storage benchmarking: https://github.com/paritytech/polkadot/pull/4982
• Paritydb support for parachains db: https://github.com/paritytech/polkadot/pull/4838

Substrate:

• Mark sync_state_gensyncspec json-rpc as safe: https://github.com/paritytech/substrate/pull/10832
• Storage benchmarking: https://github.com/paritytech/substrate/pull/10897
• Refactored block body database scheme: https://github.com/paritytech/substrate/pull/10779
• Deprecate "paritydb-experimental" cli in favour or "paritydb: https://github.com/paritytech/substrate/pull/10975
• Upgrading parity-scale-codec to v3: https://github.com/paritytech/substrate/pull/10825
• Bring back maxnominations as a metadata constant: https://github.com/paritytech/substrate/pull/10947
• Fix reentrancy of frozenbalance::died hook: https://github.com/paritytech/substrate/pull/10473

Cumulus:

• Co #5064: Use runtime dependant weights: https://github.com/paritytech/cumulus/pull/1076
• Revert "Co #5091: Runtime dependent weights for parachains: https://github.com/paritytech/cumulus/pull/1076
• Revert "Companion for primitives consolidation into v2: https://github.com/paritytech/cumulus/pull/1071
• collator-selection pallet tidy: https://github.com/paritytech/cumulus/pull/1093
• Add babe randomness to relay state proof: https://github.com/paritytech/cumulus/pull/1083

ORML:

• Return earlier if the too many currencies: https://github.com/open-web3-stack/open-runtime-module-library/pull/714
• Girazoki configurable reserve provider: https://github.com/open-web3-stack/open-runtime-module-library/pull/708
• rm without storage info: https://github.com/open-web3-stack/open-runtime-module-library/pull/717
• Changed the way values are updated: https://github.com/open-web3-stack/open-runtime-module-library/pull/720
• Polkadot v0.9.18: https://github.com/open-web3-stack/open-runtime-module-library/pull/718

Ours:

• Type, ParentIsDefault -> ParentIsPreset
• Cli tool, Replace structopt with clap
• New extrinsic to asset-manager: set_min_xcm_fee.
• codec from 2.0 to 3.0
• scale-info from 1.0 to 2.0
• pallet_collective::EnsureProportionAtLeast<_3, _4, AccountId, CouncilCollective> -> pallet_collective::EnsureProportionAtLeast<AccountId, CouncilCollective, 3, 4>
• Re-benchmark calamari-runtime.
• Improve xcm tests.

closes: #497 closes: #456

Before we can merge this PR, please make sure that all the following items have been checked off. If any of the checklist items are not applicable, please leave them but write a little note why.

• [x] Linked to Github issue with discussion and accepted design OR have an explanation in the PR that describes this work.
• [x] Wrote unit tests.
• [x] Updated relevant documentation in the code.
• [x] Added one line describing your change in <branch>/CHANGELOG.md
• [ ] Re-reviewed Files changed in the Github PR explorer.
• [ ] If runtime changes, need to update the version numbers properly:
  • authoring_version: The version of the authorship interface. An authoring node will not attempt to author blocks unless this is equal to its native runtime.
  • spec_version: The version of the runtime specification. A full node will not attempt to use its native runtime in substitute for the on-chain Wasm runtime unless all of spec_name, spec_version, and authoring_version are the same between Wasm and native.
  • impl_version: The version of the implementation of the specification. Nodes are free to ignore this; it serves only as an indication that the code is different; as long as the other two versions are the same then while the actual code may be different, it is nonetheless required to do the same thing. Non-consensus-breaking optimizations are about the only changes that could be made which would result in only the impl_version changing.
  • transaction_version: The version of the extrinsics interface. This number must be updated in the following circumstances: extrinsic parameters (number, order, or types) have been changed; extrinsics or pallets have been removed; or the pallet order in the construct_runtime! macro or extrinsic order in a pallet has been changed. If this number is updated, then the spec_version must also be updated
• [x] Verify benchmarks & weights have been updated for any modified runtime logics
• [ ] If import a new pallet, choose a proper module index for it, and allow it in BaseFilter. Ensure every extrinsic works from front-end. If there's corresponding tool, ensure both work for each other.
• [ ] If needed, update our Javascript/Typescript APIs. These APIs are offcially used by exchanges or community developers.
• [ ] If modifying existing runtime storage items, make sure to implement storage migrations for the runtime and test them with try-runtime. This includes migrations inhreited from upstream changes, and you can search the diffs for modifications of #[pallet::storage] items to check for any.

Description

This PR adds the pull method for use by wallets to request the diff from the latest state.

Unit Test Spec

Use sudo to set the storage to random UTXO and void numbers: for each round add random number (1 to 2) of random (UTXO, EncryptedNote) pairs and a random number (0 to 2) of random VoidNumbers:

• Shards: Add random (UTXO, EncryptedNote) pairs with a random first key and increasing second key
• InsertionOrderedVoidNumbers: Add random VoidNumbers with increasing key

Checkpoint Description:

• receiver_index: [usize; 256]: Lengths of each first key in Shards
• sender_index: usize: Length of InsertionOrderedVoidNumber

Testing correctness of Normal Usage:

1. Add random elements to ledger
2. Select a random checkpoint that's below the current one
3. Pull the latest state and check that it returns the data items (in order!) up to the checkpoint returned in the diff
4. If should_continue == true then pull should be called again with the returned checkpoint (GOTO step 3)
5. If it passes GOTO step 1

Testing correctness of Abnormal Usage: Same as the previous test except try to get from a checkpoint later than the current one. The pull should return nothing.

TODO

• [x] Implement diffing algorithm
• [x] Add RPC interface
• [ ] Add Unit tests / Simulation testing

Before we can merge this PR, please make sure that all the following items have been checked off. If any of the checklist items are not applicable, please leave them but write a little note why.

• [x] Linked to Github issue with discussion and accepted design OR have an explanation in the PR that describes this work.
• [ ] Wrote unit tests.
• [x] Updated relevant documentation in the code.
• [x] Added one line describing your change in <branch>/CHANGELOG.md
• [x] Re-reviewed Files changed in the Github PR explorer.
• [ ] If runtime changes, need to update the version numbers properly:
  • authoring_version: The version of the authorship interface. An authoring node will not attempt to author blocks unless this is equal to its native runtime.
  • spec_version: The version of the runtime specification. A full node will not attempt to use its native runtime in substitute for the on-chain Wasm runtime unless all of spec_name, spec_version, and authoring_version are the same between Wasm and native.
  • impl_version: The version of the implementation of the specification. Nodes are free to ignore this; it serves only as an indication that the code is different; as long as the other two versions are the same then while the actual code may be different, it is nonetheless required to do the same thing. Non-consensus-breaking optimizations are about the only changes that could be made which would result in only the impl_version changing.
  • transaction_version: The version of the extrinsics interface. This number must be updated in the following circumstances: extrinsic parameters (number, order, or types) have been changed; extrinsics or pallets have been removed; or the pallet order in the construct_runtime! macro or extrinsic order in a pallet has been changed. You can run the metadata_diff.yml workflow for help. If this number is updated, then the spec_version must also be updated
• [x] Verify benchmarks & weights have been updated for any modified runtime logics
• [x] If importing a new pallet, choose a proper module index for it, and allow it in BaseFilter. Ensure every extrinsic works from front-end. If there's corresponding tool, ensure both work for each other.
• [ ] If needed, update our Javascript/Typescript APIs. These APIs are officially used by exchanges or community developers.
• [ ] If modifying existing runtime storage items, make sure to implement storage migrations for the runtime and test them with try-runtime. This includes migrations inherited from upstream changes, and you can search the diffs for modifications of #[pallet::storage] items to check for any.

Description

This PR modifies the manta-collator-selection pallet by adding three new storage items and corresponding setter extrinsics, that - if set to anything other than their default - automatically remove collators at the end of a session that significantly ( x% ) underperformed the y-th percentile of the total collator set over the current session.

A good initial value (and we intend to set this on Dolphin) is

• x = 10 ( default value on genesis: 100)
• y = 80 ( default value on genesis: 0)

Main changes in: pallets/collator-selection/src/lib.rs

• Substitutes the kick_threshold and last_block_authored storage items with two new storage items, defining the percentile and the percentage above
• Adds a setter extrinsic for each of the new items
• Adds a storage map that records the number of blocks produced by each collator in the current session

closes: #335

Before we can merge this PR, please make sure that all the following items have been checked off. If any of the checklist items are not applicable, please leave them but write a little note why.

• [x] Targeted PR against correct branch (manta or dolphin) with right title (start with [Manta] or [Dolphin]),
• [x] Linked to Github issue with discussion and accepted design OR have an explanation in the PR that describes this work.
• [x] Wrote unit tests.
• [x] Updated relevant documentation in the code.
• [ ] Re-reviewed Files changed in the Github PR explorer.
• [ ] If runtime changes, need to update the version numbers properly:
  • authoring_version: The version of the authorship interface. An authoring node will not attempt to author blocks unless this is equal to its native runtime.
  • spec_version: The version of the runtime specification. A full node will not attempt to use its native runtime in substitute for the on-chain Wasm runtime unless all of spec_name, spec_version, and authoring_version are the same between Wasm and native.
  • impl_version: The version of the implementation of the specification. Nodes are free to ignore this; it serves only as an indication that the code is different; as long as the other two versions are the same then while the actual code may be different, it is nonetheless required to do the same thing. Non-consensus-breaking optimizations are about the only changes that could be made which would result in only the impl_version changing.
  • transaction_version: The version of the extrinsics interface. This number must be updated in the following circumstances: extrinsic parameters (number, order, or types) have been changed; extrinsics or pallets have been removed; or the pallet order in the construct_runtime! macro or extrinsic order in a pallet has been changed. If this number is updated, then the spec_version must also be updated
• ~[ ] If needed, notify the committer this is a draft-release and a tag is needed after merging the PR.~
• [ ] Verify benchmarks & weights have been updated for any modified runtime logics
• ~[ ] If needed, bump version for every crate.~
• [x] If import a new pallet, choose a proper module index for it, and allow it in BaseFilter. Ensure every extrinsic works from front-end. If there's corresponding tool, ensure both work for each other.
• ~[ ] If needed, update our Javascript/Typescript APIs. These APIs are offcially used by exchanges or community developers.~
• ~[ ] If we're going to issue a new release, freeze the code one week early(it depends, but usually it's one week), ensure we have enough time for related testing.~

Signed-off-by: Georgi Zlatarev [email protected]

Description

closes: #719 closes: #623 closes: #659

Manta

• Bumps dependencies from v0.9.22 -> v0.9.26
• https://github.com/paritytech/cumulus/pull/1382 + https://github.com/paritytech/substrate/pull/11694 fixes the memory leak issue of v3.2.1
• Switch ORML dependencies to use branch instead of a commit and to use the main fork instead of ours
• For nimbus use our own fork's garandor/v0.9.26 branch as it is not in upstream yet
• For storage deletions use new clear with limit and cursor instead of the deprecated remove_all. Call clear until there's nothing more to clear or the cursor is not progressing https://github.com/paritytech/substrate/pull/11490
• Configure cumulus_pallet_parachain_system's new CheckAssociatedRelayNumber type, which is supposed to find associated relay blocks to a parachain block, with an implementation that strictly increases the relay block between para blocks
• Removed ExportGenesisStateCommand and ExportGenesisWasmCommand as they are now defined in cumulus-client-cli https://github.com/paritytech/cumulus/pull/1325
• Use New rpc api and implementation name https://github.com/paritytech/substrate/pull/11469
• Use AccountIdConversion's renamed into_account to into_account_truncating function to explicitly refer to the fact that it will truncate larger types to fit into a smaller type https://github.com/paritytech/substrate/pull/10719
• For min-xcm-fee we can now return option instead of u128::max for invalid value https://github.com/open-web3-stack/open-runtime-module-library/pull/753
• Use EitherOfDiverse instead of deprecated EnsureOneOf. It can now distinguish between one or the other failing options https://github.com/paritytech/substrate/pull/11558
• Add Event to pallet-transaction-payment https://github.com/paritytech/substrate/pull/11618
• Configure the new SpendOrigin type of the Treasury to always fail. This would be used if we want to allow certain origins to spend configurable amounts directly from the treasury, without the approve procedure https://github.com/paritytech/substrate/pull/11124
• Add placeholder weight for the new spend extrinsic of Treasury until it is properly benchmarked in the new release PR.

Substrate:

• Full diff beween v0.9.22 and v0.9.26 https://github.com/paritytech/substrate/compare/616d33e...e8a7d16
• https://github.com/paritytech/substrate/pull/11347
• https://github.com/paritytech/substrate/pull/11314
• https://github.com/paritytech/substrate/pull/11404
• https://github.com/paritytech/substrate/pull/11428
• https://github.com/paritytech/substrate/pull/11408
• https://github.com/paritytech/substrate/pull/11434
• https://github.com/paritytech/substrate/pull/10719
• https://github.com/paritytech/substrate/pull/11387
• https://github.com/paritytech/substrate/pull/11429
• https://github.com/paritytech/substrate/pull/11441
• https://github.com/paritytech/substrate/pull/11457
• https://github.com/paritytech/substrate/pull/11232
• https://github.com/paritytech/substrate/pull/11357
• https://github.com/paritytech/substrate/pull/11459
• https://github.com/paritytech/substrate/pull/11488
• https://github.com/paritytech/substrate/pull/11469
• https://github.com/paritytech/substrate/pull/11461
• https://github.com/paritytech/substrate/pull/11492
• https://github.com/paritytech/substrate/pull/11487
• https://github.com/paritytech/substrate/pull/11381
• https://github.com/paritytech/substrate/pull/11482
• https://github.com/paritytech/substrate/pull/11454
• https://github.com/paritytech/substrate/pull/11500
• https://github.com/paritytech/substrate/pull/11476
• https://github.com/paritytech/substrate/pull/11431
• https://github.com/paritytech/substrate/pull/11531
• https://github.com/paritytech/substrate/pull/11490
• https://github.com/paritytech/substrate/pull/11124
• https://github.com/paritytech/substrate/pull/11558
• https://github.com/paritytech/substrate/pull/11547
• https://github.com/paritytech/substrate/pull/11548
• https://github.com/paritytech/substrate/pull/11564
• https://github.com/paritytech/substrate/pull/11567
• https://github.com/paritytech/substrate/pull/11582
• https://github.com/paritytech/substrate/pull/11588
• https://github.com/paritytech/substrate/pull/11600
• https://github.com/paritytech/substrate/pull/11593
• https://github.com/paritytech/substrate/pull/11616
• https://github.com/paritytech/substrate/pull/11525
• https://github.com/paritytech/substrate/pull/11112
• https://github.com/paritytech/substrate/pull/11645
• https://github.com/paritytech/substrate/pull/11655
• https://github.com/paritytech/substrate/pull/11426
• https://github.com/paritytech/substrate/pull/11659
• https://github.com/paritytech/substrate/pull/11660
• https://github.com/paritytech/substrate/pull/11302
• https://github.com/paritytech/substrate/pull/11537
• https://github.com/paritytech/substrate/pull/11694
• https://github.com/paritytech/substrate/pull/11719
• https://github.com/paritytech/substrate/pull/11735
• https://github.com/paritytech/substrate/pull/11406
• https://github.com/paritytech/substrate/pull/11094

Polkadot:

• Full diff beween v0.9.22 and v0.9.26 https://github.com/paritytech/polkadot/compare/17c7b95...d878597
• https://github.com/paritytech/polkadot/pull/5600
• https://github.com/paritytech/polkadot/pull/5568
• https://github.com/paritytech/polkadot/pull/5463
• https://github.com/paritytech/polkadot/pull/5337
• https://github.com/paritytech/polkadot/pull/5520
• https://github.com/paritytech/polkadot/pull/5423
• https://github.com/paritytech/polkadot/pull/5314
• https://github.com/paritytech/polkadot/pull/5438
• https://github.com/paritytech/polkadot/pull/5752
• https://github.com/paritytech/polkadot/pull/5744
• https://github.com/paritytech/polkadot/pull/5712
• https://github.com/paritytech/polkadot/pull/5722
• https://github.com/paritytech/polkadot/pull/5605
• https://github.com/paritytech/polkadot/pull/5597

Cumuls:

• Full diff beween v0.9.22 and v0.9.26 https://github.com/paritytech/cumulus/compare/ebdfbea...e43b8b8
• https://github.com/paritytech/cumulus/pull/1426
• https://github.com/paritytech/cumulus/pull/1378
• https://github.com/paritytech/cumulus/pull/1286
• https://github.com/paritytech/cumulus/pull/1286
• https://github.com/paritytech/cumulus/pull/1325
• https://github.com/paritytech/cumulus/pull/1382
• https://github.com/paritytech/cumulus/pull/1332
• https://github.com/paritytech/cumulus/pull/1280
• https://github.com/paritytech/cumulus/pull/1318
• https://github.com/paritytech/cumulus/pull/1306

Before we can merge this PR, please make sure that all the following items have been checked off. If any of the checklist items are not applicable, please leave them but write a little note why.

• [x] Linked to Github issue with discussion and accepted design OR have an explanation in the PR that describes this work.
• [ ] Added one line describing your change in <branch>/CHANGELOG.md
• [x] Re-reviewed Files changed in the Github PR explorer.

Situational Notes:

• If adding functionality, write unit tests!
• If importing a new pallet, choose a proper module index for it, and allow it in BaseFilter. Ensure every extrinsic works from front-end. If there's corresponding tool, ensure both work for each other.
• If needed, update our Javascript/Typescript APIs. These APIs are officially used by exchanges or community developers.
• If modifying existing runtime storage items, make sure to implement storage migrations for the runtime and test them with try-runtime. This includes migrations inherited from upstream changes, and you can search the diffs for modifications of #[pallet::storage] items to check for any.
• If runtime changes, need to update the version numbers properly:
  • authoring_version: The version of the authorship interface. An authoring node will not attempt to author blocks unless this is equal to its native runtime.
  • spec_version: The version of the runtime specification. A full node will not attempt to use its native runtime in substitute for the on-chain Wasm runtime unless all of spec_name, spec_version, and authoring_version are the same between Wasm and native.
  • impl_version: The version of the implementation of the specification. Nodes are free to ignore this; it serves only as an indication that the code is different; as long as the other two versions are the same then while the actual code may be different, it is nonetheless required to do the same thing. Non-consensus-breaking optimizations are about the only changes that could be made which would result in only the impl_version changing.
  • transaction_version: The version of the extrinsics interface. This number must be updated in the following circumstances: extrinsic parameters (number, order, or types) have been changed; extrinsics or pallets have been removed; or the pallet order in the construct_runtime! macro or extrinsic order in a pallet has been changed. You can run the metadata_diff.yml workflow for help. If this number is updated, then the spec_version must also be updated
• Verify benchmarks & weights have been updated for any modified runtime logics

Our testnet is now using calamari RT and Dolphin does not represent the state of the chain due to e.g. missing staking

Can we retire this RT codebase altogether or do we need to keep it around? Benefits of eliminating the code should be obvious, drawbacks not so much.

We'd need to merge dolphin and calamari integration tests

This actually means we can only generate the config after bumping spec versions, otherwise it'll report as manta/4000

Originally posted by @Garandor in https://github.com/Manta-Network/Manta/pull/952#discussion_r1059041475

Description

closes: #852

• Use second job in check_tests.yml for the benchmark test
• Use all threads for unit tests in check_tests.yml
• Use only calamari-local and dolphin-local chain-specs, skipping the -dev chain-specs
• Overall we're back below 2 hours total.

Before we can approve this PR for merge, please make sure that all the following items have been checked off:

• [x] Linked to Github issue with discussion and accepted design OR have an explanation in the PR that describes this work.
• [x] Added one label out of the L- group to this PR
• [x] Added one or more labels from the A- and C- groups to this PR
• [x] Explicitly labelled A-calamari, A-dolphin and/or A-manta if your changes are meant for/impact either of these (CI depends on it)
• [x] This PR is targeted against the current Milestone ( otherwise discuss if it can be added in time)
• [x] Re-reviewed Files changed in the Github PR explorer.

Situational Notes:

• If adding functionality, write unit tests!
• If importing a new pallet, choose a proper module index for it, and allow it in BaseFilter. Ensure every extrinsic works from front-end. If there's corresponding tool, ensure both work for each other.
• If needed, update our Javascript/Typescript APIs. These APIs are officially used by exchanges or community developers.
• If modifying existing runtime storage items, make sure to implement storage migrations for the runtime and test them with try-runtime. This includes migrations inherited from upstream changes, and you can search the diffs for modifications of #[pallet::storage] items to check for any.
• If runtime changes, need to update the version numbers properly:
  • authoring_version: The version of the authorship interface. An authoring node will not attempt to author blocks unless this is equal to its native runtime.
  • spec_version: The version of the runtime specification. A full node will not attempt to use its native runtime in substitute for the on-chain Wasm runtime unless all of spec_name, spec_version, and authoring_version are the same between Wasm and native.
  • impl_version: The version of the implementation of the specification. Nodes are free to ignore this; it serves only as an indication that the code is different; as long as the other two versions are the same then while the actual code may be different, it is nonetheless required to do the same thing. Non-consensus-breaking optimizations are about the only changes that could be made which would result in only the impl_version changing.
  • transaction_version: The version of the extrinsics interface. This number must be updated in the following circumstances: extrinsic parameters (number, order, or types) have been changed; extrinsics or pallets have been removed; or the pallet order in the construct_runtime! macro or extrinsic order in a pallet has been changed. You can run the metadata_diff.yml workflow for help. If this number is updated, then the spec_version must also be updated
• Verify benchmarks & weights have been updated for any modified runtime logics

Signed-off-by: Dengjianping [email protected]

Description

relates to OR closes: #929

Before we can approve this PR for merge, please make sure that all the following items have been checked off:

• [x] Linked to Github issue with discussion and accepted design OR have an explanation in the PR that describes this work.
• [x] Added one label out of the L- group to this PR
• [x] Added one or more labels from the A- and C- groups to this PR
• [x] Explicitly labelled A-calamari, A-dolphin and/or A-manta if your changes are meant for/impact either of these (CI depends on it)
• [x] This PR is targeted against the current Milestone ( otherwise discuss if it can be added in time)
• [ ] Re-reviewed Files changed in the Github PR explorer.

Situational Notes:

• If adding functionality, write unit tests!
• If importing a new pallet, choose a proper module index for it, and allow it in BaseFilter. Ensure every extrinsic works from front-end. If there's corresponding tool, ensure both work for each other.
• If needed, update our Javascript/Typescript APIs. These APIs are officially used by exchanges or community developers.
• If modifying existing runtime storage items, make sure to implement storage migrations for the runtime and test them with try-runtime. This includes migrations inherited from upstream changes, and you can search the diffs for modifications of #[pallet::storage] items to check for any.
• If runtime changes, need to update the version numbers properly:
  • authoring_version: The version of the authorship interface. An authoring node will not attempt to author blocks unless this is equal to its native runtime.
  • spec_version: The version of the runtime specification. A full node will not attempt to use its native runtime in substitute for the on-chain Wasm runtime unless all of spec_name, spec_version, and authoring_version are the same between Wasm and native.
  • impl_version: The version of the implementation of the specification. Nodes are free to ignore this; it serves only as an indication that the code is different; as long as the other two versions are the same then while the actual code may be different, it is nonetheless required to do the same thing. Non-consensus-breaking optimizations are about the only changes that could be made which would result in only the impl_version changing.
  • transaction_version: The version of the extrinsics interface. This number must be updated in the following circumstances: extrinsic parameters (number, order, or types) have been changed; extrinsics or pallets have been removed; or the pallet order in the construct_runtime! macro or extrinsic order in a pallet has been changed. You can run the metadata_diff.yml workflow for help. If this number is updated, then the spec_version must also be updated
• Verify benchmarks & weights have been updated for any modified runtime logics

    OK so if I get this correctly, the extrinsic will simply fail if we encounter an `InternalLedgerError`.

We need a followup topic to this: Trigger tx-pause freeze of chain state on InternalLedgerError

Originally posted by @Garandor in https://github.com/Manta-Network/Manta/pull/949#pullrequestreview-1231219623

Bumps clap from 3.2.23 to 4.0.32.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.