This implementation aims to optimize the task scheduling of the download block.

It contains a simple task counter to allocate the number of tasks for each node, record and filter the relatively good nodes for download.

After about a week of testing and continuous adjustments, the current PR data is relatively satisfactory, but the possibility of continued adjustments in the future is not ruled out

This PR changes a number of things, including but not limited：

1. Raise the maximum inflight block limit per node to 32-128, but the default is 16, and dynamically adjust this data
2. Remove redundant designs where the same block can be requested from two nodes
3. ~When inserting a orphan block, the countdown for 1 second at the tip + 1 corresponding to trace_number, if still not completed, clear the task and send it to another node for download (exponentially decreasing the task limit of the corresponding node)~
4. Split the getBlockTransaction task from the getBlocks task, keeping the design that getBlockTransaction can request from 2 nodes
5. Clearing out nodes that are peer_best_known < tip in IBD time
6. Separating the block fetch process
7. Clearing nodes that do not respond to getblock requests for 30 seconds
8. mark timeout on all < tip +1 block request if request window > tip + 512
9. Reduce the consumption of checking the maximum timeout time, from check all inflight to check all Less than tip + 20

Test machine configuration: 2 core 8G RAM IP Location on Hong Kong

$ cat /proc/cpuinfo | grep name | cut -f2 -d: | uniq -c
2  Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz

before:

| net state | outbound | average speed | CPU occupancy | Bandwidth consumption | - | - | - | - | - | | relatively good | 8 peer | 102-120 block/s | average 70.49%, max 92.77% | average 4 Mbps, max 7.12 Mbps | relatively poor | 8 peer | 93-100 block/s | average 49.99%, max 74.37% | average 4 Mbps, max 7.12 Mbps

after:

| net state | outbound | average speed | CPU occupancy | Bandwidth consumption | - | - | - | - | - | | relatively good | 8 peer | 219-240 block/s | average 78.34%, max 93.17% | average 2Mbps, max 5.52 Mbps | relatively poor | 8 peer | 200-220 block/s | average 70.06%, max 85.83% | average 3 Mbps, max 19.55 Mbps

Need more research on light client protocol, revert the chain_root for now, we can make a hard-fork to add it back after we decide the light client protocol.

revert https://github.com/nervosnetwork/ckb/pull/1518

This PR optimized tx_ask_for related code, replace tx_ask_for_map/tx_ask_for_set with one KeyedPriorityQueue, avoids unnecessary hashset operation in remove_ask_for_tx and pop_ask_for_txs fn.

And the memory layout is also optimized:

1. the vec in tx_ask_for_map is removed.
2. inflight_transactions is removed.
3. tx_ask_for is traced in the global state (renamed to unknown_tx_hashes)

Suggestions for Review

The best way to review this PR is reviewing each commit independently.

Brief introduction for Commits

New hardfork Features

All following features can be found in RFC PRs.

• feat(hardfork): in the "since epoch", the index should be less than length

  Ref: CKB-RFCs PR 223: Ensure that index < length in input since field using epoch

• feat(hardfork): use block timestamp of input cells as relative since start timestamp

  Ref: CKB-RFCs PR 221: Use Block Timestamp as Start Timestamp in Since

• [Reverted] ~~feat(hardfork): allow unknown block versions and transactions versions~~

  ~~Ref: CKB-RFCs PR 230: Allow unknown tx & block version~~

• feat(hardfork): allow script multiple matches on identical data for type hash-type scripts

  Ref: CKB-RFCs PR 222: Allow script multiple matches on identical code

• feat(hardfork): reuse the uncles hash in the header as the extra hash

  Ref: CKB-RFCs PR 224: Add a variable length field in the block header

• [Reverted] ~~feat(hardfork): allow loading uncommitted cell data hashes from tx pool~~

  ~~Ref: CKB-RFCs PR 228: ckb2021: fix load_cell_data_hash syscall~~

Other Important Commits

• feat(hardfork): setup the components for hard fork features

• refactor: let verifiers know the real environment that the transaction is in

  Almost all features require this refactor commit.

• refactor: remove useless parameter "with_data" because it always be true (tricky)

  So I can change less APIs and less code to apply the feature: allow loading uncommitted cell data hashes from tx pool.

About Tests

Almost all features have detailed integration tests (or unit tests):

• Many blocks before hardfork;
• Only one block before hardfork;
• The block at hardfork;
• Many blocks after hardfork.

All commits can passed all integration tests and unit tests.

Previously, BlockAssembler selection has not considered such a situation

This pr rewrite this part, make sure transaction selection including all unconfirmed ancestors.

The Issue

Here is a potential issue which was introduced by #1940.

• Since our rust-toolchain is still be 1.41.0, there will be no errors or warnings when we build the project. (This is why #1940 was merged)

  https://github.com/nervosnetwork/ckb/blob/22829ae85db1643480bd810150f9619250aac92e/rust-toolchain#L1

• But if you update the rust-toolchain to 1.42.0, when you build the whole project (just execute make prod), you will get two warnings:

  warning: ckb/Cargo.toml: Found feature = ... in target.'cfg(...)'.dependencies. This key is not supported for selecting dependencies and will not work as expected. Use the [features] section instead: https://doc.rust-lang.org/cargo/reference/features.html warning: ckb/Cargo.toml: Found feature = ... in target.'cfg(...)'.dependencies. This key is not supported for selecting dependencies and will not work as expected. Use the [features] section instead: https://doc.rust-lang.org/cargo/reference/features.html

~~Although I haven't found any evidences now, I still strongly suspect that it was related to our recent memory leak.~~ After a lot of tests, I think this issue was not related to our recent memory leak. But it's still a bug.

My Solution

I read a lot of issues about cargo in few weeks, I didn't find any official / standard solution to fix it, and I thought they wouldn't fix it in few months. (build.rs can't fix it, too)

So, I use a complex and dirty way to fix it, just review the code, please.

How can I prove my modification was valid?

• Please check this example, I have run it under Debian Buster and Windows 10.

• I have build various kinds of CKB with different features and check a lot logs of them under Debian Buster.

A Very Important Note

If any features of a rust bindings (to c library) crate was changed, please do cargo clean before build the project.

Some C libraries wouldn't re-compile by cargo after features changed, for example, jemalloc-sys.

Before this part, https://github.com/nervosnetwork/ckb/blob/develop/test/src/specs/indexer/basic.rs#L59-L75 code's log is that

2019-10-15 11:49:52.672 +08:00 ChainService INFO ckb-chain  block: 13, hash: Byte32(0x75cddba4ee0b6b15a03be7667c3e44af045282dfc0b151699be84cd4eb275556), total_diff: 0xe00, txs: 1
2019-10-15 11:49:52.673 +08:00 http.worker3 DEBUG ckb-rpc  [block_relay] announce new block 13 Byte32(0x75cddba4ee0b6b15a03be7667c3e44af045282dfc0b151699be84cd4eb275556) 1571111392673
2019-10-15 11:49:52.675 +08:00 GlobalRuntime-1 DEBUG ckb-tx-pool  tx proposed, add to gap Byte32(0x569c112c72a1248a12ece2bd1c1fbdd793d9b4ea698b93c60af21fe45efd9f42)
2019-10-15 11:49:52.676 +08:00 GlobalRuntime-1 DEBUG ckb-tx-pool  tx proposed, add to gap Byte32(0x2ab01fcf0ab6fa29d8ce39f61285002ec69d30d70257aac9e0b216b856642b35)
2019-10-15 11:49:53.614 +08:00 NetworkRuntime-8 DEBUG ckb-sync  no peers connected
2019-10-15 11:49:52.678 +08:00 GlobalRuntime-1 DEBUG ckb-tx-pool  Failed to add tx to gap Byte32(0x2ab01fcf0ab6fa29d8ce39f61285002ec69d30d70257aac9e0b216b856642b35), reason: Error { kind: Unknown([OutPoint(0x932358f16963610f3cd38fda5ed55c85b1aca0c37faeecece79fec1471b1013700000000)])

stack backtrace:
   0: failure::backtrace::internal::InternalBacktrace::new
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/backtrace/internal.rs:44
   1: failure::backtrace::Backtrace::new
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/backtrace/mod.rs:111
   2: <failure::error::error_impl::ErrorImpl as core::convert::From<F>>::from
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/error/error_impl.rs:19
   3: <failure::error::Error as core::convert::From<F>>::from
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/error/mod.rs:36
   4: <T as core::convert::Into<U>>::into
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libcore/convert.rs:544
   5: failure::context::Context<D>::with_err
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/context.rs:105
   6: failure::Fail::context
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/lib.rs:160
   7: ckb_types::core::error::<impl core::convert::From<ckb_types::core::error::OutPointError> for ckb_error::Error>::from
             at util/types/src/core/error.rs:36
   8: <T as core::convert::Into<U>>::into
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libcore/convert.rs:544
   9: ckb_types::core::cell::resolve_transaction
             at /home/u2/nervos/ckb/util/types/src/core/cell.rs:445
  10: ckb_tx_pool::pool::TxPool::resolve_tx_from_pending_and_proposed
             at tx-pool/src/pool.rs:324
  11: ckb_tx_pool::pool::TxPool::gap_tx
             at tx-pool/src/pool.rs:560
  12: ckb_tx_pool::process::chain_reorg::update_tx_pool_for_reorg
             at tx-pool/src/process/chain_reorg.rs:197
  13: <ckb_tx_pool::process::chain_reorg::ChainReorgProcess as futures::future::Future>::poll
             at tx-pool/src/process/chain_reorg.rs:57
  14: futures::future::chain::Chain<A,B,C>::poll
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/chain.rs:42
  15: <futures::future::and_then::AndThen<A,B,F> as futures::future::Future>::poll
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/and_then.rs:32
  16: <futures::future::map::Map<A,F> as futures::future::Future>::poll
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/map.rs:30
  17: <alloc::boxed::Box<F> as futures::future::Future>::poll
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/mod.rs:113
  18: <alloc::boxed::Box<F> as futures::future::Future>::poll
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/mod.rs:113
  19: futures::task_impl::Spawn<T>::poll_future_notify::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:329
  20: futures::task_impl::Spawn<T>::enter::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:399
  21: futures::task_impl::std::set
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/std/mod.rs:83
  22: futures::task_impl::Spawn<T>::enter
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:399
  23: futures::task_impl::Spawn<T>::poll_fn_notify
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:291
  24: futures::task_impl::Spawn<T>::poll_future_notify
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:329
  25: tokio_threadpool::task::Task::run::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/task/mod.rs:145
  26: core::ops::function::FnOnce::call_once
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libcore/ops/function.rs:235
  27: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panic.rs:315
  28: std::panicking::try::do_call
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panicking.rs:296
  29: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:80
  30: std::panicking::try
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panicking.rs:275
  31: std::panic::catch_unwind
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panic.rs:394
  32: tokio_threadpool::task::Task::run
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/task/mod.rs:130
  33: tokio_threadpool::worker::Worker::run_task2
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:567
  34: tokio_threadpool::worker::Worker::run_task
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:459
  35: tokio_threadpool::worker::Worker::try_steal_task
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:416
  36: tokio_threadpool::worker::Worker::try_run_task
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:301
  37: tokio_threadpool::worker::Worker::run
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:241
  38: tokio::runtime::threadpool::builder::Builder::build::{{closure}}::{{closure}}::{{closure}}::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/threadpool/builder.rs:390
  39: tokio_timer::timer::handle::with_default::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-timer-0.2.10/src/timer/handle.rs:94
  40: std::thread::local::LocalKey<T>::try_with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:262
  41: std::thread::local::LocalKey<T>::with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:239
  42: tokio_timer::timer::handle::with_default
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-timer-0.2.10/src/timer/handle.rs:81
  43: tokio::runtime::threadpool::builder::Builder::build::{{closure}}::{{closure}}::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/threadpool/builder.rs:382
  44: tokio_timer::clock::clock::with_default::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-timer-0.2.10/src/clock/clock.rs:141
  45: std::thread::local::LocalKey<T>::try_with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:262
  46: std::thread::local::LocalKey<T>::with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:239
  47: tokio_timer::clock::clock::with_default
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-timer-0.2.10/src/clock/clock.rs:124
  48: tokio::runtime::threadpool::builder::Builder::build::{{closure}}::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/threadpool/builder.rs:381
  49: tokio_reactor::with_default::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-reactor-0.1.9/src/lib.rs:237
  50: std::thread::local::LocalKey<T>::try_with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:262
  51: std::thread::local::LocalKey<T>::with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:239
  52: tokio_reactor::with_default
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-reactor-0.1.9/src/lib.rs:217
  53: tokio::runtime::threadpool::builder::Builder::build::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/threadpool/builder.rs:380
  54: tokio_threadpool::callback::Callback::call
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/callback.rs:22
  55: tokio_threadpool::worker::Worker::do_run::{{closure}}::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:127
  56: tokio_executor::global::with_default::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-executor-0.1.8/src/global.rs:209
  57: std::thread::local::LocalKey<T>::try_with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:262
  58: std::thread::local::LocalKey<T>::with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:239
  59: tokio_executor::global::with_default
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-executor-0.1.8/src/global.rs:178
  60: tokio_threadpool::worker::Worker::do_run::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:125
  61: std::thread::local::LocalKey<T>::try_with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:262
  62: std::thread::local::LocalKey<T>::with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:239
  63: tokio_threadpool::worker::Worker::do_run
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:116
  64: tokio_threadpool::pool::Pool::spawn_thread::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/pool/mod.rs:344
  65: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/sys_common/backtrace.rs:77
  66: std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/mod.rs:470
  67: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panic.rs:315
  68: std::panicking::try::do_call
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panicking.rs:296
  69: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:80
  70: std::panicking::try
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panicking.rs:275
  71: std::panic::catch_unwind
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panic.rs:394
  72: std::thread::Builder::spawn_unchecked::{{closure}}
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/mod.rs:469
  73: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libcore/ops/function.rs:235
  74: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/liballoc/boxed.rs:787
  75: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/liballoc/boxed.rs:787
      std::sys_common::thread::start_thread
             at src/libstd/sys_common/thread.rs:13
      std::sys::unix::thread::Thread::new::thread_start
             at src/libstd/sys/unix/thread.rs:79
  76: start_thread
  77: clone


OutPoint }
2019-10-15 11:49:54.378 +08:00 GlobalRuntime-1 DEBUG ckb-tx-pool  tx proposed, add to gap Byte32(0x932358f16963610f3cd38fda5ed55c85b1aca0c37faeecece79fec1471b10137)
2019-10-15 11:49:54.380 +08:00 GlobalRuntime-1 DEBUG ckb-tx-pool  Failed to add tx to gap Byte32(0x932358f16963610f3cd38fda5ed55c85b1aca0c37faeecece79fec1471b10137), reason: Error { kind: Unknown([OutPoint(0xe8886863772aee87460dbfff3ddbb06adcc4c7286492b49a13bcc82ec667bf7800000000)])

stack backtrace:
   0: failure::backtrace::internal::InternalBacktrace::new
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/backtrace/internal.rs:44
   1: failure::backtrace::Backtrace::new
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/backtrace/mod.rs:111
   2: <failure::error::error_impl::ErrorImpl as core::convert::From<F>>::from
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/error/error_impl.rs:19
   3: <failure::error::Error as core::convert::From<F>>::from
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/error/mod.rs:36
   4: <T as core::convert::Into<U>>::into
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libcore/convert.rs:544
   5: failure::context::Context<D>::with_err
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/context.rs:105
   6: failure::Fail::context
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/lib.rs:160
   7: ckb_types::core::error::<impl core::convert::From<ckb_types::core::error::OutPointError> for ckb_error::Error>::from
             at util/types/src/core/error.rs:36
   8: <T as core::convert::Into<U>>::into
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libcore/convert.rs:544
   9: ckb_types::core::cell::resolve_transaction
             at /home/u2/nervos/ckb/util/types/src/core/cell.rs:445
  10: ckb_tx_pool::pool::TxPool::resolve_tx_from_pending_and_proposed
             at tx-pool/src/pool.rs:324
  11: ckb_tx_pool::pool::TxPool::gap_tx
             at tx-pool/src/pool.rs:560
  12: ckb_tx_pool::process::chain_reorg::update_tx_pool_for_reorg
             at tx-pool/src/process/chain_reorg.rs:197
  13: <ckb_tx_pool::process::chain_reorg::ChainReorgProcess as futures::future::Future>::poll
             at tx-pool/src/process/chain_reorg.rs:57
  14: futures::future::chain::Chain<A,B,C>::poll
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/chain.rs:42
  15: <futures::future::and_then::AndThen<A,B,F> as futures::future::Future>::poll
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/and_then.rs:32
  16: <futures::future::map::Map<A,F> as futures::future::Future>::poll
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/map.rs:30
  17: <alloc::boxed::Box<F> as futures::future::Future>::poll
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/mod.rs:113
  18: <alloc::boxed::Box<F> as futures::future::Future>::poll
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/mod.rs:113
  19: futures::task_impl::Spawn<T>::poll_future_notify::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:329
  20: futures::task_impl::Spawn<T>::enter::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:399
  21: futures::task_impl::std::set
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/std/mod.rs:83
  22: futures::task_impl::Spawn<T>::enter
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:399
  23: futures::task_impl::Spawn<T>::poll_fn_notify
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:291
  24: futures::task_impl::Spawn<T>::poll_future_notify
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:329
  25: tokio_threadpool::task::Task::run::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/task/mod.rs:145
  26: core::ops::function::FnOnce::call_once
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libcore/ops/function.rs:235
  27: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panic.rs:315
  28: std::panicking::try::do_call
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panicking.rs:296
  29: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:80
  30: std::panicking::try
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panicking.rs:275
  31: std::panic::catch_unwind
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panic.rs:394
  32: tokio_threadpool::task::Task::run
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/task/mod.rs:130
  33: tokio_threadpool::worker::Worker::run_task2
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:567
  34: tokio_threadpool::worker::Worker::run_task
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:459
  35: tokio_threadpool::worker::Worker::try_steal_task
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:416
  36: tokio_threadpool::worker::Worker::try_run_task
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:301
  37: tokio_threadpool::worker::Worker::run
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:241
  38: tokio::runtime::threadpool::builder::Builder::build::{{closure}}::{{closure}}::{{closure}}::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/threadpool/builder.rs:390
  39: tokio_timer::timer::handle::with_default::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-timer-0.2.10/src/timer/handle.rs:94
  40: std::thread::local::LocalKey<T>::try_with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:262
  41: std::thread::local::LocalKey<T>::with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:239
  42: tokio_timer::timer::handle::with_default
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-timer-0.2.10/src/timer/handle.rs:81
  43: tokio::runtime::threadpool::builder::Builder::build::{{closure}}::{{closure}}::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/threadpool/builder.rs:382
  44: tokio_timer::clock::clock::with_default::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-timer-0.2.10/src/clock/clock.rs:141
  45: std::thread::local::LocalKey<T>::try_with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:262
  46: std::thread::local::LocalKey<T>::with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:239
  47: tokio_timer::clock::clock::with_default
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-timer-0.2.10/src/clock/clock.rs:124
  48: tokio::runtime::threadpool::builder::Builder::build::{{closure}}::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/threadpool/builder.rs:381
  49: tokio_reactor::with_default::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-reactor-0.1.9/src/lib.rs:237
  50: std::thread::local::LocalKey<T>::try_with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:262
  51: std::thread::local::LocalKey<T>::with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:239
  52: tokio_reactor::with_default
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-reactor-0.1.9/src/lib.rs:217
  53: tokio::runtime::threadpool::builder::Builder::build::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/threadpool/builder.rs:380
  54: tokio_threadpool::callback::Callback::call
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/callback.rs:22
  55: tokio_threadpool::worker::Worker::do_run::{{closure}}::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:127
  56: tokio_executor::global::with_default::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-executor-0.1.8/src/global.rs:209
  57: std::thread::local::LocalKey<T>::try_with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:262
  58: std::thread::local::LocalKey<T>::with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:239
  59: tokio_executor::global::with_default
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-executor-0.1.8/src/global.rs:178
  60: tokio_threadpool::worker::Worker::do_run::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:125
  61: std::thread::local::LocalKey<T>::try_with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:262
  62: std::thread::local::LocalKey<T>::with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:239
  63: tokio_threadpool::worker::Worker::do_run
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:116
  64: tokio_threadpool::pool::Pool::spawn_thread::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/pool/mod.rs:344
  65: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/sys_common/backtrace.rs:77
  66: std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/mod.rs:470
  67: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panic.rs:315
  68: std::panicking::try::do_call
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panicking.rs:296
  69: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:80
  70: std::panicking::try
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panicking.rs:275
  71: std::panic::catch_unwind
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panic.rs:394
  72: std::thread::Builder::spawn_unchecked::{{closure}}
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/mod.rs:469
  73: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libcore/ops/function.rs:235
  74: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/liballoc/boxed.rs:787
  75: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/liballoc/boxed.rs:787
      std::sys_common::thread::start_thread
             at src/libstd/sys_common/thread.rs:13
      std::sys::unix::thread::Thread::new::thread_start
             at src/libstd/sys/unix/thread.rs:79
  76: start_thread
  77: clone


OutPoint }
2019-10-15 11:49:54.388 +08:00 GlobalRuntime-1 DEBUG ckb-tx-pool  tx proposed, add to gap Byte32(0xe8886863772aee87460dbfff3ddbb06adcc4c7286492b49a13bcc82ec667bf78)
2019-10-15 11:49:54.389 +08:00 GlobalRuntime-1 DEBUG ckb-tx-pool  Failed to add tx to gap Byte32(0xe8886863772aee87460dbfff3ddbb06adcc4c7286492b49a13bcc82ec667bf78), reason: Error { kind: Unknown([OutPoint(0xd7e002e771e6e71f2bb82f5dea563c88b87bf0a42855e92b05619186e7b59aaf00000000)])

stack backtrace:
   0: failure::backtrace::internal::InternalBacktrace::new
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/backtrace/internal.rs:44
   1: failure::backtrace::Backtrace::new
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/backtrace/mod.rs:111
   2: <failure::error::error_impl::ErrorImpl as core::convert::From<F>>::from
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/error/error_impl.rs:19
   3: <failure::error::Error as core::convert::From<F>>::from
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/error/mod.rs:36
   4: <T as core::convert::Into<U>>::into
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libcore/convert.rs:544
   5: failure::context::Context<D>::with_err
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/context.rs:105
   6: failure::Fail::context
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/lib.rs:160
   7: ckb_types::core::error::<impl core::convert::From<ckb_types::core::error::OutPointError> for ckb_error::Error>::from
             at util/types/src/core/error.rs:36
   8: <T as core::convert::Into<U>>::into
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libcore/convert.rs:544
   9: ckb_types::core::cell::resolve_transaction
             at /home/u2/nervos/ckb/util/types/src/core/cell.rs:445
  10: ckb_tx_pool::pool::TxPool::resolve_tx_from_pending_and_proposed
             at tx-pool/src/pool.rs:324
  11: ckb_tx_pool::pool::TxPool::gap_tx
             at tx-pool/src/pool.rs:560
  12: ckb_tx_pool::process::chain_reorg::update_tx_pool_for_reorg
             at tx-pool/src/process/chain_reorg.rs:197
  13: <ckb_tx_pool::process::chain_reorg::ChainReorgProcess as futures::future::Future>::poll
             at tx-pool/src/process/chain_reorg.rs:57
  14: futures::future::chain::Chain<A,B,C>::poll
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/chain.rs:42
  15: <futures::future::and_then::AndThen<A,B,F> as futures::future::Future>::poll
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/and_then.rs:32
  16: <futures::future::map::Map<A,F> as futures::future::Future>::poll
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/map.rs:30
  17: <alloc::boxed::Box<F> as futures::future::Future>::poll
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/mod.rs:113
  18: <alloc::boxed::Box<F> as futures::future::Future>::poll
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/mod.rs:113
  19: futures::task_impl::Spawn<T>::poll_future_notify::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:329
  20: futures::task_impl::Spawn<T>::enter::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:399
  21: futures::task_impl::std::set
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/std/mod.rs:83
  22: futures::task_impl::Spawn<T>::enter
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:399
  23: futures::task_impl::Spawn<T>::poll_fn_notify
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:291
  24: futures::task_impl::Spawn<T>::poll_future_notify
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:329
  25: tokio_threadpool::task::Task::run::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/task/mod.rs:145
  26: core::ops::function::FnOnce::call_once
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libcore/ops/function.rs:235
  27: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panic.rs:315
  28: std::panicking::try::do_call
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panicking.rs:296
  29: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:80
  30: std::panicking::try
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panicking.rs:275
  31: std::panic::catch_unwind
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panic.rs:394
  32: tokio_threadpool::task::Task::run
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/task/mod.rs:130
  33: tokio_threadpool::worker::Worker::run_task2
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:567
  34: tokio_threadpool::worker::Worker::run_task
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:459
  35: tokio_threadpool::worker::Worker::try_steal_task
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:416
  36: tokio_threadpool::worker::Worker::try_run_task
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:301
  37: tokio_threadpool::worker::Worker::run
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:241
  38: tokio::runtime::threadpool::builder::Builder::build::{{closure}}::{{closure}}::{{closure}}::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/threadpool/builder.rs:390
  39: tokio_timer::timer::handle::with_default::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-timer-0.2.10/src/timer/handle.rs:94
  40: std::thread::local::LocalKey<T>::try_with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:262
  41: std::thread::local::LocalKey<T>::with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:239
  42: tokio_timer::timer::handle::with_default
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-timer-0.2.10/src/timer/handle.rs:81
  43: tokio::runtime::threadpool::builder::Builder::build::{{closure}}::{{closure}}::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/threadpool/builder.rs:382
  44: tokio_timer::clock::clock::with_default::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-timer-0.2.10/src/clock/clock.rs:141
  45: std::thread::local::LocalKey<T>::try_with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:262
  46: std::thread::local::LocalKey<T>::with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:239
  47: tokio_timer::clock::clock::with_default
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-timer-0.2.10/src/clock/clock.rs:124
  48: tokio::runtime::threadpool::builder::Builder::build::{{closure}}::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/threadpool/builder.rs:381
  49: tokio_reactor::with_default::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-reactor-0.1.9/src/lib.rs:237
  50: std::thread::local::LocalKey<T>::try_with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:262
  51: std::thread::local::LocalKey<T>::with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:239
  52: tokio_reactor::with_default
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-reactor-0.1.9/src/lib.rs:217
  53: tokio::runtime::threadpool::builder::Builder::build::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/threadpool/builder.rs:380
  54: tokio_threadpool::callback::Callback::call
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/callback.rs:22
  55: tokio_threadpool::worker::Worker::do_run::{{closure}}::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:127
  56: tokio_executor::global::with_default::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-executor-0.1.8/src/global.rs:209
  57: std::thread::local::LocalKey<T>::try_with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:262
  58: std::thread::local::LocalKey<T>::with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:239
  59: tokio_executor::global::with_default
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-executor-0.1.8/src/global.rs:178
  60: tokio_threadpool::worker::Worker::do_run::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:125
  61: std::thread::local::LocalKey<T>::try_with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:262
  62: std::thread::local::LocalKey<T>::with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:239
  63: tokio_threadpool::worker::Worker::do_run
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:116
  64: tokio_threadpool::pool::Pool::spawn_thread::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/pool/mod.rs:344
  65: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/sys_common/backtrace.rs:77
  66: std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/mod.rs:470
  67: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panic.rs:315
  68: std::panicking::try::do_call
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panicking.rs:296
  69: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:80
  70: std::panicking::try
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panicking.rs:275
  71: std::panic::catch_unwind
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panic.rs:394
  72: std::thread::Builder::spawn_unchecked::{{closure}}
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/mod.rs:469
  73: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libcore/ops/function.rs:235
  74: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/liballoc/boxed.rs:787
  75: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/liballoc/boxed.rs:787
      std::sys_common::thread::start_thread
             at src/libstd/sys_common/thread.rs:13
      std::sys::unix::thread::Thread::new::thread_start
             at src/libstd/sys/unix/thread.rs:79
  76: start_thread
  77: clone


OutPoint }
2019-10-15 11:49:54.399 +08:00 GlobalRuntime-1 DEBUG ckb-tx-pool  tx proposed, add to gap Byte32(0xd7e002e771e6e71f2bb82f5dea563c88b87bf0a42855e92b05619186e7b59aaf)
2019-10-15 11:49:54.400 +08:00 GlobalRuntime-1 DEBUG ckb-tx-pool  Failed to add tx to gap Byte32(0xd7e002e771e6e71f2bb82f5dea563c88b87bf0a42855e92b05619186e7b59aaf), reason: Error { kind: Unknown([OutPoint(0xcfb64326fb6bd8f244e0370f350786ce932deb97d829f176687f215e487e7f0d00000000)])

stack backtrace:
   0: failure::backtrace::internal::InternalBacktrace::new
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/backtrace/internal.rs:44
   1: failure::backtrace::Backtrace::new
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/backtrace/mod.rs:111
   2: <failure::error::error_impl::ErrorImpl as core::convert::From<F>>::from
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/error/error_impl.rs:19
   3: <failure::error::Error as core::convert::From<F>>::from
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/error/mod.rs:36
   4: <T as core::convert::Into<U>>::into
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libcore/convert.rs:544
   5: failure::context::Context<D>::with_err
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/context.rs:105
   6: failure::Fail::context
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/failure-0.1.5/src/lib.rs:160
   7: ckb_types::core::error::<impl core::convert::From<ckb_types::core::error::OutPointError> for ckb_error::Error>::from
             at util/types/src/core/error.rs:36
   8: <T as core::convert::Into<U>>::into
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libcore/convert.rs:544
   9: ckb_types::core::cell::resolve_transaction
             at /home/u2/nervos/ckb/util/types/src/core/cell.rs:445
  10: ckb_tx_pool::pool::TxPool::resolve_tx_from_pending_and_proposed
             at tx-pool/src/pool.rs:324
  11: ckb_tx_pool::pool::TxPool::gap_tx
             at tx-pool/src/pool.rs:560
  12: ckb_tx_pool::process::chain_reorg::update_tx_pool_for_reorg
             at tx-pool/src/process/chain_reorg.rs:197
  13: <ckb_tx_pool::process::chain_reorg::ChainReorgProcess as futures::future::Future>::poll
             at tx-pool/src/process/chain_reorg.rs:57
  14: futures::future::chain::Chain<A,B,C>::poll
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/chain.rs:42
  15: <futures::future::and_then::AndThen<A,B,F> as futures::future::Future>::poll
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/and_then.rs:32
  16: <futures::future::map::Map<A,F> as futures::future::Future>::poll
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/map.rs:30
  17: <alloc::boxed::Box<F> as futures::future::Future>::poll
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/mod.rs:113
  18: <alloc::boxed::Box<F> as futures::future::Future>::poll
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/mod.rs:113
  19: futures::task_impl::Spawn<T>::poll_future_notify::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:329
  20: futures::task_impl::Spawn<T>::enter::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:399
  21: futures::task_impl::std::set
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/std/mod.rs:83
  22: futures::task_impl::Spawn<T>::enter
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:399
  23: futures::task_impl::Spawn<T>::poll_fn_notify
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:291
  24: futures::task_impl::Spawn<T>::poll_future_notify
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:329
  25: tokio_threadpool::task::Task::run::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/task/mod.rs:145
  26: core::ops::function::FnOnce::call_once
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libcore/ops/function.rs:235
  27: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panic.rs:315
  28: std::panicking::try::do_call
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panicking.rs:296
  29: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:80
  30: std::panicking::try
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panicking.rs:275
  31: std::panic::catch_unwind
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panic.rs:394
  32: tokio_threadpool::task::Task::run
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/task/mod.rs:130
  33: tokio_threadpool::worker::Worker::run_task2
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:567
  34: tokio_threadpool::worker::Worker::run_task
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:459
  35: tokio_threadpool::worker::Worker::try_steal_task
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:416
  36: tokio_threadpool::worker::Worker::try_run_task
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:301
  37: tokio_threadpool::worker::Worker::run
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:241
  38: tokio::runtime::threadpool::builder::Builder::build::{{closure}}::{{closure}}::{{closure}}::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/threadpool/builder.rs:390
  39: tokio_timer::timer::handle::with_default::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-timer-0.2.10/src/timer/handle.rs:94
  40: std::thread::local::LocalKey<T>::try_with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:262
  41: std::thread::local::LocalKey<T>::with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:239
  42: tokio_timer::timer::handle::with_default
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-timer-0.2.10/src/timer/handle.rs:81
  43: tokio::runtime::threadpool::builder::Builder::build::{{closure}}::{{closure}}::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/threadpool/builder.rs:382
  44: tokio_timer::clock::clock::with_default::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-timer-0.2.10/src/clock/clock.rs:141
  45: std::thread::local::LocalKey<T>::try_with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:262
  46: std::thread::local::LocalKey<T>::with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:239
  47: tokio_timer::clock::clock::with_default
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-timer-0.2.10/src/clock/clock.rs:124
  48: tokio::runtime::threadpool::builder::Builder::build::{{closure}}::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/threadpool/builder.rs:381
  49: tokio_reactor::with_default::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-reactor-0.1.9/src/lib.rs:237
  50: std::thread::local::LocalKey<T>::try_with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:262
  51: std::thread::local::LocalKey<T>::with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:239
  52: tokio_reactor::with_default
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-reactor-0.1.9/src/lib.rs:217
  53: tokio::runtime::threadpool::builder::Builder::build::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/threadpool/builder.rs:380
  54: tokio_threadpool::callback::Callback::call
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/callback.rs:22
  55: tokio_threadpool::worker::Worker::do_run::{{closure}}::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:127
  56: tokio_executor::global::with_default::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-executor-0.1.8/src/global.rs:209
  57: std::thread::local::LocalKey<T>::try_with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:262
  58: std::thread::local::LocalKey<T>::with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:239
  59: tokio_executor::global::with_default
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-executor-0.1.8/src/global.rs:178
  60: tokio_threadpool::worker::Worker::do_run::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:125
  61: std::thread::local::LocalKey<T>::try_with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:262
  62: std::thread::local::LocalKey<T>::with
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/local.rs:239
  63: tokio_threadpool::worker::Worker::do_run
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/worker/mod.rs:116
  64: tokio_threadpool::pool::Pool::spawn_thread::{{closure}}
             at /home/u2/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.14/src/pool/mod.rs:344
  65: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/sys_common/backtrace.rs:77
  66: std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/mod.rs:470
  67: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panic.rs:315
  68: std::panicking::try::do_call
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panicking.rs:296
  69: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:80
  70: std::panicking::try
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panicking.rs:275
  71: std::panic::catch_unwind
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/panic.rs:394
  72: std::thread::Builder::spawn_unchecked::{{closure}}
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libstd/thread/mod.rs:469
  73: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/libcore/ops/function.rs:235
  74: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/liballoc/boxed.rs:787
  75: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
             at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54/src/liballoc/boxed.rs:787
      std::sys_common::thread::start_thread
             at src/libstd/sys_common/thread.rs:13
      std::sys::unix::thread::Thread::new::thread_start
             at src/libstd/sys/unix/thread.rs:79
  76: start_thread
  77: clone


OutPoint }
2019-10-15 11:49:54.408 +08:00 GlobalRuntime-1 DEBUG ckb-tx-pool  tx proposed, add to gap Byte32(0xcfb64326fb6bd8f244e0370f350786ce932deb97d829f176687f215e487e7f0d)
2019-10-15 11:49:54.425 +08:00 http.worker3 DEBUG ckb-rpc  [] submit block
2019-10-15 11:49:54.430 +08:00 ChainService INFO ckb-chain  block: 14, hash: Byte32(0x8678fd4c3802753bc6644899f6d6d78248c1011f2efb60ce3e029e6979d2a6e4), total_diff: 0xf00, txs: 1
2019-10-15 11:49:54.431 +08:00 http.worker3 DEBUG ckb-rpc  [block_relay] announce new block 14 Byte32(0x8678fd4c3802753bc6644899f6d6d78248c1011f2efb60ce3e029e6979d2a6e4) 1571111394431
2019-10-15 11:49:54.448 +08:00 GlobalRuntime-2 INFO ckb-tx-pool  [get_block_template] candidate txs count: 6, size: 1452/2157853, cycles:3222/10000000000
2019-10-15 11:49:54.488 +08:00 http.worker3 DEBUG ckb-rpc  [] submit block
2019-10-15 11:49:54.517 +08:00 ChainService INFO ckb-chain  [block_verifier] block number: 15, hash: Byte32(0xe7085b46f48cef40cab4b8eb1e0397546287f01838986e4ce18460324778bf02), size:1974/2158375, cycles: 3222/10000000000
2019-10-15 11:49:54.518 +08:00 ChainService INFO ckb-chain  block: 15, hash: Byte32(0xe7085b46f48cef40cab4b8eb1e0397546287f01838986e4ce18460324778bf02), total_diff: 0x1000, txs: 7
2019-10-15 11:49:54.520 +08:00 http.worker3 DEBUG ckb-rpc  [block_relay] announce new block 15 Byte32(0xe7085b46f48cef40cab4b8eb1e0397546287f01838986e4ce18460324778bf02) 1571111394520

What happened here? There are 6 transactions, tx1 -> tx2 -> tx3 -> tx4 -> tx5 -> tx6, -> means that the before tx is the parent of the after one.

When we get transactions by sorted_keys, the return is that [tx1, tx6, tx5, tx4, tx3, tx2].

At here https://github.com/nervosnetwork/ckb/blob/develop/tx-pool/src/process/chain_reorg.rs#L128, https://github.com/nervosnetwork/ckb/blob/develop/tx-pool/src/process/chain_reorg.rs#L197

These transactions are added to gap, but because some transactions can not be resolved, they are added to orphan. So after https://github.com/nervosnetwork/ckb/blob/develop/test/src/specs/indexer/basic.rs#L62-L70, there are 2 transactions in gap, 4 transactions in orphan.

When proposing stage https://github.com/nervosnetwork/ckb/blob/develop/tx-pool/src/process/chain_reorg.rs#L180, it will check the orphan again.

The whole process is weird and complicated. I think all the transactions should be in gap or we should put the transactions in orphan at firstly.

At firstly I tried to fix it by modifying this part of code https://github.com/nervosnetwork/ckb/blob/develop/tx-pool/src/process/chain_reorg.rs#L145-L162, but after that, I found that https://github.com/nervosnetwork/ckb/blob/develop/tx-pool/src/process/chain_reorg.rs#L128-L138 had the same problem, so I gave up this way, I think the sorted_keys should return the ordered keys by transaction relation.

After this PR, the log is this

2019-10-15 14:06:47.718 +08:00 ChainService INFO ckb-chain  block: 13, hash: Byte32(0x59cdd19f33d015c3b3cf0cd4f92e97712d926f112e9d653c65a090089aa841a1), total_diff: 0xe00, txs: 1
2019-10-15 14:06:47.719 +08:00 http.worker0 DEBUG ckb-rpc  [block_relay] announce new block 13 Byte32(0x59cdd19f33d015c3b3cf0cd4f92e97712d926f112e9d653c65a090089aa841a1) 1571119607719
2019-10-15 14:06:47.720 +08:00 GlobalRuntime-1 DEBUG ckb-tx-pool  tx proposed, add to gap Byte32(0x569c112c72a1248a12ece2bd1c1fbdd793d9b4ea698b93c60af21fe45efd9f42)
2019-10-15 14:06:47.721 +08:00 GlobalRuntime-1 DEBUG ckb-tx-pool  tx proposed, add to gap Byte32(0xcfb64326fb6bd8f244e0370f350786ce932deb97d829f176687f215e487e7f0d)
2019-10-15 14:06:47.722 +08:00 GlobalRuntime-1 DEBUG ckb-tx-pool  tx proposed, add to gap Byte32(0xd7e002e771e6e71f2bb82f5dea563c88b87bf0a42855e92b05619186e7b59aaf)
2019-10-15 14:06:47.723 +08:00 GlobalRuntime-1 DEBUG ckb-tx-pool  tx proposed, add to gap Byte32(0xe8886863772aee87460dbfff3ddbb06adcc4c7286492b49a13bcc82ec667bf78)
2019-10-15 14:06:47.723 +08:00 GlobalRuntime-1 DEBUG ckb-tx-pool  tx proposed, add to gap Byte32(0x932358f16963610f3cd38fda5ed55c85b1aca0c37faeecece79fec1471b10137)
2019-10-15 14:06:47.724 +08:00 GlobalRuntime-1 DEBUG ckb-tx-pool  tx proposed, add to gap Byte32(0x2ab01fcf0ab6fa29d8ce39f61285002ec69d30d70257aac9e0b216b856642b35)
2019-10-15 14:06:47.738 +08:00 http.worker0 DEBUG ckb-rpc  [] submit block
2019-10-15 14:06:47.742 +08:00 ChainService INFO ckb-chain  block: 14, hash: Byte32(0x2c4e70c19979bc532abaf6b145a0305501fae885c9ffe37485e31325f36381a2), total_diff: 0xf00, txs: 1
2019-10-15 14:06:47.742 +08:00 http.worker0 DEBUG ckb-rpc  [block_relay] announce new block 14 Byte32(0x2c4e70c19979bc532abaf6b145a0305501fae885c9ffe37485e31325f36381a2) 1571119607742
2019-10-15 14:06:47.754 +08:00 GlobalRuntime-2 INFO ckb-tx-pool  [get_block_template] candidate txs count: 6, size: 1452/2157853, cycles:3222/10000000000
2019-10-15 14:06:47.769 +08:00 http.worker0 DEBUG ckb-rpc  [] submit block
2019-10-15 14:06:47.781 +08:00 ChainService INFO ckb-chain  [block_verifier] block number: 15, hash: Byte32(0xc73b430bcf9dac68077d860a582e0728fe60fa7ebb732856e6a02eb3a3a285fc), size:1974/2158375, cycles: 3222/10000000000
2019-10-15 14:06:47.782 +08:00 ChainService INFO ckb-chain  block: 15, hash: Byte32(0xc73b430bcf9dac68077d860a582e0728fe60fa7ebb732856e6a02eb3a3a285fc), total_diff: 0x1000, txs: 7
2019-10-15 14:06:47.782 +08:00 http.worker0 DEBUG ckb-rpc  [block_relay] announce new block 15 Byte32(0xc73b430bcf9dac68077d860a582e0728fe60fa7ebb732856e6a02eb3a3a285fc) 1571119607782
2019-10-15 14:06:48.082 +08:00 NetworkRuntime-7 DEBUG ckb-sync  no peers connected

Because of the sorted_keys returned the ordered transactions, the transactions will add to gap instead of orphan.

Dependabot Preview will be shut down on August 3rd, 2021. In order to keep getting Dependabot updates, please merge this PR and migrate to GitHub-native Dependabot before then.

Dependabot has been fully integrated into GitHub, so you no longer have to install and manage a separate app. This pull request migrates your configuration from Dependabot.com to a config file, using the new syntax. When merged, we'll swap out dependabot-preview (me) for a new dependabot app, and you'll be all set!

With this change, you'll now use the Dependabot page in GitHub, rather than the Dependabot dashboard, to monitor your version updates, and you'll configure Dependabot through the new config file rather than a UI.

If you've got any questions or feedback for us, please let us know by creating an issue in the dependabot/dependabot-core repository.

Learn more about migrating to GitHub-native Dependabot

Please note that regular @dependabot commands do not work on this pull request.

proposal changes:

• rename ContextualTransactionVerifier -> TimeRelativeTransactionVerifier
• split NonContextualTransactionVerifier from TransactionVerifier
• check syntactic correctness first before
• refactory tx-pool rejection error
• re-broadcast when duplicated tx submit

• [x] rfc changes: https://github.com/nervosnetwork/rfcs/pull/91
• [x] ckb-system-scripts: https://github.com/nervosnetwork/ckb-system-scripts/pull/2
• [x] mruby-contracts: https://github.com/nervosnetwork/mruby-contracts/pull/7
• [x] ruby sdk: https://github.com/nervosnetwork/ckb-demo-ruby/pull/76
• [x] ruby scripts: https://github.com/nervosnetwork/ckb-ruby-scripts/pull/9
• [x] swift sdk
• [x] java sdk
• [x] js sdk

This is a BREAKING CHANGE.

1. Remove the behavior of joining the peer store when the session is established, and now only operate the peer store after the identify or feeler is successfully established
2. After the fork switch, the non-latest version of the node information will be slowly cleaned up, and no longer added to the peer store
3. Remove unnecessary runtime judgment RefCell

Bumps tokio from 1.23.0 to 1.24.1.

You can trigger a rebase of this PR by commenting @dependabot rebase.

What problem does this PR solve?

Issue Number: close #xxx

Problem Summary:

What is changed and how it works?

Proposal: xxx

What's Changed:

Related changes

• PR to update owner/repo:
• Need to cherry-pick to the release branch

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code ci-runs-only: [ quick_checks,linters ]

Side effects

• Performance regression
• Breaking backward compatibility

Release note

None: Exclude this PR from the release note.
Title Only: Include only the PR title in the release note.
Note: Add a note under the PR title in the release note.

Bumps rhai from 1.10.1 to 1.12.0.

You can trigger a rebase of this PR by commenting @dependabot rebase.

Because of a bug, the version v0.105.x can cause trouble after the light client soft fork activation.

• [ ] Yank all the v0.105.x releases
• [ ] Send CKB alerts to nodes running v0.105.x versions.

Feature Request

Is your feature request related to a problem? Please describe.

Add more bootnodes so new nodes can find better nodes to start discover the p2p network.

What problem does this PR solve?

Issue Number: close #3729

Problem Summary:

This PR want to remove nervosnetwork/faketime, and simply use static atomics and nextest to organize ckb's unit tests.

What is changed and how it works?

The nervosnetwork/faketime is complicated, and when we specified --test-threads=1 flag to cargo test, these thread_local ( https://github.com/nervosnetwork/faketime/blob/1ead0aba139f0893d04aa95c35e0179c3b4482c7/src/faketime.rs#L109-L115 ) behave like global variables, then some tests will fail.

The cargo test run all unit test in a single process, and many unit tests may run on a same thread. And there exists some faketime related test want to change the whole process's faketime by set current process's environment like this: https://github.com/nervosnetwork/ckb/blob/fafa37d979bd83f867358efb7aa176a0ec295bbd/chain/src/tests/block_assembler.rs#L135

this may affect other non-faketime related unit tests.

Fortunately, there is nextest-rs/nextest which will run every unit function on single process, providing process-level isolation granularity.

What's Changed:

Related changes

• remove nervosnetwork/faketime
• add util/systemtime to provide real systemtime, and provide faketime when enable_faketime feature is enabled
• use nextest-rs/nextest to run all unit tests

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code ci-runs-only: [ quick_checks,linters ]

Side effects

• None

Release note

Title Only: Include only the PR title in the release note.