In build.rs you should only ever write to OUT_DIR. Currently we are writing to CARGO_HOME. This is annoying because using cache in CI seems to break this for me because that part of CARGO_HOME is not saved.

Probably same treatment needs to be applied to electrum crate too.

In usage of recently added #52 , the expected behaviour for Persistent type datadir is, the new instance of core will inherent all the existing data. Which is working as expected for other data (blocks, chainstate etc), except the wallet directory.

The current behavior tries to always create a new default wallet at initiation. In case of persistent DB that gives rpc error "wallet file already exists".

This PR updates the init process to handle persistent type database properly..

A test to check the behavior is also added.

The intention here is to open possibility to have persistent data directory option for the node along with tempdir option too..

There were many ways of doing this.. I choose this one as it seemed the one with least changes..

Looking for approach acks..

Describe the bug
Cannot use -rpcuser=<user> and -rpcpassword=<password> args with bitcoind, although it's indicated to be deprecated when running bitcoind from the binaries I think it's useful and would be great to maintain compatibility.

The -rpcuser and -rpcpassword does not create a .cookie file, therefore it produces an error when trying to read from the file and get auth parameters.

It enters in an infinite loop because it cannot create and connect a RPC client in step: let client_result = Client::new(&rpc_url, Auth::CookieFile(cookie_file.clone())); when calling auth.get_user_pass it returns an io error because the file does not exists, but the error is not treated and it skip to another pass in the loop considering it's just because the process is not up yet.

It would be better to not consider the .cookie file when receiving -rpcuser and -rpcpassword arguments or raise an error recommending to use -rpcauth, which works because it uses user and password and creates a .cookie file, and I'm using as a workaround.

To Reproduce You can simply try to start a process passing the -rpcauth and -rpcpassword as arguments in conf, it won't event start the process. I've created two tests here: https://github.com/oleonardolima/bitcoind/commit/b522d5ec1c42584b81c3b6595e36b9df0b2517f6

• one for reproducing the error for -rpcuser and -rpcpassword, recommended to run with debug logs: https://github.com/RCasatta/bitcoind/commit/b522d5ec1c42584b81c3b6595e36b9df0b2517f6#diff-b1a35a68f14e696205874893c07fd24fdb88882b47c23cc0e0c80a30c7d53759R621-R643
• one to show that for -rpcauth works successfully: https://github.com/RCasatta/bitcoind/commit/b522d5ec1c42584b81c3b6595e36b9df0b2517f6#diff-b1a35a68f14e696205874893c07fd24fdb88882b47c23cc0e0c80a30c7d53759R645-R668

Expected behavior
I expect that it would work successfully or return an error indicating that it's indicated to use -rpcauth instead.

Build environment

• bitcoind tag/commit: v0.13.0 | bd952c186ffa30e328a7b12c350569856b43c7c1
• OS+version: macOS 12.4
• Rust/Cargo version: rustc 1.60.0 (7737e0b5c 2022-04-04) | cargo 1.60.0 (d1fd9fe2c 2022-03-01)
• Rust/Cargo target: aarch64-apple-darwin

Soon as https://github.com/rust-bitcoin/rust-bitcoincore-rpc/pull/234 merges we can upgrade the bitcoincore-rpc dependency allowing rust-miniscript to upgrade rust-bitcoin.

Description

As discussed in #68 trying to use the -rpcuser and -rpcpassword leads to an infinite loop, because this authentication method does not work alongside cookie authentication leading to an empty file, and the BitcoinD keeps retrying indefinitely.

This PR adds:

• a new Error variant for this RpcUserAndPasswordUse
• a new function to validate the args specified
• two new tests, one for -rpcuser and -rpcpassword usage, which asserts a failure, and another one for the usage of -rpcauth which works fine alongside cookie authentication.

Fixes #68

According to my understanding of cargo docs $CARGO_HOME should be set up by cargo, but it appears it's not always the case -> #13

This uses the suggested home crate to determine cargo_home during the build process, since it was used dirs_next to do the same things in the binary we used home also in the binary so that only one dep is used

A user reported this on one of the new m1 macs:

error[E0277]: the size for values of type `str` cannot be known at compilation time
--> /Users/s/.cargo/registry/src/github.com-1ecc6299db9ec823/bitcoind-0.17.0/build.rs:50:9
|
50 | / println!(
51 | | "filename:{} version:{} hash:{}",
52 | | download_filename, VERSION, expected_hash
53 | | );
| |__________^ doesn't have a size known at compile-time
|
= help: the trait `Sized` is not implemented for `str`
= note: this error originates in the macro `$crate::format_args_nl` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0277]: the size for values of type `str` cannot be known at compilation time
--> /Users/s/.cargo/registry/src/github.com-1ecc6299db9ec823/bitcoind-0.17.0/build.rs:55:19
|
55 | let url = format!(
| ___________________^
56 | | "https://bitcoincore.org/bin/bitcoin-core-{}/{}",
57 | | VERSION, download_filename
58 | | );
| |_________^ doesn't have a size known at compile-time
|
= help: the trait `Sized` is not implemented for `str`
= note: this error originates in the macro `$crate::__export::format_args` (in Nightly builds, run with -Z macro-backtrace for more info)

error: aborting due to 3 previous errors

Some errors have detailed explanations: E0277, E0425.
For more information about an error, try `rustc --explain E0277`.
error: could not compile `bitcoind`

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed

Can anyone confirm or deny?

The binary that this crate downloads from bitcoincore.org doesn't run on an Alpine Docker container with FROM alpine:3.15

The shared library ld-linux-x86-64.so.2 is missing and no compatible version available from the Alpine repo: `/app # /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind --version sh: /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: not found

/app # ldd /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind /lib64/ld-linux-x86-64.so.2 (0x7f53f39aa000) libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7f53f39aa000) librt.so.1 => /lib64/ld-linux-x86-64.so.2 (0x7f53f39aa000) libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f53f39aa000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x7f53f2d2e000) libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f53f39aa000) Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind) Error relocating /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: __strdup: symbol not found Error relocating /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: __rawmemchr: symbol not found Error relocating /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: __snprintf_chk: symbol not found Error relocating /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: __fread_chk: symbol not found Error relocating /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: __strftime_l: symbol not found Error relocating /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: __vsnprintf_chk: symbol not found Error relocating /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: pthread_yield: symbol not found Error relocating /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: malloc_info: symbol not found Error relocating /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: __fprintf_chk: symbol not found Error relocating /app/target/release/build/bitcoind-0886d6cbe0ab5cd4/out/bitcoin/bitcoin-22.0/bin/bitcoind: __memcpy_chk: symbol not found `

Also using the packaged version of bitcoind from the alpine repository doesn't work, because it was compiled without wallet functionality. https://git.alpinelinux.org/aports/tree/community/bitcoin/APKBUILD line 51

I'm not sure what is the best way to make it work with Alpine.

It looks like the recommended way to get the $CARGO_HOME value is from the home crate:

https://crates.io/crates/home

Also @ulrichard ran into an error that could be related (he's on a Debian Bullseye system) and got this error:

When I execute "cargo test" on the current master, I get a compiler error:
Compiling bitcoincore-rpc v0.13.0
error: failed to run custom build command for bitcoind v0.13.0

Caused by:
  process didn't exit successfully: /home/richi/src/github/bdk/target/debug/build/bitcoind-4c60e35c276999bb/build-script-build (exit status: 101)
  --- stderr
  thread 'main' panicked at 'called Result::unwrap() on an Err value: NotPresent', /home/richi/.cargo/registry/src/github.com-1ecc6299db9ec823/bitcoind-0.13.0/build.rs:39:78
  note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed

it was fixed when he manually set his $CARGO_HOME env variable.

For some reason, after I moved the location of my repository which uses bitcoind in its dev-dependencies, downloaded_exe_path() continues returning the old path. Everything from target/... onwards is correct - I've verified that there is indeed a bitcoind binary in that location - but the path to the repo seems to remain constant even across system restarts.

How to reproduce

Platform: macOS Rust version: nightly-2022-06-27-x86_64-apple-darwin bitcoind crate version: 0.26.1

Have this in Cargo.toml:

[dev-dependencies]
bitcoind = { version = "0.26", features = [ "22_0" ] }

Ensure that OUT_DIR is not set in env vars

echo $OUT_DIR

Have this test:

#[cfg(test)]
mod test {
    fn bitcoind_fails() {
        let conf = Conf::default();
        let exe_path = bitcoind::downloaded_exe_path()
            .expect("Didn't specify bitcoind version in feature flags");

        // This prints
        // [node/src/command/test/mod.rs:72] &exe_path = "<path_to_OLD_repo_location>/target/debug/build/bitcoind-65c3b20abafd4893/out/bitcoin/bitcoin-22.0/bin/bitcoind"
        // Aside from the path to the repo, everything else is correct
        dbg!(&exe_path);

        // This causes the code not to compile
        // let out_dir = std::env!("OUT_DIR");
        // This panics
        // let out_dir = std::env::var_os("OUT_DIR").unwrap();

        // This panics when running from new repository location
        let bitcoind = BitcoinD::with_conf(exe_path, &conf)
            .expect("Failed to init bitcoind");
    }
}

Run the test (should pass)

Move the repository, e.g. from $HOME/dev/myrepo to $HOME/dev/bitcoin/myrepo

Rerun the test (should fail)

What I've tried, but which still produce the same error

• cargo clean
• rm -rf <path_to_repo>/target
• rm -rf ~/.cargo/registry
• rm -rf ~/.cargo/git
• cargo cache -e
• Cloning the repo to a new location on the same computer
• Even restarting my computer

Workarounds which prevent the panic (but which don't solve the problem)

• Downgrading to 0_21_0 in Cargo.toml: bitcoind = { version = "0.26", features = [ "0_21_0" ] }
• Specifying bitcoind using a relative path bitcoind = { path = "../../../github/bitcoind", version = "0.26", features = [ "22_0" ] }
• Moving the repo back to its original location

Any idea what could be going on?

If my understanding is correct this crate launches a separate instance per test, which is nice but because of the race condition and tests being run in parallel this can cause flaky tests which is quite bad. It'd be great to fix the race completely.

I took a look at the obvious "what happens if I set -port 0 -rpcport 0" The answer is `bitcoind seems to attempt to bind the default port anyway (WTF?) and doesn't log the actual port so if we do this we have no way of figuring it out.

I see only two solutions that include existing bitcoind instances:

1. On platforms that support it we could LD_PRELOAD a library that overrides bind() to pass in port 0 and then reports the mapping over a pipe. We could read that report and learn the actual port number.
2. Create private network namespace which has a custom IP and is somehow allowed to communicate with the parent. This should be possible on all platforms which support Docker because it presumably does this but it may require root which is not nice.

Of course the long term solution is to add direct support to bitcoind itself.

I'd just like to request that the time = "=0.3.10" hack is removed once bitcoind's MSRV reaches 1.59. I have confirmed that the bitcoind build succeeds with time = "0.3" if compiling with version 1.59.

cargo new --lib repro-build

[package]
name = "repro-build"
version = "0.1.0"
edition = "2021"

[dependencies]
serde_with = "=2.1.0"
bitcoind = "=0.28.1"

$ cargo build

error: failed to select a version for `time`.
    ... required by package `serde_with v2.1.0`
    ... which satisfies dependency `serde_with = "=2.1.0"` of package `repro-build v0.1.0 (/Users/fang/temp/repro-build)`
versions that meet the requirements `~0.3.11` are: 0.3.17, 0.3.16, 0.3.15, 0.3.14, 0.3.13, 0.3.12, 0.3.11

all possible versions conflict with previously selected packages.

  previously selected package `time v0.3.10`
    ... which satisfies dependency `time = "=0.3.10"` of package `bitcoind v0.28.1`
    ... which satisfies dependency `bitcoind = "=0.28.1"` of package `repro-build v0.1.0 (/Users/fang/temp/repro-build)`

failed to select a version for `time` which could resolve this conflict

ureq = "1.0" allows ureq 1.6 to be installed. 1.6 depends on once_cell v1.16.0 which is not msrv 1.41.1. once_cell 1.9.0 is te latest version of at msrv 1.41.1. once_cell 1.10.0 breaks it.

Unfortunately, ureq does not adhere to any MSRV, and only specifies once_cell = "1"

it seems this crate only has 1 ureq call, here https://github.com/RCasatta/bitcoind/blob/2c1758a2d5d638a388906c97bc011802b12e498b/build.rs#L106

so maybe the whole dependency could be replaced with something that maintains an MSRV

This is causing problems downstream. https://github.com/Kixunil/payjoin/pull/36