I'm trying to build quill using the following Nix derivation:

quill = with pkgs; rustPlatform.buildRustPackage rec {
  name = "quill-${version}";
  version = "0.0.0-unknown";

  src = fetchFromGitHub {
    owner = "dfinity";
    repo = "quill";
    rev = "0baa53c175a831c0ad1a8c29b2dc0f437d105530";
    sha256 = "1igggarq4kyln8hzsbpamz5q2hgh5d3bfkm8vfmafh013x4h55bf";
    # date = 2021-06-18T08:16:07+02:00;
  };

  cargoSha256 = "1sv8zqj8a2ykz7c2fbdp39gzcqnz1mq6h9bs4x68mc9cs1fsb9zv";

  nativeBuildInputs = [ pkg-config ];
  buildInputs = [ openssl.dev protobuf ]
    ++ lib.optional stdenv.isDarwin libiconv;
};

However, the Rust build fails with the following error:

error: failed to run custom build command for `ic-registry-transport v0.8.0 (https://github.com/dfinity/ic?rev=779549eccfcf61ac702dfc2ee6d76ffdc2db1f7f#779549ec)`

Caused by:
  process didn't exit successfully: `/private/tmp/nix-build-quill-0.0.0-unknown.drv-0/source/target/release/build/ic-registry-transport-3a9c9d520a265434/build-script-build` (exit code: 101)
  --- stdout
  cargo:rerun-if-changed=proto/ic_registry_transport/pb/v1/transport.proto

  --- stderr
  thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Custom { kind: Other, error: "protoc failed: ../../protobuf/def: warning: directory does not exist.\nmessaging/xnet/v1/mixed_hash_tree.proto: File not found.\nic_registry_transport/pb/v1/transport.proto:30:1: Import \"messaging/xnet/v1/mixed_hash_tree.proto\" was not found or had errors.\nic_registry_transport/pb/v1/transport.proto:216:3: \"messaging.xnet.v1.MixedHashTree\" is not defined.\n" }', /private/tmp/nix-build-quill-0.0.0-unknown.drv-0/quill-0.0.0-unknown-vendor.tar.gz/ic-registry-transport/build.rs:30:10
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Are there any other steps required before beginning what is effectively just a cargo build?

Note that this required patching agent-rs with a custom version that adds the CandidType derivations needed by this library. I've pushed a branch to agent-rs containing these changes so that we can rely on them, but at some point we should update all of these dependencies to keep track with the latest development.

Notes from @Dfinity-Bjoern and @dprats

Background context

• Node providers were given bonus neurons with ICP from the Foundation (not NNS) few months ago
• Node providers have been asking how to get access to unlock them, because they do not know how
• Node providers for help asking has reached fever pitch where Josh asked Bjorn, Luis, Andrew K and me what is a good path forward to start to help them

Proposed solution we gave Josh:

1. Someone from RD make some updates to quill (which @Björn Tackmann can explain). We asked Sam and he said you would be best Candidate. Bjorn believes it’s a 1-2 day kind of simple changes.
2. Bjorn, Kendall, and me QA The quill improvements and write docs for the node providers
3. We demo this work this coming Thursday

Timeline

• This is urgent at the level that we need help outside regular COM, but we would not give anything to node providers until we are satisfied with the docs and experience
• Our goal is to demo something this Thursday to Josh. If it turns out it’s “good enough”, we’ll polish it and tell node providers that we have a plan for them. If it’s not good enough for Josh, we will try again.

Notes

NPs have NitroHSMs, just like the ones we use internally. agent-rs and dfx support those -- using PKCS#11 which is supported by the HSMs. NPs received a bonus neuron, locked up for 4 years, which they can control via the HSM. The problem is: doing this via dfx is somewhere between hard for some and impossible for others.

• it requires users to write/change textual Candid
• the PIN handling via environment variables is nice for pros but not for rookies
• it does not run on Windows.

The core idea of the ask is that quill does what the NPs need (basic neuron handling) with better user experience. The obvious problem: quill does not support the HSMs. But that should not be too hard to add, since agent-rs does all the heavy lifting already. The code needed in (a fork of) quill is more or less:

• get the path for the opensc-pkcs11.{so,dylib} -- that may be different on different platforms
• get some other parameters: slot (almost always 0 if we ask people to remove Yubikeys), key id (we had them all set it to 01), and PIN,
• initialize the Agent object using the PKCS#11 interface instead of loading the key from a file -- just like dfx or the now-defunct icx-nns do it.

Most NPs would probably not use the air-gapped mode, but I could also see a few of them appreciating it. So maybe it makes sense to just have a shorthand for send&sign.

That should be it -- for the first iteration.

Also, one reason I think forking quill instead of just extending it is that I see one possible extension we may want to add: a "backup option". That would be a slightly bigger effort, and it may not be something we want to build into the general quill, since being simple and easy to understand is a great feature for a tool like quill. That's why I feel we may want to fork rather than extend -- but that's just my personal opinion and both seem plausible options.

Regarding the parameters, my intuitive starting point would be having the path of the library, the slot, and the key id in a config file -- they are likely pretty predictable in all cases. (Just if someone installs opensc on a Mac via brew instead of via DMG, they may want to change the path, or different Linux distros may use different paths.) And we could use the typical read-but-do-not-show-on-screen for the PIN input.

Note that the tests are incorrect, I'm not sure what valid arguments to use, and what hoping maybe CI will show me what the expected transaction output should look like.

The neuron-stake command takes a neuron name and converts it to a nonce. It would be natural if the empty string was converted to the nonce value 0. Currently, clap seems to prevent the empty string from being accepted. It would be good to remove that restriction.

I removed the (panic|unwrap|expect)s from the project on bad user inputs and similar errors that are expected to happen and should be reported with a human friendly message. Some of my changes are redundant, e.g. the unwrap on opts that get also validated by validators, but the new code is not much more than unwrapping.

I also added some tests in main.rs that helped me validating my changes to that file.

This functionality has not been released yet for governance, but once it arrives then this PR will add the functionality needed to test it more fully on the testnet, and then later for users to use on mainnet until the NNS Dapp and then the Ledger Nano App are updated.

In all of our literature we talk about "6 months" being the minimum dissolve delay that can accrue voting rewards, except that we use seconds in the quill user interface and don't specify which six months, in which year, are being used to measure those seconds.

We should both document that 15_778_800 seconds is the actual minimum number of seconds required to vote, and also provide a constant string such as SIX_MONTHS that can be passed in the place of a seconds figure, to make it easier for users. Better still would be to add a simple expression parser that handles basic arithmetic, and then make all the same constants we use in Rust available, e.g.:

quill neuron-manage --additional-dissolve-delay-seconds '(8 * MONTHS) + 2 YEARS'

We currently use the crate tiny_hderive for our bip32 key derivations. Unfortunately this crate is GPL-3, rendering it unusable in this Apache-2.0 repo. This PR replaces it with the equivalent bip32 crate. For convenience it also replaces libsecp256k1 with k256, and as that is what agent-rs uses, there is one fewer dependency.

Unfortunately quill was not maintained in the last 6 months and now it points to very old commits of both ic-agent and IC itself (e.g. it still uses the IC code from 13th of May!).

In this PR we:

• remove a couple of dependencies and unify all dependencies to a s single commit on IC repo (from 16th of Sep) and to the latest ic-agent release.
• completely remove the Frankenstein implementation of a SignReplicaV2Transport which was misused for signing of payloads through a control inversion pattern.
• restructure the code by grouping singing code in a single module

This change paves the way to further splitting the offline and online functionality into separate binaries the reducing binary size and the dependency footprint for the offline tool.

There are no FUNCTIONAL changes, that’s why all tests remain untouched modulo a candid dependency related change.

Please carefully go through PR and approve.

Right now we build quill on Linux fully static, which means that the binary cannot load shared libraries. This prevents the HSM variant of quill from being able to load opensc-pkcs11.so.

Either we need to link pkcs11 statically, or release quill on Linux as not a fully-static binary.

Since I'm not quite conversant on how the GitHub action that we build our release binaries with sets up its dependencies, can you assist with this @chmllr?

Description

This changes allows one to submit motion proposal directly using quill, rather than having to involve dfx. Usage:

quill neuron-manage <neuron-id> --proposal-title "<title>" --proposal-summary-file <summary file path>"

How Has This Been Tested?

I will test it tomorrow by submitting a motion proposal. Until then, this is marked draft.

Checklist:

• [ ] I have made corresponding changes to the documentation in src/docs.

Description

This updates the Nix support to more recent versions.

How Has This Been Tested?

I was able to use nix-shell and cargo test again on my local machine.

The subcommand neuron-manage has an option called --clear-manage-neuron-followees. Given that the subcommand is already called neuron-manage that is a repetition. Better names would be --clear-all-followees or -unfollow-all.

When --follow-neurons is the last argument, only followed by <NEURON_ID> of the neuron that is being managed, then the command line parser seems to interpret <NEURON_ID> as just another followee and complains that <NEURON_ID> is missing.

A work-around is to put --follow-topic between the followees and <NEURON_ID>.

When calling the "list_proposals" method of https://bmht6-iiaaa-aaaad-qabeq-cai.raw.ic0.app/principal/rrkah-fqaaa-aaaaa-aaaaq-cai I can see that deadline_timestamp_seconds is returned in the response (see below). This is missing when invoking the same method via quill.

"proposal_info": [ { "id": [ { "id": "43574" } ], "status": 1, "topic": 8, "failure_reason": [], "ballots": [], "proposal_timestamp_seconds": "1644281103", "reward_event_round": "0", "deadline_timestamp_seconds": [ "1644626703" ],