It's not 100% obvious how this should be best implemented - iterating though all recursive child entries would be too expensive for children with many descendant entries.

Can we somehow seek to the next key prefix by simply incrementing the last byte and seeking to the next entry from there? (This may be incorrect for some encodings - we may need some generic way of incrementing keys).

We currently only expose raw queries by key. While this could potentially support every use-case, it's hard to work with since some higher-level queries require sequential access, e.g. reading the value at key A to determine key B. With the current raw queries, a client would have to make multiple round-trips plus both the client and node would need to support querying for a given past height.

It could be a lot easier to instead support higher-level queries at the orga collection level. For example, a query of 3 could be passed to a Deque, in order to query the element at index 3. When resolving the query, a node would need to look up the deque head/tail value, then use that to determine the key for the item at index 3. The resulting proof would contain the key/value entries for both the head/tail value and the item at index 3, so the client can reproduce the deque operation for themselves.

Doing it this way, the implementation does not need to do anything other than call the normal getter function, with a Store implementation which records each access and adds it to the proof.

This could be implemented by adding a general Query trait, where multiplexed state structs just route queries to their children, and collections implement logical query handlers.

Rough Example:

pub trait Query {
  fn query(&self, query: &[u8]) -> Result<()>;
}

...

let store = ProofBuilder::new(some_other_store); // special store wrapper that builds proof as keys are read
let state: SomeState = store.wrap()?; // some state object that implements Query
state.query(user_query)?; // resolve query given arbitrary query bytes from user
let proof = store.proof(); // pull out proof from store wrapper

This PR add traits which standardize calls (triggered by transactions), queries, and clients which can call and query state remotely.

TODO:

• [x] Call trait
• [x] Query trait
• [ ] ~~Client trait~~
• [x] Finalize macro expansion code
• [ ] ~~Client macro~~
• [ ] ~~Doc comments~~

Closes #29

In adding tests to deque, pop_front and pop_back API was changed to return a reference to an immutable, owned value of type T contained in Deque. This makes the interface to the Deque closer to that of the VecDeque in rust. This also changed the remove method of the map, which now returns Result<Option<ReadOnly>>.

This is an experiment and could end up getting reverted, but now I realize that essentially all Store writes can never error since they are just writing to in-memory data structures, and only ever get written to the actual on-disk database in one batch at the end of a block. If we change the Write trait methods to not return Results, we can make types that write their value when dropped and probably do other convenient tricks.

Instead of using the Infallible type, we can just change the Write trait and add a TryWrite for the lower-level database store interfaces (if necessary - they could also do the fallible writing in a flush method since they also just collect data in memory).

We can implement the Read trait for a client which queries Tendermint RPC. This isn't a Store since it's read-only, but is useful for client logic which only needs to read.

This could be a struct called something like TendermintClient which would probably belong in the abci module since that's where all the Tendermint-specific stuff lives.

let client = TendermintClient::new("localhost:26657").unwrap();

Resolves #107 in some capacity. Calling Deque's push_front or push_back calling MapEntry::or_insert() was causing panics for unwrap at None value @ line 836 of Map. This resolves the issue in the context of some usages, but there may be more paths that lead to this same issue. Blocked by #108.

The following code causes unwrap() on None value @ line 836 of orga/collections/map when followed by queue.push_back() equivalent from parent State type holding Deque as complex State type:

        let mut queue = Self::create(store.clone(), data)?;
        queue.deque.pop_back()?;
        let decoded_adapter: HeaderAdapter = Decode::decode(trusted_header.as_slice())?;
        let wrapped_header = WrappedHeader::new(decoded_adapter, trusted_height);
        let work_header = WorkHeader::new(wrapped_header.clone(), wrapped_header.work());

        queue.current_work = wrapped_header.work();
        queue.deque.push_front(work_header.into())?;

        Ok(queue)

This PR moves plugins to a new directory, renames our existing plugin providers, and moves the Context struct and GetContext trait to their own top-level module.

This PR:

• Exposes Tendermint stdout / stderr configuration
• Changes the way we specify a genesis in Node (previously took a path to the genesis, now expects bytes eg. via include_bytes!())
• Puts app data in the user's home directory instead of cwd by default
• Uses a locally-installed Tendermint binary instead of whatever's in the user's path

This is an easy safety fix to prevent a case where a key >= 256 bytes written to the store will cause the entire block to fail when writing to merk.

Will wait to merge this after #79 (will require changing the returned error) in the interest of not keeping that PR open forever.

Closes #61

In the future, we can do a better form of checking at compile-time once ed has some sort of const generic length encoding length fields.

Target for arm64 should be aarch64. This is required in both latest and v2 due to migration imports in nomic.

Nomic arm64 build confirmed to work here https://github.com/agouin/nomic/tree/andrew/aarch64 (incorporates these changes in v2 and v3)

In order to fix nomic aarch64 builds:

• Merge this PR into v2-legacy-main, and merge this same patch into develop #153 and v3-main #155
• Update v2 import in develop to latest v2-legacy-main
• Update orga v2-legacy-main import to latest in nomic branch v2-legacy-main
• Update nomic nomicv2 import in develop branch to latest v2-legacy-main and orga import to latest v3-main

We have a few derive macros which read the source of the calling file to find relevant methods outside of the item the macro is invoked on (Call, Query, and Client). This behavior depends on Span::source_file in proc_macro, which is experimental under the proc_macro_span feature.

Rust Analyzer doesn't support this feature, so the derive macros are unable to find relevant methods in a RA context. This will need a creative solution to inform the macro of the relevant methods, which may involve a build script and writing file path info to disk to be read by the macro, or a macro which writes an attribute containing the relative source path into the original source.