Currently in conductor, soft commits (gossiped) transactions are executed (via DoBlock) and firm commits (from DA) are finalized (FinalizeBlock).

We need support for various edge cases where transactions aren't seen immediately:

• gossip not working, transactions seen first in DA
• missed astria block via gossip (ie block number 10 received but 9 is not)

as title says, if we receive a block from DA that we don't have an execution hash for (ie it was not executed either because it was never received, or there was nothing to execute) we try to execute it, and if there was a block executed, we mark it final. "nothing to execute" case means there were no txs for our rollup namespace in the block.

closes #13

Because our sequencer protobufs don't have a package field the resulting rust modules are also empty, making it necessary to import them with _.rs, which is not nice:

https://github.com/astriaorg/sequencer-relayer/blob/8cd9b9e4268368c48ab3be02d9fa85b46914a32e/sequencer-relayer-proto/src/lib.rs#LL1C1

We should give them a proper namespace.

This PR implements github workflows for running tests, formatting, and linting.

Because astria-conductor needs the astria-sequencer-relayer to be running but we want to avoid building container images as part of the tests, this PR circumvents the issue by:

1. building astria-sequencer-relayer as a binary before the tests are run;
2. copying the binary to /tmp/astria/;
3. mounting /tmp/astria/astria-sequencer-relayer into a barebones debian:bookworm-slim container as a hostPath.

This strategy can be repeated for future test requirements.

Note that debian:bookworm-slim (glibc 2.36) is preferred over debian:buster-slim (2.28) because of compatibility with ubuntu-22.04 (glibc 2.35).

This PR also moves to rustls everywhere to avoid linking to openssl.

Issues that this PR will left unaddressed:

• gossipnet tests stall indefinitely and never finish on CI (but work locally): #24
• rs-cnc needs celestia running for its roundtrip test to work (needs to be updated to folow astria-conductor and astria-sequencer-relayer): #25
• astria-conductor and astria-sequencer-relayer have a ton of duplicate code. This needs to be refactored into a common testing crate: #26

this steps just moves everything out of sources/ and places it into crates/. cargo check passes after adding making the crates workspace members in the root Cargo.toml.

The crates do not refer to each other by path, and they are also not yet cleaned up.

This commit replaces log and flexilogger by tracing and tracing-subscriber. The default log level is info, which can be overriden by using the RUST_LOG environment variable with directives as described in the tracing subscriber docs: https://docs.rs/tracing-subscriber/0.3.17/tracing_subscriber/filter/struct.EnvFilter.html

Because env var directives are strictly more powerful than setting verbosity levels on the command line the log-level flag is removed.

If astria-conductor is connected to a tty it will emit tracing events using human-readable format. If it is not connected to a tty it will emit JSON.

By default logging is disabled in integration tests. It can be enabled by setting the environment variable TEST_LOG=1.

Note that the service no longer writes explicitly writes logs to a file. Because astria-conductor is intended to be run as a service by a service manager or orchestration platform, it is assumed that persisting logs is handled externally.

Partially addresses #14

to ensure that the sequencer wrote the correct transactions to Celestia, we need to validate on reading that the transactions written hash to the correct hash in the header (DataHash)

can just call this function: https://github.com/astriaorg/sequencer-relayer/blob/master/sequencer-relayer/src/sequencer_block.rs#L198

• Adds workflow to run buf lint, and push build artifacts to buf.build/astria/astria (non main pushes are drafts)
• Combines astria-execution-apis-rpc and astria-sequencer-relayer-proto into astria-proto
• Makes all packages prefixed with astria following proto standards to allow cleaner imports. (ie astria.execution.v1)
• Adds a protobuf package name to astria-sequencer-relayer-proto

Fixes https://github.com/astriaorg/astria/issues/10

Builds and pushes images for astria-conductor and astria-sequencer-relayer.

Builds are done when any of following occur

• pull requests where there are changes in corresponding folders, or to the workflow file.
• when tagged releases of format v{major}.{minor}.{patch}--{component-name} are created

Fixes #42

In crates/astria-proto/proto/astria/execution/v1/execution.proto, there is a the gRPC method DoBlockRequest has a field called prev_state_root but what's actually passed here is the parent block hash, and should be called prev_block_hash.

Therefore, field prev_state_root is renamed to prev_block_hash in DoBlockRequest.

as title says, this is because the Reader doesn't contain the namespace the conductor cares about, but the Executor does, and checks whether the block should be executed. this should be changed so we aren't validating irrelevant blocks.

note: this may not be relevant, and depends upon our design/relation of the conductor to a tendermint light client; if we run a light client internally to the conductor (ie. track the validator set) then we need to keep this, but if we assume we have a trusted external client as we do right now, we can change this.

At the moment linting protobufs and pushing them to buf are on step. They should be separate steps (with the linting step being a requirement for the push step) so that linting can be used as a branch protection rule https://github.com/astriaorg/astria/blob/4aa28c1a8e82e9d5f36313891470165f01812d36/.github/workflows/proto.yml#L17

When tendermint proposes a new block, the block contains the commit for the parent block. However, the conductor currently treats this incorrectly, as when it receives a newly-proposed block, it executes it and marks it as safe/softly-committed, even though it's not actually finalized by tendermint yet. The correct logic when receiving a newly proposed block would be:

• execute and mark newly proposed block as "head" of chain
• verify commit contained in block (which is for the parent block), if it's verified, then mark parent block as "safe"
• receiving block from DA remains the same

Requires:

• updating the gRPC service to have methods such as:

ExecuteAndMarkAsHead(txs, prev_block_hash)
SetBlockAsSafe(block_hash)
SetBlockAsFinal(block_hash)

• updating the geth fork to support these methods
• updating conductor to correctly handle newly proposed vs committed blocks

blocked by #44