failed to select a version for the requirement `anchor-lang = "^0.22.0"`
candidate versions found which didn't match: 0.24.2
location searched: crates.io index
required by package `anchor-client v0.22.0`
    ... which satisfies dependency `anchor-client = "=0.22.0"` of package `cronos-sdk v0.1.8`
    ... which satisfies dependency `cronos-sdk = "^0.1.8"` of package `sol-ipl v0.1.0 

Install this software in the Docker environment so we can automatically check if the program code passes the Soteria audit.

https://www.soteria.dev/software

This PR removes the network observer module from the plugin. The network observer was designed to listen for changes to accounts belonging to the network program. It would then cache these accounts in memory and use them to in the transaction execution flows. We began noticing issues a few days ago (see ticket below for details) where this cached data was stale or incorrect. This would cause a worker would think it's in the pool even when it is not. Or visa versa, a worker would think it's not in the pool when it is.

Looking at transaction histories, it's still clear why the cached account data was incorrect. The only theory that makes sense to me is that workers can sometimes be on the bad fork and when jumping to the correct fork, account updates are not pushed to geyser. This theory is not confirmed, but in the meantime this PR serves as a fix for the issue. Instead of relying on the cached account data, this PR updates the plugin to simply use the client to fetch the fresh account data for registry, pool, and snapshot accounts.

https://linear.app/clockwork-xyz/issue/CLO-122/workers-not-recognizing-pool-position-updates

Users (specifically, Helium) have been hitting the compute unit limit in their transactions. This PR adds a ComputeBudgetInstruction::set_compute_unit_limit instruction to the front of every transaction. During simulation, we request the maximum 1.4M compute units available, giving transactions the flexibility of using as many compute units as they need. After simulation, we update the transaction to request only the compute units that were actually consumed. This way, Clockwork transactions will always request exactly the number of compute units they need. This will help with prioritization when https://github.com/solana-labs/solana/issues/28751 lands on Solana.

Use this action publish Github issue and PR activity into a Discord channel. If you take on this task, contact me to setup the Discord webhook key.

https://github.com/marketplace/actions/actions-for-discord

clockwork explorer get -k <address> clockwork explorer get <label>

thread: https://explorer.clockwork.xyz/thread/CjJqb4J3gib8xy6xcsxSz2Fo7AMYcsSCNkeUouMkznWf?network=devnet&programID=3XXuUFfweXBwFgFfYaejLvZE4cGZiHgKiGfMtdxNzYmv

Problem

No real problem lol. Eventually it might be useful to know which program is running which build/code. We do have the information, on the git's readme. But it can be more truthful to get it from the program directly.

Solution

Use a getter instruction to return data. I used set_return_data/get_return_data to return the information instead of an account (cost saving yay!)

Currently the crate info struct includes:

• The link to the Cargo.toml which was used for the deployment so we can know the version + the list of dependencies (I see a future where we deploy a program with a faulty dep)
• An empty blob, no idea for not but can be used to store metadata or another borsch structure later?

Also, might want to refactor this later into its own crate or macro so we can add this to the other network and pool programs too.

New feature

clockwork thread crate-info

CrateInfo {
    spec: "https://github.com/clockwork-xyz/clockwork/blob/v1.3.0/programs/thread/Cargo.toml",
    blob: "",
}

This PR address comments throughout the queue program in a format compatible with docs.rs. It additionally:

• Renames state module to objects.
• Renames is_paused to paused in queue account schema.
• Merges id.rs into lib.rs
• Adds a pubkey(&self) function to account traits, allowing other programs to easily verify account addresses without using seeds and bumps.

Similar to the ./version script, create a ./bump-solana script to update the Solana dependency versions across the repo. We should probably do this for Anchor dependencies too.

Addresses #1

--

script to update crate version for dependents given crate name and desired version.

How to use

simply run this command from either the root directory or the ci module.

yarn bump:crates

Config Options

Handles updates to Cargo.toml specific to each crate using a config:

Anchor repo has a publish script that seems to update the version across the repo using bash find-and-replace.

Modifying that script we could unify all the crate/package versions to v0.1.0 within a single commit

Just wanted to confirm whether each next_instruction is invoked by validators via separate transactions, or are they typically bundled within a single transaction?

I'm asking because I have a use case where a Thread will have 3 separate instructions, and there are enough combined accounts in these instructions that bundling them into the same transaction would surely lead to "transaction size limit" issues (e.g. >40 accounts in total).

Issue #104 proposed an implementation for static instruction sets using an enum. While this proposal sounded great in theory, it introduced a lot of extra complexity in implementation. By migrating to an enum, it requires every instance where the thread program references a kickoff_instruction or next_instruction to use a match statement and implement conditional logic depending on whether the thread has a "dynamic" instruction set or a "static" instruction set. Given the number of occurrences where this happens, this solution began to feel very fragile and the code was increasingly difficult to understand.

This enum-based interface additionally calls into question the use of the next_instruction property in the ThreadResponse. Can one use this with static instruction sets? Does it only work for dynamic instruction sets? Does it break static threads if I return a dynamic instruction? Would my dynamic instruction just be ignored? These questions and their answers are confusing and suggest that an enum is probably not the ideal interface for introducing static instruction sets.

This PR introduces an alternative interface. In short, the fundamental schema change is to replace kickoff_instruction: InstructionData with instructions: Vec<InstructionData>. It's fairly straightforward to see how this change adds support for static instruction sets. One can simply initialize a thread with a list of static instructions, and upon kickoff, the thread will execute the list of instructions sequentially. What is less obvious are the subtle implications this change has for flow control with dynamic instructions.

There are many potential ways of handling dynamic instructions here, but personally I think the best approach is to introduce as little change as possible and leave the current interface guarantees unchanged. That is, if any instruction returns a ThreadResponse with a next_instruction value, that instruction will be executed immediately following the current one. The noteworthy thing here is how dynamic instructions can "interrupt" the execution of a static instruction set. If dynamic instructions are executed immediately after they are returned, this means they can be injected or inserted between two instructions of the static set.

Confused? One way to visualize how a thread will process instructions (the flow control) is with a grid, visualized below. In the left-most column, going vertically from top to bottom, are the static instructions our example thread was initialized with (1, 2, 3, and 4 in the diagram below). Going horizontally, each row represents the dynamic instruction that were returned by the invoked program (1a, 1b, 1c, 2a, 4a, 4b, etc.). The diagram can be read like a book. Instructions are executed line by line, left-to-right, top-to-bottom. If a dynamic instruction is not returned, the thread simply proceeds to the next instruction in the static set and we use a newline to represent this.

1 → 1a → 1b →1c 2 → 2a 3 4 → 4a → 4b → 4c → 4d

This is a more complex method of flow control than the single "linked list" that we support today. One could describe this new flow control as a "list of linked lists". This approach has many benefits. It achieves the desired goal of supporting both static instruction sets and dynamic instructions sets. It also allows for a much simpler implementation than the enum-based approach. And perhaps most interestingly, it allows threads to combine dynamic instructions with static instructions to process complex workflows.

Is this added complexity necessary? Well, fortunately most use-cases will never need to delve into this complexity. The vast majority of threads today only use a single instruction or two. But this new model can be very useful for situations where the workflows themselves are complex. Take for example the network program. The network program uses a thread to drive epoch transactions. On each epoch transition, the thread is responsible for executing a series of "jobs". Depending on the state of the network, each job may have zero, one, few, or many "subtasks". At a high-level, the jobs of the epoch transitions can be broken down into the following checklist:

1. Lock the registry
2. Distribute fees to workers
3. Process unstake requests
4. Delegate stake to workers
5. Create a snapshot
6. Cutover to the new snapshot
7. Delete the old snapshot

With the thread interfaces currently available in v1.X, the network program is forced to "flatten" all of these jobs into a single long linked list of instructions. This flattening requires us to implement complex branching logic at the end of each instruction to dynamically jump to the correct next instruction depending on whether the current job is done or not. This branching logic is verbose, easy to screw up, and difficult to reason about. With the new thread interfaces proposed in this PR, a lot of this can be reduced and simplified.

One could use a thread's static instruction set as a series of kickoff instructions for sequential "jobs". If a job has some work to do, it can return dynamic instructions to do that work. If a job has no work to do, it can return a null response to simply proceed to the next job. By taking this approach, we can a lot of the branching logic in the network program and organize the code in a much more coherent way. Where we currently have a giant instructions/ folder with all the automated instructions and all the manual instructions co-mingled together, we can refactor the automated instructions into a jobs/ folder and create subfolders for each particular job (e.g. jobs/distribute_fees/, jobs/process_unstakes/, jobs/take_snapshot/, etc.). When we create the thread, we now only need to initialize it with the set of kickoff instructions for each job.

Note that "jobs" here are not a new abstraction that's being introduced or required by the thread program. They're simply an abstraction that is "allowed" by the flow control of the new interfaces. Developers can choose to use this abstraction if they find it useful, or they can chose to follow the programming patterns we currently recommend (put everything in a single giant linked list). The flows allowed by the new interface are a pure superset of the instruction flows supported today. This means existing threads can easily migrate to the new model without a significant rewrite (on thread creation, users simply to wrap their existing kickoff instruction in a vec![]).

As I was going through it, this PR grew and got a bit sprawling. At a high-level it contains the following changes:

• It updates the thread program for the new interfaces.
• It updates the network program to use a "jobs" abstraction.
• It also removes the kickoff_instruction value from ThreadResponse. (This value doesn't make as much sense with the new schema, and I've never actually seen anyone use it correctly. I've only ever seen it cause confusion and be used incorrectly.)

When building, we get a few of these warnings. Let's migrate from from_timestamp() to from_timestamp_opt().

warning: use of deprecated associated function `chrono::NaiveDateTime::from_timestamp`: use `from_timestamp_opt()` instead
   --> src/state/thread.rs:557:28
    |
557 |             NaiveDateTime::from_timestamp(after, 0),
    |                            ^^^^^^^^^^^^^^
    |
    = note: `#[warn(deprecated)]` on by default

When we added support for dynamic instruction threads, we unlocked a lot of new development possibilities. We also threw away our previous threading model which relied on a pre-defined list of instructions. This may have been too extreme of a pivot.

There are many use-cases where a static list of instructions would be preferable to dynamic threading. For example, users want to compose instructions of various programs together in a client-side frontend.

We could support both dynamic and static instruction sets by refactoring kickoff_instruction and next_instruction out of Thread and introducing a new enum:

pub enum InstructionSet {
  Dynamic {
    kickoff_instruction: InstructionData,
    next_instruction: Option<InstructionData>
  },
  Static {
    current_idx: i64,
    instructions: Vec<InstructionData>
  }
}

pub struct Thread {
  pub instruction_set: InstructionSet,
  // ...
}

In thread_exec, we would need to update the instruction set according to its type. It may also make sense to touch up the ThreadResponse schema.

Many threads need to share state between instructions. Programs are forced to save this state into an account and add the account as an input to all downstream instructions which need to read/write to it. Adding an extra account to programs can be tricky, deriving its PDA is error-prone, and mutating its data is restricted only to the program that owns it. Can we do better?

We know the thread account will be a signer on every executed instruction. Knowing this, thread accounts could provide a convenient location to save information that needs to be shared to downstream instructions. Any instruction would be able to write to a thread's cache simply by returning a KV map of data in the ThreadResponse. Downstream instructions would be able to access this data from the cache and update it.

Open questions:

• [ ] Should programs be forced to sign the data they write to the cache? This would help programs verify data authorship.
• [ ] Do programs need to care about data authorship? Or do threads provide a "trusted" computing environment?
• [ ] Should a thread's cache be deleted between each kickoff? It doesn't need to be.