Solana JSON-RPC caching server

ZUBR

Last update: Dec 16, 2022

Related tags

Caching solana

Overview

Solana JSON-RPC caching server

Disclaimer: This project is an early stage Work-In-Progress and is not ready for production use.

This cache server implementation aims to provide a general solution for both offloading Solana validator RPC service and improving the overall speed and stability of the RPC. It achieves it by caching and updating some of the heaviest and most frequent requests and keeps the requested info updated with the use of PubSub API.

The server itself is a singe binary which is designed to be deployed in front of the validator as its public RPC entrypoint.

Running the server

To build and run the server you will need the Cargo package manager installed, which comes together with Rust compiler. Those two can be obtained here by following "Installing Rust" guideline.

# build
$ cargo build --release
# run
./target/release/rpccache

Configuration

The server supports a number of configuration options, which are the following:

-r, --rpc-api-url — validator or cluster JSON-RPC HTTP endpoint.
-w, --websocket-url — validator or cluster PubSub endpoint.
-l, --listen — cache server bind address.
-a, --account-request-limit — sets a maximum number of concurrent getAccountInfo requests the cache is allowed to send to the cluster/validator.
-p, --program-request-limit — sets a maximum number of concurrent getProgramAccounts requests the cache is allowed to send to the cluster/validator.
-b, --body-cache-size — sets the maximum amount of cached responses.
-c, --websocket-connections — sets the number of websocket connections to validator
-t, --time-to-live — duration of time for which values will be kept in cache
-d, --slot-distance — sets the maximum slot distance for health check purposes
--ignore-base58-limit — flag whether to ignore base58 overflowing size limit
--log-format — the format, in which to output the logs: plain | json
--rules — path to firewall rules written in lua
--control-socket-path — path to socket file, e.g. /run/cacherpc.sock

Features

Implemented methods

In the current version caching is implemented for these methods:

Requests to other methods are passed through to the validator.

Please note that "encoding": "jsonParsed" is not yet supported.

Unlikely to be implemented

Features that are unlikely to be implemented:

Root and Single commitments.

Disclaimer: This project is an early stage Work-In-Progress and is not ready for production use.

Comments

Cannot request Stake11111 (Gateway timeout)

When requesting getProgramAccounts(Stake11111...) without filters via the cache I get a gateway timeout error, however it works without the cache.

In logs I can see the following error:

Sep 15 12:39:24.950  WARN cache_rpc::rpc: request: Request { jsonrpc: "2.0", id: Num(1), method: "getProgramAccounts", params: Some((Pubkey([6, 161, 216, 23, 145, 55, 84, 42, 152, 52, 55, 189, 254, 42, 122, 178, 85, 127, 83, 92, 138, 120, 114, 43, 104, 164, 157, 192, 0, 0, 0, 0]), ProgramAccountsConfig { encoding: Base64, commitment: None, data_slice: None, filters: None, with_context: Some(true) })) } error: Overflow
Sep 15 12:39:24.950  INFO cache_rpc::rpc: reporting gateway timeout req.id=Num(1)

This is the RPC request:

curl api.mainnet-beta.solana.com -X POST -H Content-Type: application/json -d 
  {"jsonrpc":"2.0","id":1, "method":"getProgramAccounts", "params":["Stake11111111111111111111111111111111111111", { "encoding": "base64" }]}

opened by linuskendall 7

getAccountInfo occasionally returns error
It's been reported that getAccountInfo occasionally returns

Error: failed to get info about account 2CSEjyDtAtgCjTKyXHZyfUVe7EERtL7rjYjJSgcBPYLf: Invalid params: invalid type: map, expected string at line 1 column 34

Gist to reproduce here https://gist.github.com/barry-the-sender/d877a87f23d8d52d32fdd71c01f71481.
opened by armaniferrante 5
Track subscriptions by key

The problem:

We use subscription_active method to check whether we can retrieve the value from cache for a particular key. Currently it's implemented by checking AtomicBool flag for a websocket connection to which this key is routed. This flag is updated in update_status by comparing number of active subscriptions to the number of desired subscriptions, which means that if one of subscription confirmations is delayed, all keys routed to this worker are considered not active, and are not retrievable from cache, which is obviously not desirable.

Proposed solution:

Track subscription status by key.
enhancement

opened by polachok 4
Added signal handling to prevent new subscriptions

Signal handler listens for USR1 signal and upon receiving one, toggles the switch in PubSub manager. In true state the pubsub manager will allow unlimited number of new subscriptions, in false state no new subscriptions can be created, and thus only the present ones will continue to exist, until they are too removed due to TTL.

opened by bobs4462 3
Implementation of bounded wait queue for rpc requests

It was done via wrapping semaphore type, for the limit of concurrent rpc requests, in a new type which also manages the number of awaiting tasks on that particular semaphore, if that number becomes more than the preconfigured value, then all the subsequent requests of that type will be rejected.

Once the task manages to acquire semaphore, it releases its slot in wait queue, so other tasks will be able to try to acquire a semaphore and be queued if they cannot.

opened by bobs4462 2
Modified web socket subscription check for given cache entry

For a given subscription and commitment level, if a cache entry is found, and web socket connection is estabilished, then check whether we have a corresponding active websocket subscription for that subscription+commitment pair

Closes #152

opened by bobs4462 1
Added support for batch requests

Cacher will try to perform preliminary parsing of the request, and if it encounters a request containing an array of subqueries, then it will pass it on to validator, otherwise request handling proceeds normally.

Closes SOL 29

opened by bobs4462 1
WAF

Optional WAF support was added to cacher, which can be turned on during the application launch via --rules=<path to rules> option. Rules itself should be lua file, analogous to example file provided in the root of the project.

opened by bobs4462 1
Fixed failing tests due to range error

The issue was that helper account method generated AccountData values with len() ranging from 1, in case if len turned out to be 1, then valid_ranges couldn't generate range, since it also tried to build range starting from value 1 and ending with whatever is the len() of AccountData, 1..1 in case which triggered error.

As a solution, the minimum AccountData length was set to 2, since there's no logic requirements for it to start from 1.

Closes #159

opened by bobs4462 1
Some fixes: config loggin and clippy warnings

Clippy warnings related to large number of function arguments has been fixed by collecting some of those into a separate struct

Configuration update logging has been rewritten using standard tracing package tools with no custom formatting

opened by bobs4462 1
Added command line parameter to control SLOT_DISTANCE value

Things to consider: AccountUpdateManager struct potentially might have a lot of instances, and withc current implementation every instance will eat up additional 4 bytes. Alternative would be to create a mutable static, and initialize it once at program startup, while this would introduce unsafe into code, it will remove unnecessary overhead of 4 bytes.

opened by bobs4462 1
Detect infinite reconnections on websocket

Sometimes, some worker threads, that manager webosket connections, go into infinite loop trying to establish working websocket connection to validator. Need to figure out the reason, why it happens, and implement detection and correction for such cases.

opened by bobs4462 0
Detect situations when when ws subscriptions are active but no updates received

Cacher relies on the fact that cache entries have corresponding active ws subscriptions, in order to detect whether the cache entry might be stale or not. But sometimes it may be that subscription exists but no updates are being received from it, which possibly may indicate that the cache entry became stale.

There's a need for a way to detect such situations and not to serve request from cache, if it happens.
enhancement

opened by bobs4462 1
Any path component should be accepted

To follow the behaviour of the solana validator and be as transparent as possible the cache should accept any path "/" component of the URL. I have seen some applications use this in the wild and it is supported by the validator.

There are also special URL paths in the validator that get translated into non RPC requests like "/health".

Currently cacherpc returns 404 on any path a part from "/". We work around it in our proxy by just overwriting the path component to any request we pass to zubr cache.

opened by linuskendall 1

Releases(0.2.17)

0.2.17(May 25, 2022)
Highlights:

Deduplication of getProgramAccounts requests. When gPA request is made, cacherpc will "remember" it as being in progress. If the same gPA request is made again before the first one had a chance to complete, the second one (and all those that come after) will wait for the original to finish, after this event they will respond from cache. getProgramAccounts request is defined by three values: public key, commitment level and set of filters, and other parameters like encoding and dataSlice are ignored. All subsequent requests will wait a limited amount of time before returning timeout error to the user, this timeout parameter can be separately configured via in_progress_request_wait: u64 field of [rpc.timeouts] sections of configuration file. In case if error occurs during the execution of original request, no waiters will be notified, the cacherpc will "forget" the request as being in progress, all current waiters will eventually abort with timeout, single new request will be forwarded to validator. Not all gPA requests can be subjected to this new logic, specifically if the gPA is not cacheable, then cacherpc will not remember it as being in progress, cacheability is satisfied if: encoding is not jsonParsed, no dataSlice is requested, cacherpc has active websocket connection.

What's Changed

feat: wait for exectuting gpa requests, instead of making new ones by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/261

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.2.16...0.2.17
Source code(tar.gz)
Source code(zip)
0.2.16(May 24, 2022)
Highlights

Implement client-independent validator response caching. Previously only requests which were successfully streamed to clients could be cached. This release changes this behavior, so that response is cached independently of client connection (even if client disconnects) as long as it's a successful one.

Fixed lint errors reported by clippy

What's Changed

feat: spawn task to collect response by @polachok in https://github.com/zubr-exchange/cacherpc/pull/259

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.2.15...0.2.16
Source code(tar.gz)
Source code(zip)
0.2.15(Apr 5, 2022)
Highlights

X-Request-ID is added to several error responses: a. Invalid request b. Waf rejection c. Request timeout

Improved logging: added new logs, made existing logs more informative. Also it's now possible to control the log level, which is done by setting environment variable CACHER_LOG_LEVEL to values debug|info|warn|error, by default it's set to info

Configurable keep-alive parameter for server. After new connection is accepted and new request is made over it, server will keep connection alive (if not closed by client) for the period of time specified in --keep-alive <NUM>s cli parameter

Client request timeouts are disabled, which makes it possible for clients to open connection pool to server without actually sending any data over those connections, and server will keep connections alive for the period specified in keep-alive startup parameter

What's Changed

Keep-alive management, no client request timeout + technical debt by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/252

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.2.14...0.2.15
Source code(tar.gz)
Source code(zip)
0.2.15-beta-5(Apr 1, 2022)
Highlights:

Increased keep-alive duration for connection to 90 secs

What's Changed

Keep alive by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/257

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.2.14...0.2.15-beta-5
Source code(tar.gz)
Source code(zip)
0.2.15-beta-4(Apr 1, 2022)
Highlights:

Added logging when new connection is accepted by server

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.2.14...0.2.15-beta-4
Source code(tar.gz)
Source code(zip)
0.2.15-beta-3(Mar 28, 2022)
What's Changed

Extra logs by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/254

Extra logs by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/255

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.2.15-beta-1...0.2.15-beta-3
Source code(tar.gz)
Source code(zip)
0.2.15-beta-1(Mar 23, 2022)
Highlights

X-Request-ID is added to several error responses: a. Invalid request b. Waf rejection c. Request timeout

Improved logging: added new logs, made existing logs more informative. Also it's now possible to control the log level, which is done by setting environment variable CACHER_LOG_LEVEL to values debug|info|warn|error, by default it's set to info

Source code(tar.gz)
Source code(zip)
cache-rpc(21.75 MB)
0.2.14(Mar 21, 2022)
Highlights:

Fixed X-Request-iD header handling when cache hits occured (both lru and data)

What's Changed

X request header related fixes by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/251

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.2.13...0.2.14
Source code(tar.gz)
Source code(zip)
cache-rpc(21.72 MB)
0.2.14-beta-3(Mar 18, 2022)

Modified x-request-id to support cache hits as well (#249)
Source code(tar.gz)
Source code(zip)
cache-rpc(21.72 MB)
0.2.14-beta-2(Mar 18, 2022)

Fixed x-request-header for passthrough requests
Source code(tar.gz)
Source code(zip)
cache-rpc(21.72 MB)
0.2.13(Mar 17, 2022)
Highlights:

Added support for X-Request-ID HTTP header: if the header is present in client request, it will be forwarded to upstream, if the header is absent, then the cacher will generate it locally and attach it outbound requests, and even cache hits responses

Updated project dependencies, particularly actix-web related ones

What's Changed

Added X-Request-ID header + updated dependencies by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/248

Modified x-request-id to support cache hits as well by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/249

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.2.12...0.2.13
Source code(tar.gz)
Source code(zip)
cache-rpc(21.72 MB)
0.2.12(Feb 15, 2022)
Highlights

Added new command for forcing re-connection of websockets. The command for re-connection has three modes, namely:

cache-rpc force-reconnect init <delay sec> <interval sec> - will initiate re-connection, starting after delay seconds, and performing 1 re-connection every interval seconds

cache-rpc force-reconnect status - get status of reconnection, i.e. whether it's running, number of re-connection done/remaining

cache-rpc force-reconnect abort - stop re-connection process, reports how many have been performed and how many remain

Added 2 metrics to indicate re-connection progress: forced_reconnections_remaining - number of re-connection to perform yet, forced_reconnections_finished - number of re-connections already performed

Made wide-filter based program accounts prefetching (#243) to be an optional feature, now by default cache-rpc will start with this feature disabled, while superset based request serving will still be enabled without an option to turn it off

Added new command for turning on the feature for wide-filter based getProgramAccounts requests, which has 3 modes of operation:

cache-rpc wide-filters on - turns the account prefetching for given program with wider filter

cache-rpc wide-filters off - turns the above feature off

cache-rpc wide-filters status - shows the current status of the feature

Added new metric for counting the number of self initiated getProgramAccounts requests - self_initiated_gpa

What's Changed

Implemented support for additional ws-reconnection command by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/245

Made wide filters based gpa requests optional by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/246

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.2.11...0.2.12
Source code(tar.gz)
Source code(zip)
cache-rpc(23.14 MB)
0.2.11(Feb 11, 2022)
Highlights

Main and the biggest change of this release is the way that getProgramAccounts are handled. The following logic has been implemented

If the getProgramAccount request is made with filter, and no cache entry is found for the given filter, then search for existence of this filter's superset, i.e. a different filter which when applied to all accounts of the program, will produce a subset of it which will include the accounts for the original filter (which was present in request). Such a filter is a less restrictive one. If found, then the list of accounts is filtered in-place by cacherpc itself before being served to client.

If superset filter cannot be found in cache, then try to search for valid intersections of the requested filter with others present in cache, i.e. new filter which will be a superset of the original filter in request.

In case such an intersection can be found, a separate asynchronous gPA request is made to cluster/validator, which is identical to the one made by client, except for the filter.

The original request is also made, as cacherpc cannot wait for second request to finish and should start streaming data to client as soon as possible

The result of second request (with superset filter) is saved to cache, and should prevent future cache misses

When an entry for the given filter is not found in cache, but the request still can be served from superset, then the ttl for the superset filter will be reset, and not for the originally requested filter, so that hot supersets will linger for long time in cache.

Refactored pubsub module, splitting it into several files, for better maintainability

What's Changed

Refactored pubsub module by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/244

Implemented optimized filter based caching for gPA by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/243

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.2.10...0.2.11
Source code(tar.gz)
Source code(zip)
cache-rpc(23.04 MB)
0.2.10(Feb 8, 2022)
Highlights

Added handling of getIdentity method

In order to enable cacherpc service to serve getIdentity requests, one should start it with --identity <public key> option

If the identity option is not provided, all requests are forwarded to validators as usual

What's Changed

Support getIdentity method by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/242

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.2.9...0.2.10
Source code(tar.gz)
Source code(zip)
cache-rpc(22.90 MB)
0.2.9(Feb 3, 2022)
Highlights

Fixed the content-type header guard in request handler, so that headers like Content-Type: application-json;charset=utf-8 will be accepted

What's Changed

Fixed content-type header guard for request handler by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/241

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.2.8...0.2.9
Source code(tar.gz)
Source code(zip)
cache-rpc(22.90 MB)
0.2.8(Feb 1, 2022)
Highlights

Added X-Cache-Request-Header (describing json-rpc method) to passthrough requests, getAccountsInfo and getProgramAccounts already include this header.

What's Changed

Extra request header to differentiate json-rpc methods by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/240

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.2.7...0.2.8
Source code(tar.gz)
Source code(zip)
cache-rpc(22.89 MB)
0.2.7(Jan 31, 2022)
Highlights

Mostly maintenance release with dependency updates and minor bug-fixes

What's Changed

Moved cache hit metric increment inside conditional by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/238

Maintenance by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/239

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.2.6...0.2.7
Source code(tar.gz)
Source code(zip)
cache-rpc(22.86 MB)
0.2.6(Dec 17, 2021)
Highlights

Bugfix: slot number not being saved for gPA requests made with withContext parameter

Implemented cache overwrite control so that new account entries won't be overwritten with more outdated data

What's Changed

Fix: gPA slot not being saved immediately, if request made withContext by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/236

chore: bump version to 0.2.6 by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/237

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.2.5...0.2.6
Source code(tar.gz)
Source code(zip)
cache-rpc(22.47 MB)
0.2.5(Dec 16, 2021)
Highlights:

Fixed bug, when cache miss occured for getProgramAccounts request, when requesting the same program with withContext flag being explicitly set to false.

What's Changed

Hotfix: correct handling of withContext flag for gPA by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/234

chore: version bump to 0.2.5 by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/235

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.2.4...0.2.5
Source code(tar.gz)
Source code(zip)
cache-rpc(22.45 MB)
0.2.4(Dec 15, 2021)
Highlights:

Configurable timeouts were added. Now both request and retry timeouts can configured via command line and config files: --request-timeout 60s parameter specifies the request timeout for passthrough requests, gAI and gPA requests and retry timeouts can be configured via configuration file and can be reloaded during runtime.

Optimized cache hits for gAI after owner program is purged from cache, but some of its accounts persist in cache with renewed ttl. Now if account request is made during program lifetime and is served from cache, then account slot info will be updated (if missing) with that of owner program (if exists), thus consequent requests still will be served from cache, even if owner program is purged from cache. Solves https://zubr-exchange.atlassian.net/browse/SOL-94.

Bug-Fix for gPA requests alternating between without | with context. Now if program is present in cache but doesn't have context info, all gPA requests which require context will be forwarded to validator, as to avoid returning context = 0 to client. Solves https://zubr-exchange.atlassian.net/browse/SOL-93

What's Changed

Rpc module refactoring by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/227

Added configurable timeouts for requests, and retries by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/232

Fixed slot = 0 case for gPA requests with context by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/231

Improved cache hit probability for accounts after owner sub ends by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/230

chore: version bump to 0.2.4 + readme update by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/233

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.2.3...0.2.4
Source code(tar.gz)
Source code(zip)
cache-rpc(22.45 MB)
0.2.3(Dec 9, 2021)
Highlights:

The old behavior for handling filtered getProgramAccounts requests, when cache contains data for unfiltered one, has been restored. Now, if cache has an entry for gPA requests, which has been made without filters, it follows that we have all the data for the given program available in cache, and as such we perform filtering in caching service, and serve the request from cache.

What's Changed

Fixed cache miss, when cache contains unfiltered gpa response by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/228

chore: version bump to 0.2.3 by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/229

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.2.2...0.2.3
Source code(tar.gz)
Source code(zip)
cache-rpc(22.40 MB)
0.2.2(Dec 8, 2021)
Highlights

Fix: wrong argument was used to limit the number of waiting gPA requests (--program-request-queue-size parameter)

New metrics were added to gain some insights related to response serialization:

serialization_time - compound histogram metric, which uses triple (request type, request encoding, rpc worker thread id) for labeling purposes. It shows the amount of time (in seconds) that caching server spent serializing response after cache hit occurred (not LRU).

account_data_len - histogram metric, which uses rpc worker thread id as its labels. It shows the number of bytes in account's data field before this account was serialized, which should provide some information as to how large accounts can be (in binary), and how their size affects the rpc thread load during serialization.

What's Changed

Fixed wrong queue size argument passing for program accounts by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/223

Added serialization related metrics by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/225

chore: bumped version to 0.2.2 + updated readme by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/226

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.2.1...0.2.2
Source code(tar.gz)
Source code(zip)
cache-rpc(22.40 MB)
0.2.1(Dec 2, 2021)
Highlights:

Fix: hot reload of waf-rules now works as expected via cache-rpc waf-reload command

Bounded wait queue was implemented for RPC requests. Now when application starts, it can accept 2 additional parameters:

account-request-queue-size - number of client connections that can wait for available permits for getAccountInfo

program-request-queue-size - number of connections that can wait for available permits for getProgramAccounts Those two parameters complement the other two parameters account-request-limit and program-request-limit respectively. What they do is, they can prevent new client requests, which cannot be served from cache, from trying to acquire available permits for their corresponding request type, and as such indirectly allow to control how many requests can be made to validator in short bursts. It basically works by forcing every request (not serviceable from cache) to get into queue (which has limited size), before even trying to acquire permit to make the actual request, if it doesn't succeed (due to queue being full), the response will be returned immediately to client with message explaining that there are too many requests of this type is being made. If request does succeed, it can go ahead and try to acquire a permit for making the request, and upon success it will move out of wait queue, so that other requests can also attempt to acquire a permit (by getting into queue). These parameters can be hot reloaded from config file, the same way that account-request-limit and program-request-limit can, example config might look like this:

[rpc] ignore_base58_limit = true [rpc.request_limits] account_info = 500 program_accounts = 15 [rpc.request_queue_size] account_info = 1000 # default is set to value 2 ^ 19, effectively disabling any restrictions program_accounts = 75 # default is set to value 2 ^ 18, effectively disabling any restrictions

Completely reworked subscription tracking (biggest change of release), here's the list of most notable updates:

When getAccountInfo triggers subscription, account key starts being tracked in program cache, by means of the public key of account owner, it's done to transfer subscription management responsibility to owner program, when request for its owner will be made in future.

If during subscription attempt for account, it's found that owner program already has cache entry and subscription, then no new account subscription is created, but account key is still added to the list of tracked keys of its owner.

When getProgramAccouns gets cached, it unsubscribes from all tracked child accounts accounts without deleting them from cache.

All accounts, which are being tracked by their owner, get served from cache, as long as parent has an active subscription, and do not create their own subscription, but resetting their ttl nonetheless.

When getAccountInfo hits the cache it creates a purge task for itself, and when timer expires, it removes itself from tracked accounts in its owner's cache entry, and tries to remove itself from accounts cache, succeeding only if no other getProgramAccounts request is tracking it.

When getAccountInfo misses the cache, while it's owner still having active subscription, the result is similar to case 5, it fetches account by RPC, puts it into accounts cache, starts tracking it in owner's cache entry, and doesn't create extra subscription for itself, while still making sure that it will be removed from cache when it's ttl ends.

When getProgramAccounts cache entry expires, it will perform cleanup for all of its child accounts, except for those which had recent cache hits, in which case it will resubscribe for those before terminating its own subscription.

What's Changed

Fixed waf rules reload by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/219

Implementation of bounded wait queue for rpc requests by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/187

Subsription tracking refactoring by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/215

Wait queue size for rpc requests was made optional by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/222

chore: bump version to 0.2.1 + remove mlua patch by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/221

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.2.0...0.2.1
Source code(tar.gz)
Source code(zip)
cache-rpc(22.38 MB)
0.2.0(Nov 22, 2021)
Highlights:

More even distribution of heavy program accounts subscriptions. Previously all subscriptions with the same pubkey would be served on the same websocket connection, now the choice of connection is additionally determined by commitment level.

New metric for counting requests. backend_requests_count is vector of 3 counters: getAccountInfo, getProgramAccounts, passthrough type of requests.

Control interface now has additional command to reload WAF rules from lua file. File path should be provided on cacher startup, reload will take place from the same file. Usage: cache-rpc waf-reload. As each worker thread of server has it's own copy of filters, it will take N rpc requests for them to reload their copies from disk, where N is the number of logical CPU cores. It happens because reloading of rules takes place right before handling of RPC request.

What's Changed

Removed unwrap when starting control interface by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/211

chore: removed patch on actix-tls by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/210

Added commitment to pubsub worker index computation by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/212

Added new metric for counting requests of each type by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/213

fix: index observed filters by key and commitment by @00nktk in https://github.com/zubr-exchange/cacherpc/pull/216

Added cotrol command to reload waf rules from disk by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/217

chore: version bump to 0.2.0 by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/218

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.1.38...0.2.0
Source code(tar.gz)
Source code(zip)
cache-rpc(22.27 MB)
0.1.38(Nov 12, 2021)
Highlights:

Fixed command handling for cache-rpc subscriptions status

Reverted subscriptions deduplication feature, which was introduced in last release, until a fix is proved to be logically correct

Added extra header to getProgramAccounts and getAccountInfo requests, that are sent from cacher to balancer. This header has a key: X-Cache-Request-Method and a value: either getProgramAccounts or getAccountInfo.

What's Changed

hotfix: fixed handling of subscriptions status command by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/204

Added extra header for load balancer by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/206

Revert "Added logic to deduplicate subscriptions (#185)" by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/207

chore: bump version to 0.1.38 by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/208

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.1.37...0.1.38
Source code(tar.gz)
Source code(zip)
cache-rpc(22.20 MB)
0.1.37(Nov 11, 2021)
Highlights:

WAF rules are now applied to batch requests as well. Every single requests in batch will be subjected to evaluation before further processing can be performed, and in case if one of the requests fail to satisfy checks, entire batch request will be rejected.

Subscriptions deduplication has been improved. Now before creating a new subscription for account, cacher first checks whether there's already exists a subscription for that account's owner (given one is present). Also when creating a subscription for a program, we unsubscribe from all of it's child accounts, if separate subscriptions exist for them.

New metric was added subscriptions_skipped for counting the number of skipped subscriptions, because of improved subscriptions tracking feature.

New metric was added streaming_errors for counting the number of connection drops or other error which occur in the middle of streaming the response from validator to client.

Control interface was implemented in order to make changes to dynamic settings a bit more convenient. Right now control interface supports 4 commands:

cacherpc config-reload - forces running instance of cacherpc to reload its configuration file

cacherpc subscriptions on - allows running instance of cacherpc to create new websocket subscriptions

cacherpc subscriptions off - prevent running instance of cacherpc from creating any new websocket subscriptions

cacherpc subscriptions status - get current status of whether subscriptions are on or off If command succeeds, it returns exits with code 0, code 1 is returned otherwise, along with description of what has gone wrong

What's Changed

Added WAF support for batch requests by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/199

chore: commited Cargo.lock by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/200

Added logic to deduplicate subscriptions by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/185

feat: count streaming errors by @polachok in https://github.com/zubr-exchange/cacherpc/pull/201

Implemented control interface to issue commands to running instance by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/198

chore(ci): add all-targets/all-features for clippy by @polachok in https://github.com/zubr-exchange/cacherpc/pull/203

chore: bump version to 0.1.37 by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/202

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.1.36...0.1.37
Source code(tar.gz)
Source code(zip)
cache-rpc(22.28 MB)
0.1.36(Nov 2, 2021)
Highlights:

The project was migrated to Actix-Web version 4 (from 3), and tokio 1 (from 0.2). This migration should solve the thread panicking issue due to bug in DelayQueue implementation of tokio time.

Response parsing for getHealth rpc request was fix, so that the error updating rpc health err=Json deserialize error: invalid type: newtype struct should not happen anymore. Additional logging was added, just in case, if the fix doesn't work

What's Changed

Changed the data field for rpc error response by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/190

refactor: update to actix 4.0/tokio 1.0 by @polachok in https://github.com/zubr-exchange/cacherpc/pull/192

chore: bump version to 0.1.36 by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/194

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.1.35...0.1.36
Source code(tar.gz)
Source code(zip)
cache-rpc(21.20 MB)
0.1.35(Nov 1, 2021)
Highlights:

Added logging and metrics counter (waf_rejections), when WAF rejection takes place.

Moved handling of WAF rule evaluation to higher level, so that every request (single ones, not batch requests) will be subjected to checks.

Added a new signal handler, to intercept USR1 signals, sent to application process. This signal can be used to toggle a new flag, which is responsible for allowing new websocket subscriptions, i.e. when the flag is set to true (default) new subscriptions can be made, if it's switched to false, cacher will refuse to put any new responses to cache and respectively to create a new subscriptions for them. Present subscription and cache entries will continue to exist, until they are removed due to absence of cache hits.

What's Changed

Created metric for counting the number of WAF-caused request rejection by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/183

Moved WAF filtering to rpc handler by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/188

Added signal handling to prevent new subscriptions by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/189

chore: bump version to 0.1.35 by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/191

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.1.34...0.1.35
Source code(tar.gz)
Source code(zip)
cache-rpc(23.93 MB)
0.1.34(Oct 22, 2021)
Highlights:

Some minor error result consistency improvements (SOL-33)

Support added for batch requests (with multiple subqueries), but for now those requests aren't handled by cacher and passed on directly to validators, and as a consequence their results are not cached either (SOL-29)

Added new metric to track the number of batch requests called batch_requests which is just a normal integer counter.

Web application firewall was added as an optional feature, in order to use it one has to provide a file containing filtering rules, written in Lua language, via --rules flag. See the example file on how to write rules.

Improved cache hit rate for program related accounts, this is done via sharing the data which was cached by getProgramAccounts with getAccountInfo. But the cache hit is not guaranteed for now, because getProgramAccounts responses don't necessarily contain context with slot, which is required by getAccountInfo requests. So in order for cache hit to occur, one of the related accounts to getProgramAccounts should be sent via websockets (as a notification), and thus updating the owner program's slot. This mechanism should reduce the amount of active subscriptions (but probably not much).

What's Changed

Invalid parameters error: consistency with validator by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/179

Added support for batch requests by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/178

WAF by @polachok in https://github.com/zubr-exchange/cacherpc/pull/156

Added default slot for getProgramAccount requests by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/180

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.1.33...0.1.34
Source code(tar.gz)
Source code(zip)
cache-rpc(23.92 MB)
0.1.33(Oct 13, 2021)
Highlights:

Two new metrics were added:

counter for websocket reconnect attempts websocket_reconnects

counter for rpc request retries request_retries

Number of websocket reconnect attempts has been made infinite, with interval between attempts not exceeding 10 seconds

Random failure of tree_matches_overall proptest has been fixed

In case of invalid request, the error response has been made to match that of validator

jsonParsed encoding support was added to cacher for the already cached values (optional feature, on by default)

What's Changed

Made backoff for websocket connection attempts infinite by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/169

AccountUpdateManager mini refactoring + readme update by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/168

Added 2 counters to metrics: rpc request retries, websocket reconnects by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/167

Fixed failing tests due to range error by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/170

Changed the error type in case of invalid request by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/172

chore: bump version by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/173

Added feature to support jsonParsed encoding by @bobs4462 in https://github.com/zubr-exchange/cacherpc/pull/174

Full Changelog: https://github.com/zubr-exchange/cacherpc/compare/0.1.32...0.1.33
Source code(tar.gz)
Source code(zip)
cache-rpc(19.90 MB)