Network-agnostic, high-level game networking library for client-side prediction and server reconciliation.

Overview
WARNING: This crate currently depends on nightly rust unstable and incomplete features.

crystalorb

Network-agnostic, high-level game networking library
for client-side prediction and server reconciliation.

crates.io docs.rs ci cd Coverage



Quick start

You may copy the standalone example to use as a starting template, and build off from there. You may also want to check out crystalorb-mock-network and crystalorb-bevy-networking-turbulence, either to use directly in your projects, or as examples for you to integrate your own choice of networking layer.

If you prefer a more visual and interactive example, there is the demo shown above that uses the Rapier physics engine. Feel free to use the demo's source code as a starting template for your next project.

For more information about how to implement the required traits, refer to the docs.

About

Game networking is hard, because we usually want to give the illusion that the physics simulation is responsive, fair, and consistent between multiple game clients despite having significant network latency between the clients and the server. There are many different ways to solve the game networking problem. CrystalOrb tries implementing one of such ways.

CrystalOrb is a young networking library that implements:

  • Client-side prediction. Clients immediately apply their local input to their simulation before waiting for the server, so that the player's inputs feel responsive.
  • Server reconciliation. Server runs a delayed, authoritative version of the simulation, and periodically sends authoritative snapshots to each client. Since the server's snapshots represent an earlier simulation frame, each client fast-forwards the snapshot they receive until it matches the same timestamp as what's being shown on screen. Once the timestamps match, clients smoothly blend their states to the snapshot states.
  • Display state interpolation. The simulation can run at a different time-step from the render framerate, and the client will automatically interpolate between the two simulation frames to get the render display state.

CrystalOrb does not (yet) implement "lag compensation" (depending on your definition of "lag compensation"), because crystalorb clients currently simulate all entities (including other players) in the present anyway. Some netcode clients would rather simulate entities in the past except for the local player. There are pros and cons to both methods:

  • By simulating everything in the present, collision timings will be consistent across all clients provided that no player input significantly changes the course of the simulation. This might be beneficial for physics-heavy games like... Rocket League? This is what crystalorb does. If you have two clients side-by-side, and you issue a "jump" command for the player using the left client, then the right client will see a jump after a small delay, but both clients will see the player land at the exact same time.

  • By simulating most remote entities in the past, remote entities require little correction in the client, so other players' movement will look more natural in response to their player inputs. If you have two clients side-by-side, and issue a "jump" command for the player using the left client, then the right client will not see any jerk in the movement, but the jump and the landing will occur slightly later after the left client.

Caveat: You need to bring your own physics engine and "mid-level" networking layer. CrystalOrb is only a "sequencer" of some sort, a library that encapsulates the high-level algorithm for synchronizing multiplayer physics games. CrystalOrb is physics-engine agnostic and networking agnostic (as long as you can implement the requested traits).

Is crystalorb any good?

Doubt it. This is my first time doing game networking, so expect it to be all glitter and no actual gold. For more information on game networking, you might have better luck checking out the following:

(Yes, those were where I absorbed most of my small game-networking knowledge from. Yes, their designs are probably much better than crystalorb)

Unstable Rust Features

CrystalOrb currently uses the following unstable features:

#![feature(const_fn_floating_point_arithmetic)]
#![feature(map_first_last)]
#![feature(const_generics)]
#![feature(generic_associated_types)]
Comments
  • WIP: Make ClientId type configurable

    WIP: Make ClientId type configurable

    This PR makes client_id type configurable (instead of hardcoded usize type).

    I am using net::SocketAddr to identify clients: https://github.com/smokku/soldank/blob/981f49b1/server/src/networking.rs#L35 This change allows me to work directly with connections: HashMap<SocketAddr, Connection> without keeping usize <-> SocketAddr mapping.

    There is one FIXME: though, for World::command_is_valid(). It receives client_id and I cannot get it to work without tying World type to NetworkResource trait. I don't want to do that, as World should not need to know networking implementation details. I'm not really sure where this method belongs. If I put it in NetworkResource, it will need to know World::CommandType type, which is the same problem, just reversed.

    opened by smokku 16
  • Degeneralize recv

    Degeneralize recv

    Implementation of #7. Split recv into three functions instead of a single generic one. Includes the necessary changes to crystalorb-mock-network but not crystalorb-bevy-networking-turbulence.

    opened by vilcans 6
  • Update rust nightly version and fix builds

    Update rust nightly version and fix builds

    proc_macro_is_available is now stabilised in rust 1.57, and updated 3rd party libraries have started using them, causing them to break with old rust compiler versions.

    When updating the rust version, the generic associated types in NetworkResource::ConenctionType now requires additional trait bounds.

    The test_env_log is also deprecated and renamed to test_log.

    opened by ErnWong 4
  • Fix compilation on the latest nightly

    Fix compilation on the latest nightly

    The feature const_generics was spitted to adt_const_params and generic_const_exprs, see more infor here. So in this PR I replaced const_generics with adt_const_params. But the compiler complains that the feature is incomplete, so I added incomplete_features to allow.

    Also I removed extended_key_value_attributes since it was stabilized in 1.54.0.

    Fixes #16.

    opened by Shatur 4
  • De-generalize Connection::recv

    De-generalize Connection::recv

    When I tried to integrate Crystalorb with custom networking, I found that it's not as simple as it can be.

    The function Connection::recv is declared as follows:

        /// Interface for CrystalOrb to request the next message received, if there is one.
        ///
        /// CrystalOrb will invoke this method with the three message types as specified in
        /// [`NetworkResource`].
        fn recv<MessageType>(&mut self) -> Option<MessageType>
        where
            MessageType: Debug + Clone + Serialize + DeserializeOwned + Send + Sync + 'static;
    

    It is generic on the MessageType, but it actually only supports three messages, as the documentation says. Having it generic on MessageType is as far as I can tell an unnecessary complication. I found it hard to follow the code in crystalorb_mock_network because of the code required to manage this.

    If I check the callers of the recv function in the crystalorb code base, it always looks like one of the following statements:

    connection.recv::<Timestamped<WorldType::CommandType>>()
    connection.recv::<Timestamped<WorldType::SnapshotType>>()
    connection.recv::<ClockSyncMessage>()
    

    I.e. the type of the message is known at the call-site and does not have to be generic.

    Wouldn't it be better if Connection instead declared those specific messages? I.e. something like this:

    fn recv_command(&mut self) -> Option<Timestamped<WorldType::CommandType>>;
    fn recv_snapshot(&mut self) -> Option<Timestamped<WorldType::SnapshotType>>;
    fn recv_clock_sync(&mut self) -> Option<ClockSyncMessage>;
    

    Then the code would not have to rely on run-time type information (TypeId) and would be more straight-forward.

    Let me know what you think. I may have missed the reasoning behind the design.

    opened by vilcans 4
  • Different code on github and crates.io.

    Different code on github and crates.io.

    Hello,

    I was trying to use this library for a project I am working on. The crates.io version doesn't have the same code base, and there are several compile errors. If it were possible to update the crates.io code to the newest version in the GitHub repo, that would be awesome.

    Thanks

    opened by kohlten 2
  • Crash report - Unexpected status change into: Blending(0.0)

    Crash report - Unexpected status change into: Blending(0.0)

    Full logs: https://gist.github.com/ErnWong/b63fa650bb013b572e1ba89d193a5c9c

    Relevant part of the log:

    ...
    wasm.js:384 Channel Incoming Error: packet channel is full
    imports.wbg.__wbg_error_712bad4931fcf046 @ wasm.js:384
    wasm.js:384 Channel Incoming Error: packet channel is full
    imports.wbg.__wbg_error_712bad4931fcf046 @ wasm.js:384
    wasm.js:384 Channel Incoming Error: packet channel is full
    imports.wbg.__wbg_error_712bad4931fcf046 @ wasm.js:384
    wasm.js:430 WARN /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/crystalorb-0.2.1/src/fixed_timestepper.rs:195 Attempted to advance more than the allowed delta seconds (48.8829). This should not happen too often.
    wasm.js:430 WARN /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/crystalorb-0.2.1/src/fixed_timestepper.rs:237 TimeKeeper is too far behind. Skipping timestamp from Timestamp(14192) to Timestamp(17110) with overshoot from 0.0018708340833272566 to 0.0023041674166355127
    wasm.js:430 WARN /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/crystalorb-0.2.1/src/fixed_timestepper.rs:195 Attempted to advance more than the allowed delta seconds (0.6154). This should not happen too often.
    wasm.js:430 WARN /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/crystalorb-0.2.1/src/fixed_timestepper.rs:184 Timestamp has drifted by -0.36539999999998446 seconds. This should not happen too often.
    wasm.js:430 WARN /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/crystalorb-0.2.1/src/fixed_timestepper.rs:195 Attempted to advance more than the allowed delta seconds (0.5785999999999845). This should not happen too often.
    wasm.js:430 WARN /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/crystalorb-0.2.1/src/client.rs:698 Abandoning previous snapshot for newer shapshot! Couldn't fastforward the previous snapshot in time.
    wasm.js:451 panicked at 'Unexpected status change into: Blending(0.0)', /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/crystalorb-0.2.1/src/client.rs:704:17
    
    Stack:
    
    Error
        at imports.wbg.__wbg_new_59cb74e423758ede (http://ernestwong.nz/dango-tribute/client/target/wasm.js:439:19)
        at console_error_panic_hook::hook::hbfe68726ffa79411 (http://ernestwong.nz/dango-tribute/client/target/wasm_bg.wasm:wasm-function[4500]:0x576a8e)
        at core::ops::function::Fn::call::hb6c015bf4f9077a4 (http://ernestwong.nz/dango-tribute/client/target/wasm_bg.wasm:wasm-function[20732]:0x7d482a)
        at std::panicking::rust_panic_with_hook::h894f2bdeea4d0ce8 (http://ernestwong.nz/dango-tribute/client/target/wasm_bg.wasm:wasm-function[8899]:0x6ebce6)
        at std::panicking::begin_panic_handler::{{closure}}::he30feeb51ae9f99b (http://ernestwong.nz/dango-tribute/client/target/wasm_bg.wasm:wasm-function[10701]:0x7396a5)
        at std::sys_common::backtrace::__rust_end_short_backtrace::h83a4032c6b997446 (http://ernestwong.nz/dango-tribute/client/target/wasm_bg.wasm:wasm-function[15727]:0x7ba229)
        at rust_begin_unwind (http://ernestwong.nz/dango-tribute/client/target/wasm_bg.wasm:wasm-function[15055]:0x7b11d5)
        at std::panicking::begin_panic_fmt::h9c5d99ded08e9f1a (http://ernestwong.nz/dango-tribute/client/target/wasm_bg.wasm:wasm-function[15627]:0x7b8edc)
        at <crystalorb::client::ClientWorldSimulations<WorldType> as crystalorb::fixed_timestepper::Stepper>::step::hc045531ebe8a0dc6 (http://ernestwong.nz/dango-tribute/client/target/wasm_bg.wasm:wasm-function[406]:0x10d539)
        at crystalorb::fixed_timestepper::TimeKeeper<T,_>::update::h658dbcd5fc9bfe5e (http://ernestwong.nz/dango-tribute/client/target/wasm_bg.wasm:wasm-function[1440]:0x32b3d0)
    
    
    imports.wbg.__wbg_error_4bb6c2a97407129a @ wasm.js:451
    wasm_bg.wasm:0x7d63c4 Uncaught RuntimeError: unreachable
        at __rust_start_panic (wasm_bg.wasm:0x7d63c4)
        at rust_panic (wasm_bg.wasm:0x7bcc33)
        at std::panicking::rust_panic_with_hook::h894f2bdeea4d0ce8 (wasm_bg.wasm:0x6ebd0a)
        at std::panicking::begin_panic_handler::{{closure}}::he30feeb51ae9f99b (wasm_bg.wasm:0x7396a5)
        at std::sys_common::backtrace::__rust_end_short_backtrace::h83a4032c6b997446 (wasm_bg.wasm:0x7ba229)
        at rust_begin_unwind (wasm_bg.wasm:0x7b11d5)
        at std::panicking::begin_panic_fmt::h9c5d99ded08e9f1a (wasm_bg.wasm:0x7b8edc)
        at <crystalorb::client::ClientWorldSimulations<WorldType> as crystalorb::fixed_timestepper::Stepper>::step::hc045531ebe8a0dc6 (wasm_bg.wasm:0x10d539)
        at crystalorb::fixed_timestepper::TimeKeeper<T,_>::update::h658dbcd5fc9bfe5e (wasm_bg.wasm:0x32b3d0)
        at crystalorb::client::ActiveClient<WorldType>::update::h8deb65d7204b2c76 (wasm_bg.wasm:0x2b1b10)
    
    bug 
    opened by ErnWong 2
  • Bump codecov-action version to v2 (v1 is deprecated) and try fixing PR checks

    Bump codecov-action version to v2 (v1 is deprecated) and try fixing PR checks

    GitHub secrets are not available during PR checks, and the codecov token is documented to be not needed for public repos (although I had difficulty getting it to work without a token last time).

    codecov-action v1 is deprecated and will be removed in February 2022.

    Testing this PR on a different account to check to see if the test-coverage check works now (see #11).

    Risk Areas:

    • Low risk on the build Pipeline.
    opened by NotErnWong 1
  • Test that we have covered all possible reconciliation status transitions

    Test that we have covered all possible reconciliation status transitions

    This PR:

    • Adds a quickcheck test that randomly invokes ClientWorldSimulations::step and ClientWorldSimulations::receive_snapshot with random timestamps.
    • Fixes any missing status transitions.

    Fixes #6

    Risk Areas:

    • None since we are only relaxing an existing runtime assertion.
    opened by ErnWong 1
  • Fix test-coverage checks for PRs

    Fix test-coverage checks for PRs

    I think it currently fails for PRs because GitHub actions don't send secrets to PR-triggered actions, and so it couldn't get the codecov key to publish the results. Either

    • we disable the codecov upload step for PRs, or
    • we disable the entire check for PRs altogether, or
    • we try and fix the GitHub workflow so that publishing to codecov works for PRs, if it is possible.

    Priority: medium-low since it would be confusing for new contributors to see their checks failing.

    opened by ErnWong 1
  • Unable to compile with latest nightly

    Unable to compile with latest nightly

    I have the following error (I removed duplicated errors to reduce log):

    error[E0557]: feature has been removed
     --> /home/gena/.cargo/registry/src/github.com-1ecc6299db9ec823/crystalorb-0.2.1/src/lib.rs:4:12
      |
    4 | #![feature(const_generics)]
      |            ^^^^^^^^^^^^^^ feature has been removed
      |
      = note: removed in favor of `#![feature(adt_const_params)]` and `#![feature(generic_const_exprs)]`
    
       Compiling time v0.2.27
    error: `TerminationCondition` is forbidden as the type of a const generic parameter
      --> /home/gena/.cargo/registry/src/github.com-1ecc6299db9ec823/crystalorb-0.2.1/src/fixed_timestepper.rs:92:56
       |
    92 | impl<T: FixedTimestepper, const TERMINATION_CONDITION: TerminationCondition>
       |                                                        ^^^^^^^^^^^^^^^^^^^^
       |
       = note: the only supported types are integers, `bool` and `char`
       = help: more complex types are supported with `#![feature(adt_const_params)]`
    
    For more information about this error, try `rustc --explain E0557`.
    error: could not compile `crystalorb` due to 10 previous errors
    warning: build failed, waiting for other jobs to finish...
    error: build failed
    

    I use rustc 1.57.0-nightly (11491938f 2021-09-29)

    opened by Shatur 0
  • Bevy 0.7

    Bevy 0.7

    As a game developer using the bevy engine, I want to continue using crystalorb with bevy 0.7, so I can have access to new bevy 0.7 features.

    Out of scope: Improving bevy integration and ergonomics. That is handled in #14 .

    To investigate: what needs to change to use, for example, the existing crystalorb-bevy-networking-turbulence crate with bevy 0.7?

    opened by ErnWong 0
  • Schedule Github Actions to run on latest rust nightly builds

    Schedule Github Actions to run on latest rust nightly builds

    Detect when the library could be broken on someone else's local rust nightly.

    Possible options:

    • Run like a dependabot update, but instead of making a PR that tries updating cargo dependencies, make a PR that tries updating ./rust-toolchain and run tests on the result. Downside is that dependabot PRs don't run the checks automatically I think.
    • Use scheduled events
    opened by ErnWong 0
  • Improve bevy ECS integration

    Improve bevy ECS integration

    Right now, if we were to use the provided bevy plugin, we would need to write most of the game logic outside of bevy's ECS. It would be good to integrate crystalorb with bevy's ECS.

    See other similar plugins for examples

    • https://github.com/gschup/bevy_ggrs
    • https://github.com/HouraiTeahouse/backroll-rs/tree/main/bevy_backroll

    Also investigate any new or upcoming bevy ECS features that would make the plugin more ergonomic.

    enhancement help wanted 
    opened by ErnWong 19
  • Networked example needed

    Networked example needed

    The example uses a mock connection, so when I tried to implement real networking, that took a while. It would be nice to have an example of a simple but more "real" project.

    FWIW, here's what I made: https://github.com/vilcans/orbgame. It's maybe not simple enough to use as an example, and maybe I'm not doing things the right way either, as I'm new to Bevy and hence also crystalorb in Bevy. I haven't figured out how to get events for disconnects yet, so players never disappear.

    opened by vilcans 3
Releases(v0.3.0)
  • v0.3.0(Dec 7, 2021)

    Breaking Changes

    1. (Issue #7 | PR #9). NetworkResource::Connection::recv<MessageType> has now been split (monomorphised) into three separate methods: recv_command, recv_snapshot, and recv_clock_sync to make it easier to implement a NetworkResource.

    2. (Issue #16 | PRs #17, #22). Update the rust nightly toolchain (nightly-2021-12-06).

      • The associated type NetworkResource::ConnectionType now has additional bounds as now required by the rust compiler (see rust-lang/rust#87479 and an insightful rust forum post). These bounds restrict how CrystalOrb can use ConnectionType, and so it should now be easier for your libraries to implement your own NetworkResource with fewer struggles with the borrow checker.

    Bugfixes

    1. (Issue #6 | PR #19). Fix a possible crash when the client sleeps for a long time and its state transitions from ReconciliationStatus::Fastforwarding(FastforwardingHealth::Obsolete) to ReconciliationStatus::Blending(0.0).
    Source code(tar.gz)
    Source code(zip)
  • v0.2.1(Jun 11, 2021)

    Fixed

    • crystalorb-bevy-networking-turbulence plugin used the wrong settings resource for registering the Timestamped<Snapshot> message channel (it used CommandChannelSettings instead of SnapshotChannelSettings), causing both incorrect configuration as well as panic, since the same channel number will be used twice.
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Jun 8, 2021)

    Fixed

    • Documentation and example codes in subcrate READMEs
    • Previously, it was possible to disable the serde/derive feature of the crystalorb crate by specifying default-features = false. However, this is unintended since crystalorb does not compile without this feature. Therefore, serde/derive was moved from the default feature and into the dependency specification itself.
    • The crystalorb-bevy-networking-turbulence crate did not even compile. This is now fixed.

    Added

    • A bevy plugin implementation for the crystalorb-bevy-networking-turbulence crate.
    Source code(tar.gz)
    Source code(zip)
Owner
Ernest Wong
💻 + 🎹 + 🔨 + 🎼 = 🕴
Ernest Wong
An Implementation of the Context Tree Weighting (CTW) Sequence Prediction Algorithm

Context Tree Weighting (CTW) CTW is a lightweight, practical and well performing sequence prediction algorithm discovered by Frans Willems, Yuri Shtar

null 7 Dec 23, 2022
Xaynet represents an agnostic Federated Machine Learning framework to build privacy-preserving AI applications.

xaynet Xaynet: Train on the Edge with Federated Learning Want a framework that supports federated learning on the edge, in desktop browsers, integrate

XayNet 196 Dec 22, 2022
High-level non-blocking Deno bindings to the rust-bert machine learning crate.

bertml High-level non-blocking Deno bindings to the rust-bert machine learning crate. Guide Introduction The ModelManager class manages the FFI bindin

Carter Snook 14 Dec 15, 2022
A high level, easy to use gpgpu crate based on wgpu

A high level, easy to use gpgpu crate based on wgpu. It is made for very large computations on powerful gpus

null 18 Nov 26, 2022
A light wheight Neural Network library with a focus on ease of use and speed.

Smarty Pants This goal of this library is to: Produce NeuralNetworks that will always give the same result when given the same input. Provide methods

Coding Wizard 3 Mar 7, 2022
A high performance python technical analysis library written in Rust and the Numpy C API.

Panther A efficient, high-performance python technical analysis library written in Rust using PyO3 and rust-numpy. Indicators ATR CMF SMA EMA RSI MACD

Greg 210 Dec 22, 2022
Simple neural network library for classification written in Rust.

Cogent A note I continue working on GPU stuff, I've made some interesting things there, but ultimately it made me realise this is far too monumental a

Jonathan Woollett-Light 41 Dec 25, 2022
Rust wrapper for the Fast Artificial Neural Network library

fann-rs Rust wrapper for the Fast Artificial Neural Network (FANN) library. This crate provides a safe interface to FANN on top of the low-level bindi

Andreas Fackler 12 Jul 17, 2022
n2 is a library implementation of a feedforward, backpropagation artificial neural network.

n2 is a library implementation of a feedforward, backpropagation artificial neural network. Usage Add the following to the [dependencies] section o

Søren Mortensen 0 Feb 21, 2021
A fun, hackable, GPU-accelerated, neural network library in Rust, written by an idiot

Tensorken: A Fun, Hackable, GPU-Accelerated, Neural Network library in Rust, Written by an Idiot (work in progress) Understanding deep learning from t

Kurt Schelfthout 44 May 6, 2023
RustFFT is a high-performance FFT library written in pure Rust.

RustFFT is a high-performance FFT library written in pure Rust. It can compute FFTs of any size, including prime-number sizes, in O(nlogn) time.

Elliott Mahler 411 Jan 9, 2023
A pure, low-level tensor program representation enabling tensor program optimization via program rewriting

Glenside is a pure, low-level tensor program representation which enables tensor program optimization via program rewriting, using rewriting frameworks such as the egg equality saturation library.

Gus Smith 45 Dec 28, 2022
A neural network, and tensor dynamic automatic differentiation implementation for Rust.

Corgi A neural network, and tensor dynamic automatic differentiation implementation for Rust. BLAS The BLAS feature can be enabled, and requires CBLAS

Patrick Song 20 Nov 7, 2022
A neural network crate

RustNN An easy to use neural network library written in Rust. Crate Documentation Description RustNN is a feedforward neural network library. The libr

Jack Montgomery 316 Dec 29, 2022
Simple Neural Network on rust

Simple Artificial Neural Network A crate that implements simple usage of dense neural networks. Instalation Add this to your dependencies on Cargo.tom

null 6 Jul 1, 2022
Machine learning Neural Network in Rust

vinyana vinyana - stands for mind in pali language. Goal To implement a simple Neural Network Library in order to understand the maths behind it. This

Alexandru Olaru 3 Dec 26, 2022
SelfOrgMap 5 Nov 4, 2020
A neural network model that can approximate any non-linear function by using the random search algorithm for the optimization of the loss function.

random_search A neural network model that can approximate any non-linear function by using the random search algorithm for the optimization of the los

ph04 2 Apr 1, 2022
A gpu accelerated (optional) neural network Rust crate.

Intricate A GPU accelerated library that creates/trains/runs neural networks in pure safe Rust code. Architechture overview Intricate has a layout ver

Gabriel Miranda 11 Dec 26, 2022