Rust implementation of Hyperswarm, a networking stack for connecting peers

Overview

hyperswarm-rs

Peer to peer networking stack

NOTE: This is still in early stages. See the roadmap below. Please feel free to open issues and send PRs :-)

Installation

$ cargo add hyperswarm --git https://github.com/Frando/hyperswarm-rs.git

Usage

Hyperswarm is a networking stack for connecting peers who are interested in a topic. This project is a port of the Node.js implementation of Hyperswarm.

This crate exposes a Hyperswarm struct. After binding it, this will:

  • Start and bootstrap a local DHT node
  • Bind a socket for mDNS discovery
  • Announce and lookup any 32 byte topic key over both mDNS and the DHT
  • Connect to all peers that are found over both TCP and UTP

It currently depends on the unreleased hyperswarm-dht crate and therefore is also not yet released on crates.io.

The API is designed to be very simple:

use async_std::task;
use futures_lite::{AsyncReadExt, AsyncWriteExt, StreamExt};
use hyperswarm::{Config, Hyperswarm, HyperswarmStream, TopicConfig};
use std::io;

#[async_std::main]
async fn main() -> io::Result<()> {
    // Bind and initialize the swarm with the default config.
    // On the config you can e.g. set bootstrap addresses.
    let config = Config::default();
    let mut swarm = Hyperswarm::bind(config).await?;

    // A topic is any 32 byte array. Usually, this would be the hash of some identifier.
    // Configuring the swarm for a topic starts to lookup and/or announce this topic
    // and connect to peers that are found.
    let topic = [0u8; 32];
    swarm.configure(topic, TopicConfig::announce_and_lookup());

    // The swarm is a Stream of new HyperswarmStream peer connections.
    // The HyperswarmStream is a wrapper around either a TcpStream or a UtpSocket.
    // Usually you'll want to run some loop over the connection, so let's spawn a task
    // for each connection.
    while let Some(stream) = swarm.next().await {
        task::spawn(on_connection(stream?));
    }

    Ok(())
}

// A HyperswarmStream is AsyncRead + AsyncWrite, so you can use it just
// like a TcpStream. Here, we'll send an initial message and then keep
// reading from the stream until it is closed by the remote.
async fn on_connection(mut stream: HyperswarmStream) -> io::Result<()> {
    stream.write_all(b"hello there").await?;
    let mut buf = vec![0u8; 64];
    loop {
        match stream.read(&mut buf).await {
            Ok(0) => return Ok(()),
            Err(e) => return Err(e),
            Ok(n) => eprintln!("received: {}", std::str::from_utf8(&buf[..n]).unwrap()),
        }
    }
}

See examples/simple.rs for a working example that also runs a bootstrap node. That example can also find and connect to NodeJS peers. To try it out:

cargo run --example simple
# in another terminal
node js/simple.js

Currently, the DHT bootstrap node has to be run from Rust. The Rust implementation does not find peers on a NodeJS bootstrap node.

Roadmap

  • Initial implementation
  • Find peers over the Hyperswarm DHT
    • Both NodeJS and Rust peers are found if connecting to a Rust bootstrap node
    • Fix hyperswarm-dht to work with NodeJS bootstrap nodes
  • Find peers over mDNS
    • Change colmeia-mdns to better fit the architecture here or copy and adapt the mdns code over into the mdns module
  • Connect to peers over TCP
  • Connect to peers over UTP
    • Can connect to peers over UTP
    • Fix issues in libutp-rs - sometimes the connection is not flushed properly

Safety

This crate uses #![deny(unsafe_code)] to ensure everything is implemented in 100% Safe Rust.

Contributing

Want to join us? Check out our "Contributing" guide and take a look at some of these issues:

License

Licensed under either of Apache License, Version 2.0 or MIT license at your option.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this crate by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.
You might also like...
Rust implementation of reactor pattern for I/O resources

I/O reactor Concurrent I/O without rust async problems This repository provides a set of libraries for concurrent access to I/O resources (file, netwo

Pure Rust implementation of Arbitrum sequencer feed reader with built-in transaction decoding and MEV features

Sequencer-Client (WIP 🚧 ) Pure Rust implementation of Arbitrum sequencer feed reader with built-in transaction decoding and MEV features Design Goal

Simple and flexible queue implementation for Rust with support for multiple backends (Redis, RabbitMQ, SQS, etc.)

Omniqueue Omniqueue is an abstraction layer over queue backends for Rust. It includes support for RabbitMQ, Redis streams, and SQS out of the box. The

Another minimal Raft implementation in Rust.

raft-rs Not my first time implementing Raft. I wrote about another implementation in Go I did. But you don't learn a concept well until you've impleme

Ratchet is a fast, robust, lightweight and fully asynchronous implementation of RFC6455 (The WebSocket protocol).

Ratchet 🚧 Ratchet is a fast, robust, lightweight and fully asynchronous implementation of RFC6455 (The WebSocket protocol). Complete with an optional

Incomplete Redis client and server implementation using Tokio - for learning purposes only

mini-redis mini-redis is an incomplete, idiomatic implementation of a Redis client and server built with Tokio. The intent of this project is to provi

Diesel async connection implementation

A async interface for diesel Diesel gets rid of the boilerplate for database interaction and eliminates runtime errors without sacrificing performance

🦀 REST API client implementation for freee, auto-generated from OpenAPI specification.

freee-rs REST API client implementation for freee, auto-generated from OpenAPI specification. Getting Started Add to your Cargo.toml as follows: [depe

Implementation of generic IBC queries in CosmWasm.

CosmWasm IBC Queries Implements generic IBC queries in CosmWasm. This implementation requires the same contract to be deployed on both chains wishing

Comments
  • DHT is not reset after connections are closed?

    DHT is not reset after connections are closed?

    cargo run --example hyperchat -- bootstrap
    cargo run --example hyperchat -- join -t foo
    cargo run --example hyperchat -- join -t foo
    

    This works and connects the two peers. Then close the two peers but keep the bootstrap node running. Then run the peers again. No connections are happening.

    opened by Frando 0
  • Replaces uTP library with async-std-utp

    Replaces uTP library with async-std-utp

    The current libutp-rs uses a C lib and has some use of libc that are only available on Unix platforms. This did not compile on Windows, and this PR proposes a replacement lib on top of utp crate, which is written in Rust.

    The crate has been integrated with async-std, so there is no need to compat with tokio

    opened by bltavares 0
  • [WIP] [Does not compile] Exploring dynamic discovery mechanism

    [WIP] [Does not compile] Exploring dynamic discovery mechanism

    I'm trying to change how the config of the Hyperswarm structure is defined, so we can change both the transport and discovery mechanism when declaring it.

    It has lots of major changes going down this path, so I'm opening this so we can discuss an alternative than this major refactoring.

    opened by bltavares 0
Owner
Franz Heinzmann
friendly tech
Franz Heinzmann
rinflux is Rust based influx client implementation that have been inspired from influx other language implementation, developed with 💖

Unofficial InfluxDB Driver for Rust This library is a work in progress. This means a feature you might need is not implemented yet or could be handled

Workfoxes 1 Apr 7, 2022
A pure Rust database implementation using an append-only B-Tree file format.

nebari nebari - noun - the surface roots that flare out from the base of a bonsai tree Warning: This crate is early in development. The format of the

Khonsu Labs 194 Jan 3, 2023
A firebase HTTP v1 client implementation in Rust using google-authz

firebase-client A firebase HTTP v1 client implementation in Rust using the google_authz library. Example There are two ways to send notifications, one

Mobiltracker 2 Dec 14, 2022
The spatial message broker and database for real-time multiplayer experiences. Official Rust implementation.

WorldQL Server Rust implementation of WorldQL, the spatial message broker and database for real-time multiplayer experiences Setup Instructions ⚠️ Thi

null 214 Jan 2, 2023
Dataloader-rs - Rust implementation of Facebook's DataLoader using async-await.

Dataloader Rust implementation of Facebook's DataLoader using async-await. Documentation Features Batching load requests with caching Batching load re

cksac 229 Nov 27, 2022
An implementation of the tz database for the time-rs Rust crate.

time-tz An implementation of the tz database for the time-rs Rust crate. This implementation is based off of chrono-tz

null 12 Jul 27, 2022
HTTP 2.0 client & server implementation for Rust.

H2 A Tokio aware, HTTP/2 client & server implementation for Rust. More information about this crate can be found in the crate documentation. Features

null 1.1k Dec 30, 2022
A mini kv database demo that using simplified bitcask storage model with rust implementation

A mini kv database demo that using simplified bitcask storage model with rust implementation.

Wancheng Long 17 Nov 28, 2022
Go like sync.WaitGroup implementation in Rust. (sync/async)

WAG Go like sync.WaitGroup implementation in Rust. (sync/async) | Examples | Docs | Latest Note | wag = "0.3.0" How to use, use wag::WaitGroup; let w

Doha Lee 2 Dec 15, 2022
An intentionally-limited Rust implementation of the Redis server with no external dependencies.

lil-redis An intentionally-limited Rust implementation of the Redis server. lil redis is an accessible implementation of a very basic Redis server (wi

Miguel Piedrafita 37 Jan 1, 2023