Fast Hilbert space-filling curve transformation using a LUT

Overview

Fast Hilbert

Build Status doc crates.io usage license

Fast Hilbert 2D curve computation using an efficient Lookup Table (LUT).

hilbert

  • Convert from discrete 2D space to 1D hilbert space and reverse
  • Very fast using an efficient 512 Byte LUT
  • No additional dependencies except for rust std lib.

Benchmarking with criterion shows that fast_hilbert is about 2.5 times faster compared to the fastest 2D hilbert transformation libs written in rust. Benchmarked on a Intel i5-6400 CPU @ 2.70 GHz, 4 Cores with 8 GB RAM:

Library Time Description
fast_hilbert 30 ns Optimized for fast computation in 2D discrete space using an efficient LUT
hilbert_2d 75 ns Also allows other variants such as Moore and LIU
hilbert_curve 85 ns Implements algorithm described on Wikipedia
hilbert 798 ns Allows computation of higher dimensional Hilbert curves
Comments
  • 90 re-orientation on iteration breaks locality property.

    90 re-orientation on iteration breaks locality property.

    I've been using this (awesome) library for generating Hilbert locations in building a map tiling engine. As the README states, for every zoom, the Hilbert curve rotates. A core property of the Hilbert curve is that relative position of items in the curve is maintained as the zoom changes. For example, this property should hold:

    lower_zoom_hilbert = hilbert / (4^n) * (4^n-1)
    

    Or more nicely:

    lower_zoom_hilbert = hilbert >> 2
    

    Because the rotation happens, these nice properties do not apply. I instead have to calculate a Hilbert location at a higher zoom and shift down for a lower zoom tile.

    Is it possible to tweak the LUT so that this rotation does not happen?

    z3 x4 y3 h31 z2 x2 y1 h13 z1 x1 y0 h1

    Here we are with a tile at zoom 1. The hilbert location resolves to 1. Screen Shot 2022-10-28 at 9 12 46 PM

    At zoom 2, the hilbert location resolves to 13 due to the rotation. Screen Shot 2022-10-28 at 9 15 07 PM

    If I were to sort geographic data by their Hilbert location at one zoom, the locality will change for another, negating the static quadtree.

    And then one more zoom in, and the direction of the preceding tile comes from the north. Screen Shot 2022-10-28 at 9 18 38 PM

    opened by hallahan 8
  • Create… disorder.

    Create… disorder.

    There is a compiler intrinsic (and, I believe, also machine instruction) that counts how many zeros there are in the beginning of a given number. Using this, we can get rid of the order parametre, as follows:

    1. Examine how many leading zeros x and y have
    2. Select the smaller of those two (which is equivalent to (x|y).leading_zeros())
    3. Round it down to an even number if it is odd (equivalent to a & !1 interpreted as the amount of useless bits)
    4. Subtract it from the total amount of bits of each coordinate (32 in this case, interpreted as the amount of useful bits) and tada! you have the ideal number for the order parametre. No panics, no asserts, no nothing.

    All that remains to be seen is whether this causes a slow-down, but given that the asserts are gone I'd wager that its speed has at worst remained the same.

    opened by DoubleHyphen 7
  • Benchmarking against, and possibly supplementing, `lindel`

    Benchmarking against, and possibly supplementing, `lindel`

    1. Could you also include the lindel crate in the benchmarks you produce? Mr Chernoch's code has some… inefficiencies, let's say, which I took upon myself to correct. I don't think it'll come anywhere remotely close to your own speed, but I also suspect it'll at least fall within the same ball-park as the other crates.
    2. One of the parametres to your functions is the so-called “order” of the Hilbert curve. Correct me if I'm wrong: I think that in your specific case all that matters is whether the order is an even or odd number. Is this indeed what's happening?
    3. Do you have any idea if the tricks you used here can be generalised for arbitrary dimensions? I suspect that generalising them to eg u8s or u16s would be trivial, but the part about the arbitrary dimensions will take a bit more thought methinks.

    Thanks in advance!

    opened by DoubleHyphen 5
  • Generalise functions for other uints

    Generalise functions for other uints

    OK, so this was way more difficult than I thought, and the code's become a bit uglier as a result. Nonetheless, the functions now ought to function not just for u32s as before but also for u8s, u16s and u64s.

    Admittedly, I'd like it to be refactored at some point so the code can become cleaner. For now, however, this ought to suffice. In the meantime I'll try to think if there are other optimisations we can perform.

    Oh, and also: Seeing as the LUTs for each function were only being used within that one function, I took the liberty of moving them inside those functions.

    opened by DoubleHyphen 2
  • `num-traits` `v0.2.0` dependency is too old

    `num-traits` `v0.2.0` dependency is too old

    num-traits v0.2.0, which fast-hilbert depends on, doesn't support impl PrimInt for u128 but fast-hilbert requires it.

    This caused a compilation error:

    | error[E0277]: the trait bound `u128: PrimInt` is not satisfied
    |   --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/fast_hilbert-1.0.0/src/lib.rs:59:16
    |    |
    | 59 |     type Key = u128;
    |    |                ^^^^ the trait `PrimInt` is not implemented for `u128`
    |    |
    |    = help: the following other types implement trait `PrimInt`:
    |              i16
    |              i32
    |              i64
    |              i8
    |              isize
    |              u16
    |              u32
    |              u64
    |            and 2 others
    | note: required by a bound in `Unsigned::Key`
    

    I'll send a PR to fix this! :rocket:

    opened by finnbear 0
  • Support Hilbert derivatives.

    Support Hilbert derivatives.

    How hard would it be to get the performance gains of fast_hilbert, but also be able to create Xian Liu's variants?

    https://www.sciencedirect.com/science/article/abs/pii/S0096300302008081?via%3Dihub

    Looks like hilbert_2d does this.

    https://crates.io/crates/hilbert_2d

    I wonder if the variants provide significant performance differences regarding the use of the curve as a spatial index?

    enhancement 
    opened by hallahan 2
  • Get rid of `state` to improve performance (…failed?)

    Get rid of `state` to improve performance (…failed?)

    I thought I'd try to figure out a way to get rid of the state parametre in both the LUT's index and the results it outputs. I succeeded in achieving that, but to my astonishment it ended up consistently ~50% slower! I tried optimising things further here and there, but no consistent result came up. Here is the main.rs file that explains how I did all those things; in the lib.rs file you'll see the actual methods I used.

    That said…

    You said in your benchmarks that fast_hilbert is around 2-2.5 times faster than other comparable methods. My own benchmarks show the improvement to be on the order of 8-10 times faster! I strongly suspect that our benchmarks lead to different results. Thus, if this seems interesting to you, feel free to experiment as well and see; otherwise, just close the issue.

    PS: The main reason I ultimately decided to show you the work I did is because I think that this will help illuminate the way to generalise to other dimensions.

    opened by DoubleHyphen 3
Owner
Armin Becher
Armin Becher
X25519 elliptic curve Diffie-Hellman key exchange in pure-Rust, using curve25519-dalek.

x25519-dalek A pure-Rust implementation of x25519 elliptic curve Diffie-Hellman key exchange, with curve operations provided by curve25519-dalek. This

dalek cryptography 252 Dec 26, 2022
Implements ERC-5564 for the bn254 curve using arkworks-rs

erc-5564-bn254 Uses the arkworks-rs suite of libraries, and utilities from rln Usage Note: this scheme should be used with the fork of circom-rln. use

Aaryamann Challani 21 Sep 8, 2023
Multi Party Key Management System (KMS) for Secp256k1 Elliptic curve based digital signatures.

Key Management System (KMS) for curve Secp256k1 Multi Party Key Management System (KMS) for Secp256k1 Elliptic curve based digital signatures. Introdu

[ZenGo X] 61 Dec 28, 2022
Implementation of the BLS12-381 pairing-friendly elliptic curve group

bls12_381 This crate provides an implementation of the BLS12-381 pairing-friendly elliptic curve construction. This implementation has not been review

Zero-knowledge Cryptography in Rust 183 Dec 27, 2022
Rust implementation of {t,n}-threshold ECDSA (elliptic curve digital signature algorithm).

Multi-party ECDSA This project is a Rust implementation of {t,n}-threshold ECDSA (elliptic curve digital signature algorithm). Threshold ECDSA include

[ZenGo X] 706 Jan 5, 2023
Elliptic-curves - Collection of pure Rust elliptic curve implementations (e.g. P-256, P-384, secp256k1)

RustCrypto: Elliptic Curves General purpose Elliptic Curve Cryptography (ECC) support, including types and traits for representing various elliptic cu

Rust Crypto 386 Dec 27, 2022
Implementation of the Grumpkin curve in Rust.

Grumpkin curve implementation in Rust This repository implements the Grumpkin curve for use in Rust, by building off of the code provided by ZCash and

Jules 3 Dec 26, 2022
Elliptic curve cryptography on Soroban.

Elliptic Curve Cryptography on Soroban Contract examples and reusable primitives. Groth 16 verifier. This crate provides a SorobanGroth16Verifier obje

Xycloo Labs 5 Feb 10, 2023
Flexible secp256k1 curve math library.

secp A flexible and secure secp256k1 elliptic curve math library, with constant-time support, and superb ergonomics. secp takes full advantage of Rust

null 7 Nov 1, 2023
Fast & reliable user space NAT64

protomask A user space NAT64 implementation. Protomask started as a challenge to create a NAT64 implementation in a weekend. The goal of protomask is

Evan Pratten 6 Aug 4, 2023
EXPERIMENTAL: Bitcoin Core Prometheus exporter based on User-Space, Statically Defined Tracing and eBPF.

bitcoind-observer An experimental Prometheus metric exporter for Bitcoin Core based on Userspace, Statically Defined Tracing and eBPF. This demo is ba

0xB10C 24 Nov 8, 2022
Safe, fast, small crypto using Rust

THE SOFTWARE IS PROVIDED "AS IS" AND BRIAN SMITH AND THE AUTHORS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES

Brian Smith 3k Jan 2, 2023
A blazingly fast, ShareX uploader coded in Rust (using actix web) which utilizes AES-256-GCM-SIV to securely store uploaded content.

Magnesium Oxide ❔ What is this? Magnesium-Oxide (MGO) is a secure file uploader with support for ShareX. ?? Features ?? Blazingly fast uploads and enc

Nitrogen Development 26 Nov 25, 2022
The fast, light, and robust client for the Ethereum mainnet.

OpenEthereum Fast and feature-rich multi-network Ethereum client. » Download the latest release « Table of Contents Description Technical Overview Bui

OpenEthereum 1.6k Dec 28, 2022
Fast and efficient ed25519 signing and verification in Rust.

ed25519-dalek Fast and efficient Rust implementation of ed25519 key generation, signing, and verification in Rust. Documentation Documentation is avai

dalek cryptography 563 Dec 26, 2022
Sodium Oxide: Fast cryptographic library for Rust (bindings to libsodium)

sodiumoxide |Crate|Documentation|Gitter| |:---:|:-----------:|:--------:|:-----:|:------:|:----:| |||| NaCl (pronounced "salt") is a new easy-to-use h

sodiumoxide 642 Dec 17, 2022
A fast tool to scan prototype pollution vulnerability written in Rust. 🦀

ppfuzz Prototype Pollution Fuzzer A fast tool to scan prototype pollution vulnerability written in Rust. ?? Installation Binary Source Dependencies Us

Dwi Siswanto 410 Dec 27, 2022
A fast and secure multi protocol honeypot.

Medusa A fast and secure multi protocol honeypot that can mimic realistic devices running ssh, telnet, http, https or any other tcp and udp servers. W

Simone Margaritelli 268 Dec 26, 2022
Web-Scale Blockchain for fast, secure, scalable, decentralized apps and marketplaces.

Building 1. Install rustc, cargo and rustfmt. $ curl https://sh.rustup.rs -sSf | sh $ source $HOME/.cargo/env $ rustup component add rustfmt When buil

Solana Foundation 9.8k Jan 3, 2023