Python module implemented in Rust for counting the number of one bits in a numpy array.

Overview

bit-counter

Package for counting the number of one bits in a numpy array of uint8 values. Implemented as a Python module using Rust, providing high performance counting.

Building

To build this package an installation of Rust and Python with the maturin package is required. The Maturin documentation on maturin local development is a useful reference for more details.

This project is configured by default to target CPUs with the POPCNT instruction. The builds available on PyPI have been built with this configuration. If you require a version for an older CPU without popcnt support, build with the RUSTFLAGS environment variable to exclude the popcnt target-feature. i.e. RUSTFLAGS='-C target-feature=-popcnt' maturin build -r.

Example usage

import numpy as np
from bit_counter import count_ones

arr = np.packbits(np.random.choice([True, False], 1000000))

count_of_true_values = count_ones(arr)

Performance

When built to target a CPU with popcnt support the count_ones method provided is substantially faster than a naive np.unpackbits(arr).sum(). The count_ones method also doesn't require unpacking the bit packed numpy array, so doesn't require any addtional memory to do the calculation.

For example with the following test code for 100 million True/False values that are packed:

import numpy as np
from bit_counter import count_ones

arr = np.packbits(np.random.choice([True, False], 100000000))

Using this package on a mid-2013 MacBook Air (1.7 GHz Intel Core i7) yields:

%timeit count_ones(arr)

45.3 ms ± 2.08 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

Compared to the simple unpack and sum case:

%timeit np.unpackbits(arr).sum()

108 ms ± 6.89 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
You might also like...
Implementation of Monte Carlo PI approximation algorithm in Rust Python bindings

rusty_pi Implementation of Monte Carlo PI approximation algorithm in Rust Python bindings. Time of 100M iterations approximation on Core i7 10th gen:

A simple library to allow for easy use of python from rust.

Rustpy A simple library to allow for easy use of python from rust. Status Currently this library has not received much love (pull requests welcome for

Pyo3 - Rust bindings for the Python interpreter

PyO3 Rust bindings for Python, including tools for creating native Python extension modules. Running and interacting with Python code from a Rust bina

RustPython - A Python Interpreter written in Rust
RustPython - A Python Interpreter written in Rust

RustPython A Python-3 (CPython = 3.9.0) Interpreter written in Rust 🐍 😱 🤘 . Usage Check out our online demo running on WebAssembly. RustPython req

Robust and Fast tokenizations alignment library for Rust and Python
Robust and Fast tokenizations alignment library for Rust and Python

Robust and Fast tokenizations alignment library for Rust and Python Demo: demo Rust document: docs.rs Blog post: How to calculate the alignment betwee

Very experimental Python bindings for the Rust biscuit-auth library

Overview This is a very experimental take on Python bindings for the biscuit_auth Rust library. It is very much a work in progress (limited testing, m

Python bindings for heck, the Rust case conversion library

pyheck PyHeck is a case conversion library (for converting strings to snake_case, camelCase etc). It is a thin wrapper around the Rust library heck. R

Arrowdantic is a small Python library backed by a mature Rust implementation of Apache Arrow

Welcome to arrowdantic Arrowdantic is a small Python library backed by a mature Rust implementation of Apache Arrow that can interoperate with Parquet

lavalink-rs bindings for Python

lavasnek_rs Dev Docs: Main Site | Fallback: GitHub Pages GitHub repo GitLab repo Using the library The library is available on PyPi, and you can insta

Comments
  • rustflags in cargo config?

    rustflags in cargo config?

    I wonder if it would be worth putting something like the following in .cargo/config.toml:

    [build]
    rustflags = ["-C", "target-feature=+popcnt"] 
    

    That way builds would automatically use of popcnt by default and you could simplify the documentation and CI a bit. If you did go down this path you could add a note about the inverse: If someone wants to build for a CPU that doesn't support popcnt then they should build with RUSTFLAGS="-C target-feature=-popcnt"

    opened by wezm 1
Owner
Andrew MacDonald
Andrew MacDonald
Create, open, manage your Python projects with ease, a project aimed to make python development experience a little better

Create, open, manage your Python projects with ease, a project aimed to make python development experience a little better

Dhravya Shah 7 Nov 18, 2022
Easy way to write Node.js module using Rust

node-bindgen Easy way to write native Node.js module using idiomatic Rust Features Easy: Just write idiomatic Rust code, node-bindgen take care of gen

InfinyOn 346 Jan 3, 2023
Another cursed Garry's Mod module. This time, it adds the C preprocessor to Lua scripts

gm_cpreprocessor Another cursed Garry's Mod module. This time, it adds the C preprocessor to Lua scripts. It works by detouring RunStringEx and overri

William 6 Aug 14, 2022
Mod_wasm - an extension module for the Apache HTTP Server (httpd) that enables the usage of WebAssembly (Wasm).

mod_wasm is an extension module for the Apache HTTP Server (httpd) that enables the usage of WebAssembly (Wasm). This module will allow to execute certain tasks in the backend in a very efficient and secure way.

VMware  Labs 67 Dec 21, 2022
Slitter is a C- and Rust-callable slab allocator implemented primarily in Rust, with some C for performance or to avoid unstable Rust features.

Slitter is a less footgunny slab allocator Slitter is a classically structured thread-caching slab allocator that's meant to help write reliable long-

Backtrace Labs 133 Dec 5, 2022
lens implemented in rust

Overview This project contains two crates: lens-rs and lens-rs_derive. The lens-rs provide some definitions of lens. The lens-rs_derive provide the ma

Danube 84 Dec 27, 2022
Rust <-> Python bindings

rust-cpython Rust bindings for the python interpreter. Documentation Cargo package: cpython Copyright (c) 2015-2020 Daniel Grunwald. Rust-cpython is l

Daniel Grunwald 1.7k Dec 29, 2022
Rust bindings for the Python interpreter

PyO3 Rust bindings for Python. This includes running and interacting with Python code from a Rust binary, as well as writing native Python modules. Us

PyO3 7.2k Jan 4, 2023
A script language like Python or Lua written in Rust, with exactly the same syntax as Go's.

A script language like Python or Lua written in Rust, with exactly the same syntax as Go's.

null 1.4k Jan 1, 2023
Rust Python modules for interacting with Metaplex's NFT standard.

Simple Metaplex Metadata Decoder Install the correct Python wheel for your Python version with pip: pip install metaplex_decoder-0.1.0-cp39-cp39-manyl

Samuel Vanderwaal 11 Mar 31, 2022