Instance Distance is a fast pure-Rust implementation of the Hierarchical Navigable Small Worlds paper

Overview

Cover logo

Instant Distance: fast HNSW indexing

Build status License: MIT License: Apache 2.0

Instance Distance is a fast pure-Rust implementation of the Hierarchical Navigable Small Worlds paper by Malkov and Yashunin for finding approximate nearest neighbors. This implementation powers the InstantDomainSearch.com backend services used for word vector indexing.

What it does

Instant Distance is an implementation of a fast approximate nearest neighbor search algorithm. The algorithm is used to find the closest point(s) to a given point in a set.

Using the library

Rust

[dependencies]
instant-distance = "0.5.0"

Example

f32 { // Euclidean distance metric (((self.0 - other.0).pow(2) + (self.1 - other.1).pow(2) + (self.2 - other.2).pow(2)) as f32) .sqrt() } } ">
use instant_distance::{Builder, Search};

fn main() {
    let points = vec![Point(255, 0, 0), Point(255, 0, 0), Point(255, 0, 0)];
    let values = vec!["red", "green", "blue"];

    let map = Builder::default().build(points, values);
    let mut search = Search::default();

    let cambridge_blue = Point(163, 193, 173);

    let closest_point = map.search(&cambridge_blue, &mut search).next().unwrap();

    println!("{:?}", closest_point.value);
}

#[derive(Clone, Copy, Debug)]
struct Point(isize, isize, isize);

impl instant_distance::Point for Point {
    fn distance(&self, other: &Self) -> f32 {
        // Euclidean distance metric
        (((self.0 - other.0).pow(2) + (self.1 - other.1).pow(2) + (self.2 - other.2).pow(2)) as f32)
            .sqrt()
    }
}

Testing

Rust:

cargo t -p instant-distance --all-features

Python:

make test-python
Comments
  • Add a translation example

    Add a translation example

    This pull request adds an example of translating words using the aligned word vectors available here: https://fasttext.cc/docs/en/aligned-vectors.html

    As far as I can tell, the script is working as intended. Results are mixed but I think this is as close as we can reasonably get using word vectors.

    > python3.9 translate.py translate hello
    
    Loading indexes from filesystem...
    Language: fr, Translation: hello
    Language: it, Translation: hello
    Language: it, Translation: ciao
    Language: fr, Translation: bonjours
    Language: fr, Translation: bonjour
    Language: fr, Translation: bonsoir
    Language: fr, Translation: !
    Language: fr, Translation: salutations
    Language: it, Translation: buongiorno
    Language: it, Translation: hey
    
    > python3.9 translate.py translate apple
    
    Loading indexes from filesystem...
    Language: it, Translation: apple
    Language: fr, Translation: apple
    Language: it, Translation: raspberry
    Language: it, Translation: iphone
    Language: it, Translation: macintosh
    Language: it, Translation: ipod
    Language: it, Translation: android
    Language: fr, Translation: acorn
    Language: it, Translation: blackberry
    Language: it, Translation: marshmallow
    
    opened by nrempel 6
  • Building index ahead of time lacks information required for future mapping

    Building index ahead of time lacks information required for future mapping

    By building an index ahead of time, the order of the candidates returned are lost. This makes it hard to map an embedding back onto it's originating word if we load the instant-distance instance from a dump.

    Unless I'm missing something, does it make sense to add this information to the bincode dump?

    opened by nrempel 3
  • First cut of python bindings update

    First cut of python bindings update

    A quick addition for python bindings to include HnswMap.

    @djc can you elaborate on your question/comment regarding how to index the values? I didn't quite catch the meaning there.

    opened by nrempel 2
  • Update pyo3 requirement from 0.16 to 0.17

    Update pyo3 requirement from 0.16 to 0.17

    Updates the requirements on pyo3 to permit the latest version.

    Release notes

    Sourced from pyo3's releases.

    PyO3 0.17.1

    This release contains some minor bug fixes for PyO3 0.17.0. In particular the new PyDictItems, PyDictKeys and PyDictValues types are actually accessible!

    Thanks to @​davidhewitt, @​messense and @​PrettyWood for the fixes.

    Changelog

    Sourced from pyo3's changelog.

    [0.17.1] - 2022-08-28

    Fixed

    • Fix visibility of PyDictItems, PyDictKeys, and PyDictValues types added in PyO3 0.17.0.
    • Fix compile failure when using #[pyo3(from_py_with = "...")] attribute on an argument of type Option<T>. #2592
    • Fix clippy redundant-closure lint on **kwargs arguments for #[pyfunction] and #[pymethods]. #2595

    [0.17.0] - 2022-08-23

    Packaging

    • Update inventory dependency to 0.3 (the multiple-pymethods feature now requires Rust 1.62 for correctness). #2492

    Added

    • Add timezone_utc. #1588
    • Implement ToPyObject for [T; N]. #2313
    • Add PyDictKeys, PyDictValues and PyDictItems Rust types. #2358
    • Add append_to_inittab. #2377
    • Add FFI definition PyFrame_GetCode. #2406
    • Add PyCode and PyFrame high level objects. #2408
    • Add FFI definitions Py_fstring_input, sendfunc, and _PyErr_StackItem. #2423
    • Add PyDateTime::new_with_fold, PyTime::new_with_fold, PyTime::get_fold, and PyDateTime::get_fold for PyPy. #2428
    • Accept #[pyo3(name)] on enum variants. #2457
    • Add CompareOp::matches to implement __richcmp__ as the result of a Rust std::cmp::Ordering comparison. #2460
    • Add PySuper type. #2486
    • Support PyPy on Windows with the generate-import-lib feature. #2506
    • Add FFI definitions Py_EnterRecursiveCall and Py_LeaveRecursiveCall. #2511
    • Add PyDict::get_item_with_error. #2536
    • Add #[pyclass(sequence)] option. #2567

    Changed

    • Change datetime constructors taking a tzinfo to take Option<&PyTzInfo> instead of Option<&PyObject>: PyDateTime::new, PyDateTime::new_with_fold, PyTime::new, and PyTime::new_with_fold. #1588
    • Move PyTypeObject::type_object method to the PyTypeInfo trait, and deprecate the PyTypeObject trait. #2287
    • Methods of Py and PyAny now accept impl IntoPy<Py<PyString>> rather than just &str to allow use of the intern! macro. #2312
    • Change the deprecated pyproto feature to be opt-in instead of opt-out. #2322
    • Emit better error messages when #[pyfunction] return types do not implement IntoPy. #2326
    • Require T: IntoPy for impl<T, const N: usize> IntoPy<PyObject> for [T; N] instead of T: ToPyObject. #2326
    • Deprecate the ToBorrowedObject trait. #2333
    • Iterators over PySet and PyDict will now panic if the underlying collection is mutated during the iteration. #2380
    • Iterators over PySet and PyDict will now panic if the underlying collection is mutated during the iteration. #2380
    • Allow #[classattr] methods to be fallible. #2385
    • Prevent multiple #[pymethods] with the same name for a single #[pyclass]. #2399
    • Fixup lib_name when using PYO3_CONFIG_FILE. #2404
    • Add a message to the ValueError raised by the #[derive(FromPyObject)] implementation for a tuple struct. #2414
    • Allow #[classattr] methods to take Python argument. #2456
    • Rework PyCapsule type to resolve soundness issues: #2485
      • PyCapsule::new and PyCapsule::new_with_destructor now take name: Option<CString> instead of &CStr.

    ... (truncated)

    Commits
    • caaf7bb release: 0.17.1
    • 511303a Merge pull request #2599 from davidhewitt/no-main-gh-pages
    • 3e15bb9 gh-pages: stop building guide for main
    • 9e9e913 Merge pull request #2595 from davidhewitt/kwargs-clippy
    • 058af11 pyfunction: fix clippy lint on **kwargs argument
    • 73c8532 Merge pull request #2592 from davidhewitt/issue-2280
    • 9d543b3 pyfunction: fix from_py_with on Option<T> argument
    • f927cdb Merge pull request #2558 from mejrs/cargo
    • 611ecc1 fix: export new dict views types (#2590)
    • c28e919 Merge pull request #2589 from davidhewitt/netlify-redirects
    • Additional commits viewable in compare view

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Update pyo3 requirement from 0.14.1 to 0.15.0

    Update pyo3 requirement from 0.14.1 to 0.15.0

    Updates the requirements on pyo3 to permit the latest version.

    Release notes

    Sourced from pyo3's releases.

    PyO3 0.15.0

    This release of PyO3 brings support for Python 3.10 and PyPy 3.8. In addition, new optional dependencies on anyhow and eyre have been added for easy integration of the popular error-handling libraries with Python code.

    A number of consistency improvements have been made to PyList, PyTuple and PySequence APIs. They now all exclusively use usize- based indexing, and now also support Rust's indexing operator.

    In this release #[pymethods] are now able to implement many magic methods such as __str__ and __repr__, removing the need for #[pyproto] macro implementations. For the 0.15 release series both #[pymethods] and #[pyproto] will be supported; #[pyproto] is expected to be deprecated in the future.

    For full details of all changes, see the CHANGELOG. For help with upgrading, see the migration guide.

    Thank you to everyone who contributed code, documentation, design ideas, bug reports, and feedback.

    Changelog

    Sourced from pyo3's changelog.

    [0.15.0] - 2021-11-03

    Packaging

    • pyo3's Cargo.toml now advertises links = "python" to inform Cargo that it links against libpython. #1819
    • Added optional anyhow feature to convert anyhow::Error into PyErr. #1822
    • Support Python 3.10. #1889
    • Added optional eyre feature to convert eyre::Report into PyErr. #1893
    • Support PyPy 3.8. #1948

    Added

    • Add PyList::get_item_unchecked and PyTuple::get_item_unchecked to get items without bounds checks. #1733
    • Support #[doc = include_str!(...)] attributes on Rust 1.54 and up. #1746
    • Add PyAny::py as a convenience for PyNativeType::py. #1751
    • Add implementation of std::ops::Index<usize> for PyList, PyTuple and PySequence. #1825
    • Add range indexing implementations of std::ops::Index for PyList, PyTuple and PySequence. #1829
    • Add PyMapping type to represent the Python mapping protocol. #1844
    • Add commonly-used sequence methods to PyList and PyTuple. #1849
    • Add as_sequence methods to PyList and PyTuple. #1860
    • Add support for magic methods in #[pymethods], intended as a replacement for #[pyproto]. #1864
    • Add abi3-py310 feature. #1889
    • Add PyCFunction::new_closure to create a Python function from a Rust closure. #1901
    • Add support for positional-only arguments in #[pyfunction]. #1925
    • Add PyErr::take to attempt to fetch a Python exception if present. #1957

    Changed

    • PyList, PyTuple and PySequence's APIs now accepts only usize indices instead of isize. #1733, #1802, #1803
    • PyList::get_item and PyTuple::get_item now return PyResult<&PyAny> instead of panicking. #1733
    • PySequence::in_place_repeat and PySequence::in_place_concat now return PyResult<&PySequence> instead of PyResult<()>, which is needed in case of immutable sequences such as tuples. #1803
    • PySequence::get_slice now returns PyResult<&PySequence> instead of PyResult<&PyAny>. #1829
    • Deprecate PyTuple::split_from. #1804
    • Deprecate PyTuple::slice, new method PyTuple::get_slice added with usize indices. #1828
    • Deprecate FFI definitions PyParser_SimpleParseStringFlags, PyParser_SimpleParseStringFlagsFilename, PyParser_SimpleParseFileFlags when building for Python 3.9. #1830
    • Mark FFI definitions removed in Python 3.10 PyParser_ASTFromString, PyParser_ASTFromStringObject, PyParser_ASTFromFile, PyParser_ASTFromFileObject, PyParser_SimpleParseStringFlags, PyParser_SimpleParseStringFlagsFilename, PyParser_SimpleParseFileFlags, PyParser_SimpleParseString, PyParser_SimpleParseFile, Py_SymtableString, and Py_SymtableStringObject. #1830
    • #[pymethods] now handles magic methods similarly to #[pyproto]. In the future, #[pyproto] may be deprecated. #1864
    • Deprecate FFI definitions PySys_AddWarnOption, PySys_AddWarnOptionUnicode and PySys_HasWarnOptions. #1887
    • Deprecate #[call] attribute in favor of using fn __call__. #1929
    • Fix missing FFI definition _PyImport_FindExtensionObject on Python 3.10. #1942
    • Change PyErr::fetch to panic in debug mode if no exception is present. #1957

    Fixed

    • Fix building with a conda environment on Windows. #1873
    • Fix panic on Python 3.6 when calling Python::with_gil with Python initialized but threading not initialized. #1874
    • Fix incorrect linking to version-specific DLL instead of python3.dll when cross-compiling to Windows with abi3. #1880
    • Fix FFI definition for PyTuple_ClearFreeList incorrectly being present for Python 3.9 and up. #1887

    ... (truncated)

    Commits
    • 4774744 release: 0.15.0
    • 64df791 Merge pull request #1964 from PyO3/pymethods-args
    • 9ce363a guide: add hints for the signature of pymethods protos
    • 39d2b9d Merge pull request #1957 from davidhewitt/fetch-if-set
    • f801c19 err: add PyErr::take
    • 7b9ae8e Clean up Python documentation (#1963)
    • 0f92f28 Merge pull request #1958 from davidhewitt/pymethods-protos-arguments-cleanup
    • 6a3e1e7 macros: clean up protocol argument extraction a bit
    • bfe7086 Merge pull request #1954 from PyO3/feature-fix
    • 50df2c7 Merge pull request #1955 from PyO3/cargo-toml-deps
    • Additional commits viewable in compare view

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Don't index English words

    Don't index English words

    This PR omits English words from the HNSW index for fr and it. The results look a little more accurate:

    > python3.9 translate.py translate apple
    
    Loading indexes from filesystem...
    Language: fr, Translation: pastèque
    Language: fr, Translation: abricot
    Language: it, Translation: nutella
    Language: fr, Translation: cerise
    Language: it, Translation: ciliegia
    Language: it, Translation: palmari
    Language: it, Translation: cocco
    Language: it, Translation: fragola
    Language: fr, Translation: noisettes
    Language: fr, Translation: ananas
    
    
    > python3.9 translate.py translate hello
    
    Loading indexes from filesystem...
    Language: fr, Translation: bonjours
    Language: fr, Translation: bonsoir
    Language: fr, Translation: salutations
    Language: it, Translation: buongiorno
    Language: it, Translation: buonanotte
    Language: fr, Translation: rebonjour
    Language: it, Translation: auguri
    Language: fr, Translation: bonjour,
    Language: it, Translation: buonasera
    Language: it, Translation: chiamatemi
    
    > python3.9 translate.py translate dog
    
    Loading indexes from filesystem...
    Language: it, Translation: cani
    Language: fr, Translation: chiens
    Language: fr, Translation: chienne
    Language: it, Translation: cucciolo
    Language: fr, Translation: chiot
    Language: it, Translation: coniglio
    Language: fr, Translation: cochon
    Language: it, Translation: maialino
    Language: it, Translation: cagnolino
    Language: fr, Translation: canin
    
    opened by nrempel 1
  • Update ordered-float requirement from 2.0 to 3.0

    Update ordered-float requirement from 2.0 to 3.0

    Updates the requirements on ordered-float to permit the latest version.

    Release notes

    Sourced from ordered-float's releases.

    v3.0.0

    • Breaking change: Remove Add/Sub/Mul/Div/Rem impls for &OrderedFloat to fix type inference errors (#91).
    • Breaking change: Update optional schemars dependency to version 0.8.
    • Add NotNan::as_f32 method (#109).
    Commits
    • 96db24e Update README
    • e4b7c07 Version 3.0.0
    • a849e73 Remove Add/Sub/Mul/Div/Rem impls for &OrderedFloat
    • 61975f1 Add lossy conversion for NotNan\<f64> to NotNan\<f32> (#109)
    • 158a3ae Explains needless mut borrow
    • ba12cb5 Adds clippy and fmt check to CI
    • ed9dcd4 Bumps up schemars to the latest version (0.8.8)
    • 77b1457 Removes two needless borrows
    • 6d2f2d2 Version 2.10.0
    • fd3583f impl arbitrary::Arbitrary for NotNan and OrderedFloat.
    • Additional commits viewable in compare view

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • Update pyo3 requirement from 0.15 to 0.16

    Update pyo3 requirement from 0.15 to 0.16

    Updates the requirements on pyo3 to permit the latest version.

    Release notes

    Sourced from pyo3's releases.

    PyO3 0.16.0

    This release contains substantial changes and improvements over PyO3 0.15.1. To support these improvements, support has been dropped for the end-of-life Python 3.6 and Rust versions older than 1.48.

    The pyo3::ffi submodule has been split out into a separate pyo3-ffi crate, so that users who want to use PyO3's Python bindings without any of the safe APIs or macros are able to do so.

    #[pyclass] can now be used on simple "C-like" enums to create Python enums.

    The #[pyproto] macro has been deprecated, and can be disabled by disabling the optional #[pyproto] feature. The "magic methods" such as __repr__ which previously were implemented by #[pyproto] gained support in #[pymethods] in 0.15, and now in PyO3 0.16 #[pymethods] is intended to be the only attribute macro needed to write class method implementations.

    There are numerous other reworks, improvements, and bugfixes.

    For full details of all changes, see the CHANGELOG.

    Thank you to everyone who contributed code, documentation, design ideas, bug reports, and feedback. The following users' commits are included in this release:

    @​adamreichold @​aganders3 @​Amanieu @​aviramha @​birkenfeld @​b05902132 @​CarlKCarlK @​cmpute @​danielhenrymantilla @​davidhewitt @​DSPOM2 @​ghuls @​Gobot1234 @​kevinheavey @​konstin @​mejrs @​messense @​milesgranger @​mrl5 @​parsons20 @​RicoHageman @​saidvandeklundert @​Tom1380 @​vxgmichel

    Changelog

    Sourced from pyo3's changelog.

    [0.16.0] - 2022-02-27

    Packaging

    • Update MSRV to Rust 1.48. #2004
    • Update indoc optional dependency to 1.0. #2004
    • Drop support for Python 3.6, remove abi3-py36 feature. #2006
    • pyo3-build-config no longer enables the resolve-config feature by default. #2008
    • Update inventory optional dependency to 0.2. #2019
    • Drop paste dependency. #2081
    • The bindings found in pyo3::ffi are now a re-export of a separate pyo3-ffi crate. #2126
    • Support PyPy 3.9. #2143

    Added

    • Add PyCapsule type exposing the Capsule API. #1980
    • Add pyo3_build_config::Sysconfigdata and supporting APIs. #1996
    • Add Py::setattr method. #2009
    • Add #[pyo3(crate = "some::path")] option to all attribute macros (except the deprecated #[pyproto]). #2022
    • Enable create_exception! macro to take an optional docstring. #2027
    • Enable #[pyclass] for fieldless (aka C-like) enums. #2034
    • Add buffer magic methods __getbuffer__ and __releasebuffer__ to #[pymethods]. #2067
    • Add support for paths in wrap_pyfunction and wrap_pymodule. #2081
    • Enable wrap_pyfunction! to wrap a #[pyfunction] implemented in a different Rust module or crate. #2091
    • Add PyAny::contains method (in operator for PyAny). #2115
    • Add PyMapping::contains method (in operator for PyMapping). #2133
    • Add garbage collection magic magic methods __traverse__ and __clear__ to #[pymethods]. #2159
    • Add support for from_py_with on struct tuples and enums to override the default from-Python conversion. #2181
    • Add eq, ne, lt, le, gt, ge methods to PyAny that wrap rich_compare. #2175
    • Add Py::is and PyAny::is methods to check for object identity. #2183
    • Add support for the __getattribute__ magic method. #2187

    Changed

    • PyType::is_subclass, PyErr::is_instance and PyAny::is_instance now operate run-time type object instead of a type known at compile-time. The old behavior is still available as PyType::is_subclass_of, PyErr::is_instance_of and PyAny::is_instance_of. #1985
    • Rename some methods on PyErr (the old names are just marked deprecated for now): #2026
      • pytype -> get_type
      • pvalue -> value (and deprecate equivalent instance)
      • ptraceback -> traceback
      • from_instance -> from_value
      • into_instance -> into_value
    • PyErr::new_type now takes an optional docstring and now returns PyResult<Py<PyType>> rather than a ffi::PyTypeObject pointer. #2027
    • Deprecate PyType::is_instance; it is inconsistent with other is_instance methods in PyO3. Instead of typ.is_instance(obj), use obj.is_instance(typ). #2031
    • __getitem__, __setitem__ and __delitem__ in #[pymethods] now implement both a Python mapping and sequence by default. #2065
    • Improve performance and error messages for #[derive(FromPyObject)] for enums. #2068
    • Reduce generated LLVM code size (to improve compile times) for:
    • Add modulo argument to __ipow__ magic method. #2083

    ... (truncated)

    Commits

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • Update parking_lot requirement from 0.11 to 0.12

    Update parking_lot requirement from 0.11 to 0.12

    Updates the requirements on parking_lot to permit the latest version.

    Changelog

    Sourced from parking_lot's changelog.

    parking_lot 0.12.0, parking_lot_core 0.9.0, lock_api 0.4.6 (2022-01-28)

    • The MSRV is bumped to 1.49.0.
    • Disabled eventual fairness on wasm32-unknown-unknown. (#302)
    • Added a rwlock method to report if lock is held exclusively. (#303)
    • Use new asm! macro. (#304)
    • Use windows-rs instead of winapi for faster builds. (#311)
    • Moved hardware lock elision support to a separate Cargo feature. (#313)
    • Removed used of deprecated spin_loop_hint. (#314)

    parking_lot 0.11.2, parking_lot_core 0.8.4, lock_api 0.4.5 (2021-08-28)

    • Fixed incorrect memory orderings on RwLock and WordLock. (#294, #292)
    • Added Arc-based lock guards. (#291)
    • Added workaround for TSan's lack of support for fence. (#292)

    lock_api 0.4.4 (2021-05-01)

    • Update for latest nightly. (#281)

    lock_api 0.4.3 (2021-04-03)

    • Added [Raw]ReentrantMutex::is_owned. (#280)

    parking_lot_core 0.8.3 (2021-02-12)

    • Updated smallvec to 1.6. (#276)

    parking_lot_core 0.8.2 (2020-12-21)

    • Fixed assertion failure on OpenBSD. (#270)

    parking_lot_core 0.8.1 (2020-12-04)

    • Removed deprecated CloudABI support. (#263)
    • Fixed build on wasm32-unknown-unknown. (#265)
    • Relaxed dependency on smallvec. (#266)

    parking_lot 0.11.1, lock_api 0.4.2 (2020-11-18)

    • Fix bounds on Send and Sync impls for lock guards. (#262)
    • Fix incorrect memory ordering in RwLock. (#260)

    lock_api 0.4.1 (2020-07-06)

    • Add data_ptr method to lock types to allow unsafely accessing the inner data without a guard. (#247)

    parking_lot 0.11.0, parking_lot_core 0.8.0, lock_api 0.4.0 (2020-06-23)

    ... (truncated)

    Commits
    • a75875b Release parking_lot 0.12.0, parking_lot_core 0.9.0, lock_api 0.4.6
    • 78a16dd Minor fixes
    • 4d64bc6 Merge pull request #314 from Amanieu/spin_loop
    • a4e4a02 Only run CI for bors and pull requests
    • 930caab Use core::hint::spin_loop instead of the deprecated spin_loop_hint.
    • b3c9290 Merge pull request #313 from Amanieu/hle_feature
    • b271f84 Move hardware lock elision support to a separate Cargo feature
    • 8019473 Merge pull request #311 from clemenswasser/adopt-windows-rs
    • 1f94288 Adopt windows-rs
    • c73dd43 Merge pull request #309 from Amanieu/stable_asm
    • Additional commits viewable in compare view

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • Update pyo3 requirement from 0.13.2 to 0.14.1

    Update pyo3 requirement from 0.13.2 to 0.14.1

    Updates the requirements on pyo3 to permit the latest version.

    Release notes

    Sourced from pyo3's releases.

    PyO3 0.14.1

    This release addresses some incorrect FFI definitions in PyO3 0.14.0 which caused crashes when targeting PyPy. In addition, IntoPy<PyObject> has been implemented for &PathBuf and &OsString for additional convenience when working with the new conversions.

    Thank you to @​alex and @​jameshilliard for the quick reports, and @​birkenfeld and @​messense for the quick fixes!

    Changelog

    Sourced from pyo3's changelog.

    [0.14.1] - 2021-07-04

    Added

    • Implement IntoPy<PyObject> for &PathBuf and &OsString. #1721

    Fixed

    • Fix crashes on PyPy due to incorrect definitions of PyList_SET_ITEM. #1713

    [0.14.0] - 2021-07-03

    Packaging

    • Update num-bigint optional dependency to 0.4. #1481
    • Update num-complex optional dependency to 0.4. #1482
    • Extend hashbrown optional dependency supported versions to include 0.11. #1496
    • Support PyPy 3.7. #1538

    Added

    • Extend conversions for [T; N] to all N using const generics (on Rust 1.51 and up). #1128
    • Add conversions between OsStr/ OsString and Python strings. #1379
    • Add conversions between Path/ PathBuf and Python strings (and pathlib.Path objects). #1379 #1654
    • Add a new set of #[pyo3(...)] attributes to control various PyO3 macro functionality:
      • #[pyo3(from_py_with = "...")] function arguments and struct fields to override the default from-Python conversion. #1411
      • #[pyo3(name = "...")] for setting Python names. #1567
      • #[pyo3(text_signature = "...")] for setting text signature. #1658
    • Add FFI definition PyCFunction_CheckExact for Python 3.9 and later. #1425
    • Add FFI definition Py_IS_TYPE. #1429
    • Add FFI definition _Py_InitializeMain. #1473
    • Add FFI definitions from cpython/import.h.#1475
    • Add tuple and unit struct support for #[pyclass] macro. #1504
    • Add FFI definition PyDateTime_TimeZone_UTC. #1572
    • Add support for #[pyclass(extends=Exception)]. #1591
    • Add PyErr::cause and PyErr::set_cause. #1679
    • Add FFI definitions from cpython/pystate.h. #1687
    • Add wrap_pyfunction! macro to pyo3::prelude. #1695

    Changed

    • Allow only one #[pymethods] block per #[pyclass] by default, to remove the dependency on inventory. Add a multiple-pymethods feature to opt-in the original behavior and dependency on inventory. #1457
    • Change PyTimeAccess::get_fold to return a bool instead of a u8. #1397
    • Deprecate FFI definition PyCFunction_Call for Python 3.9 and up. #1425
    • Deprecate FFI definition PyModule_GetFilename. #1425
    • The auto-initialize feature is no longer enabled by default. #1443
    • Change PyCFunction::new() and PyCFunction::new_with_keywords() to take &'static str arguments rather than implicitly copying (and leaking) them. #1450
    • Deprecate PyModule::call, PyModule::call0, PyModule::call1 and PyModule::get. #1492
    • Add length information to PyBufferErrors raised from PyBuffer::copy_to_slice and PyBuffer::copy_from_slice. #1534
    • Automatically set -undefined and dynamic_lookup linker arguments on macOS with the extension-module feature. #1539

    ... (truncated)

    Commits
    • 3f8c4ad release: 0.14.1
    • fde4211 Merge pull request #1707 from nw0/doc
    • 4ed0383 Merge pull request #1713 from PyO3/pypy_macros
    • f723548 PyPy: the PyList_GET/SET macros are defined as functions, the PyTuple_ ones d...
    • fa7588a Apply suggestions that make doc take the plural form
    • 9cdb6a0 Merge pull request #1712 from messense/pathbuf-into-py
    • eaf5dca Implement IntoPy\<PyObject> for &PathBuf and &OsString
    • 4b88dd1 Merge pull request #1698 from PyO3/release-0.14
    • adf6bdb release: 0.14.0
    • 5715683 Merge pull request #1711 from mejrs/macro
    • Additional commits viewable in compare view

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • Upgrade to GitHub-native Dependabot

    Upgrade to GitHub-native Dependabot

    Dependabot Preview will be shut down on August 3rd, 2021. In order to keep getting Dependabot updates, please merge this PR and migrate to GitHub-native Dependabot before then.

    Dependabot has been fully integrated into GitHub, so you no longer have to install and manage a separate app. This pull request migrates your configuration from Dependabot.com to a config file, using the new syntax. When merged, we'll swap out dependabot-preview (me) for a new dependabot app, and you'll be all set!

    With this change, you'll now use the Dependabot page in GitHub, rather than the Dependabot dashboard, to monitor your version updates, and you'll configure Dependabot through the new config file rather than a UI.

    Your account was configured to allow an unlimited number of open pull requests. This option is no longer supported in the new config file so it has been changed to 99.

    If you've got any questions or feedback for us, please let us know by creating an issue in the dependabot/dependabot-core repository.

    Learn more about migrating to GitHub-native Dependabot

    Please note that regular @dependabot commands do not work on this pull request.

    dependencies 
    opened by dependabot-preview[bot] 0
  • Generalize SIMD distance implementation to n-length vectors

    Generalize SIMD distance implementation to n-length vectors

    Our Rust API is currently completely abstract over types that implement the Point trait:

    pub trait Point: Clone + Sync {
        fn distance(&self, other: &Self) -> f32;
    }
    

    An important part of making instant-distance fast is making the distance() implementation fast, which comes down to using a SIMD implementation. I wrote such an implementation which is specialized for the case of [f32; 300] because that's what we typically use for InstantDomainSearch (since Meta's FastText vectors have 300 elements).

    However, for the Python bindings, we have some other needs. There, we don't really have the opportunity to make use of compile-time generics; the vector length should be run-time property of the vector. Since we probably don't want to dereference through a pointer for every vector element access, this also means we might want to change the Hnsw implementations to hold a Vec<f32> instead of a Vec<[f32; 300]> (for example), without losing the performance benefits of avoiding bounds checking where possible. For the Python API, we should then also do a SIMD distance implementation that can adjust to the size of the vector at run-time, ideally without much performance loss compared to the current, fixed-length implementation.

    opened by djc 0
Owner
Instant Domain Search, Inc.
Shows domain name availability and suggestions as you type.
Instant Domain Search, Inc.
Fast hierarchical agglomerative clustering in Rust.

kodama This crate provides a fast implementation of agglomerative hierarchical clustering. This library is released under the MIT license. The ideas a

Diffeo 61 Oct 7, 2022
A fast and cross-platform Signed Distance Function (SDF) viewer, easily integrated with your SDF library.

SDF Viewer (demo below) A fast and cross-platform Signed Distance Function (SDF) viewer, easily integrated with your SDF library. A Signed Distance Fu

null 39 Dec 21, 2022
Python package to compute levensthein distance in rust

Contents Introduction Installation Usage License Introduction Rust implementation of levensthein distance (https://en.wikipedia.org/wiki/Levenshtein_d

Thibault Blanc 2 Feb 21, 2022
Signed distance functions + Rust (CPU & GPU) = ❤️❤️

sdf-playground Signed distance functions + Rust (CPU & GPU) = ❤️❤️ Platforms: Windows, Mac & Linux. About sdf-playground is a demo showcasing how you

Patryk Wychowaniec 5 Nov 16, 2023
Adjustment Identification Distance: A gadjid for Causal Structure Learning

Adjustment Identification Distance: A ???????????? for Causal Structure Learning This is an early release of ???????????? ?? and feedback is very welc

CausalDisco 6 Feb 29, 2024
Hamming Weight Tree from the paper Online Nearest Neighbor Search in Hamming Space

hwt Hamming Weight Tree from the paper Online Nearest Neighbor Search in Hamming Space To understand how the data structure works, please see the docs

Rust Computer Vision 7 Oct 9, 2021
A small, basical and unoptimized version of RWKV in Rust written by someone with no math or ML knowledge.

Smol Rust RWKV What is it? A simple example of the RWKV approach to language models written in Rust by someone that knows basically nothing about math

Kerfuffle 32 Apr 9, 2023
RustFFT is a high-performance FFT library written in pure Rust.

RustFFT is a high-performance FFT library written in pure Rust. It can compute FFTs of any size, including prime-number sizes, in O(nlogn) time.

Elliott Mahler 411 Jan 9, 2023
Tensors and dynamic neural networks in pure Rust.

Neuronika is a machine learning framework written in pure Rust, built with a focus on ease of use, fast prototyping and performance. Dynamic neural ne

Neuronika 851 Jan 3, 2023
Deep learning superresolution in pure rust

Rusty_SR A Rust super-resolution tool, which when given a low resolution image utilises deep learning to infer the corresponding high resolution image

zza 189 Dec 9, 2022
Automatic differentiation in pure Rust.

Niura is an automatic differentiation library written in Rust. Add niura to your project [dependencies] niura = { git = "https://github.com/taminki/n

null 10 Jun 16, 2022
A tiny embedding database in pure Rust.

tinyvector - a tiny embedding database in pure Rust ✨ Features Tiny: It's in the name. It's literally just an axum server. Extremely easy to customize

Miguel Piedrafita 210 Jul 12, 2023
Small crate to work with URL in miniquad/macroquad.

quad-url This is the crate to work with URL and open links in miniquad/macroquad environment. Web demo. Usage Add this to your Cargo.toml dependencies

ilya sheprut 3 Jun 11, 2022
Small program which groups images based on the GPS position.

gps-cluster This small program will take some pictures in input, and based on the metadata on every image, it will group them by their GPS position, i

Alessio Bandiera 2 Sep 12, 2022
A small game about solving a mystery aboard a train... if there even is one

Train Mystery A small game about solving a mystery aboard a train... if there even is one. ?? Jeu d'enquête gagnant du Palm'Hackaton 2023. ?? A propos

Aloïs RAUTUREAU 4 May 3, 2023
A pure, low-level tensor program representation enabling tensor program optimization via program rewriting

Glenside is a pure, low-level tensor program representation which enables tensor program optimization via program rewriting, using rewriting frameworks such as the egg equality saturation library.

Gus Smith 45 Dec 28, 2022
scalable and fast unofficial osu! server implementation

gamma! the new bancho server for theta! built for scalability and speed configuration configuration is done either through gamma.toml, or through envi

null 3 Jan 7, 2023
A fast, safe and easy to use reinforcement learning framework in Rust.

RSRL (api) Reinforcement learning should be fast, safe and easy to use. Overview rsrl provides generic constructs for reinforcement learning (RL) expe

Thomas Spooner 139 Dec 13, 2022
Fwumious Wabbit, fast on-line machine learning toolkit written in Rust

Fwumious Wabbit is a very fast machine learning tool built with Rust inspired by and partially compatible with Vowpal Wabbit (much love! read more abo

Outbrain 115 Dec 9, 2022