Msgpack serialization/deserialization library for Python, written in Rust using PyO3, and rust-msgpack. Reboot of orjson. msgpack.org[Python]

Overview

ormsgpack

PyPI PyPI - Downloads

ormsgpack is a fast msgpack library for Python. It is a fork/reboot of orjson It serializes faster than msgpack-python and deserializes a bit slower (right now). It supports serialization of: dataclass, datetime, numpy, pydantic and UUID instances natively.

Its features and drawbacks compared to other Python msgpack libraries:

  • serializes dataclass instances natively.
  • serializes datetime, date, and time instances to RFC 3339 format, e.g., "1970-01-01T00:00:00+00:00"
  • serializes numpy.ndarray instances natively and faster.
  • serializes pydantic.BaseModel instances natively (disregards the configuration ATM).
  • serializes arbitrary types using a default hook

ormsgpack supports CPython 3.6, 3.7, 3.8, 3.9, and 3.10. ormsgpack does not support PyPy. Releases follow semantic versioning and serializing a new object type without an opt-in flag is considered a breaking change.

ormsgpack is licensed under both the Apache 2.0 and MIT licenses. The repository and issue tracker is github.com/aviramha/ormsgpack, and patches may be submitted there. There is a CHANGELOG available in the repository.

  1. Usage
    1. Install
    2. Quickstart
    3. Serialize
      1. default
      2. option
    4. Deserialize
  2. Types
    1. dataclass
    2. datetime
    3. enum
    4. float
    5. int
    6. numpy
    7. uuid
    8. pydantic
  3. Latency
  4. Questions
  5. Packaging
  6. License

Usage

Install

To install a wheel from PyPI:

pip install --upgrade "pip>=19.3" # manylinux2014 support
pip install --upgrade ormsgpack

Notice that Linux environments with a pip version shipped in 2018 or earlier must first upgrade pip to support manylinux2014 wheels.

To build a wheel, see packaging.

Quickstart

This is an example of serializing, with options specified, and deserializing:

>>> import ormsgpack, datetime, numpy
>>> data = {
    "type": "job",
    "created_at": datetime.datetime(1970, 1, 1),
    "status": "🆗",
    "payload": numpy.array([[1, 2], [3, 4]]),
}
>>> ormsgpack.packb(data, option=ormsgpack.OPT_NAIVE_UTC | ormsgpack.OPT_SERIALIZE_NUMPY)
b'\x84\xa4type\xa3job\xaacreated_at\xb91970-01-01T00:00:00+00:00\xa6status\xa4\xf0\x9f\x86\x97\xa7payload\x92\x92\x01\x02\x92\x03\x04'
>>> ormsgpack.unpackb(_)
{'type': 'job', 'created_at': '1970-01-01T00:00:00+00:00', 'status': '🆗', 'payload': [[1, 2], [3, 4]]}

Serialize

def packb(
    __obj: Any,
    default: Optional[Callable[[Any], Any]] = ...,
    option: Optional[int] = ...,
) -> bytes: ...

packb() serializes Python objects to msgpack.

It natively serializes bytes, str, dict, list, tuple, int, float, bool, dataclasses.dataclass, typing.TypedDict, datetime.datetime, datetime.date, datetime.time, uuid.UUID, numpy.ndarray, and None instances. It supports arbitrary types through default. It serializes subclasses of str, int, dict, list, dataclasses.dataclass, and enum.Enum. It does not serialize subclasses of tuple to avoid serializing namedtuple objects as arrays. To avoid serializing subclasses, specify the option ormsgpack.OPT_PASSTHROUGH_SUBCLASS.

The output is a bytes object containing UTF-8.

The global interpreter lock (GIL) is held for the duration of the call.

It raises MsgpackEncodeError on an unsupported type. This exception message describes the invalid object with the error message Type is not JSON serializable: .... To fix this, specify default.

It raises MsgpackEncodeError on a str that contains invalid UTF-8.

It raises MsgpackEncodeError if a dict has a key of a type other than str or bytes, unless OPT_NON_STR_KEYS is specified.

It raises MsgpackEncodeError if the output of default recurses to handling by default more than 254 levels deep.

It raises MsgpackEncodeError on circular references.

It raises MsgpackEncodeError if a tzinfo on a datetime object is unsupported.

MsgpackEncodeError is a subclass of TypeError. This is for compatibility with the standard library.

default

To serialize a subclass or arbitrary types, specify default as a callable that returns a supported type. default may be a function, lambda, or callable class instance. To specify that a type was not handled by default, raise an exception such as TypeError.

>>> import ormsgpack, decimal
>>>
def default(obj):
    if isinstance(obj, decimal.Decimal):
        return str(obj)
    raise TypeError

>>> ormsgpack.packb(decimal.Decimal("0.0842389659712649442845"))
MsgpackEncodeError: Type is not JSON serializable: decimal.Decimal
>>> ormsgpack.packb(decimal.Decimal("0.0842389659712649442845"), default=default)
b'\xb80.0842389659712649442845'
>>> ormsgpack.packb({1, 2}, default=default)
ormsgpack.MsgpackEncodeError: Type is not msgpack serializable: set

The default callable may return an object that itself must be handled by default up to 254 times before an exception is raised.

It is important that default raise an exception if a type cannot be handled. Python otherwise implicitly returns None, which appears to the caller like a legitimate value and is serialized:

>>> import ormsgpack, json, rapidjson
>>>
def default(obj):
    if isinstance(obj, decimal.Decimal):
        return str(obj)

>>> ormsgpack.unpackb(ormsgpack.packb({"set":{1, 2}}, default=default))
{'set': None}

option

To modify how data is serialized, specify option. Each option is an integer constant in ormspgack. To specify multiple options, mask them together, e.g., option=ormspgack.OPT_NON_STR_KEYS | ormspgack.OPT_NAIVE_UTC.

OPT_NAIVE_UTC

Serialize datetime.datetime objects without a tzinfo as UTC. This has no effect on datetime.datetime objects that have tzinfo set.

>>> import ormsgpack, datetime
>>> ormsgpack.unpackb(ormsgpack.packb(
        datetime.datetime(1970, 1, 1, 0, 0, 0),
    ))
"1970-01-01T00:00:00"
>>> ormsgpack.unpackb(ormsgpack.packb(
        datetime.datetime(1970, 1, 1, 0, 0, 0),
        option=ormsgpack.OPT_NAIVE_UTC,
    ))
"1970-01-01T00:00:00+00:00"
OPT_NON_STR_KEYS

Serialize dict keys of type other than str. This allows dict keys to be one of str, int, float, bool, None, datetime.datetime, datetime.date, datetime.time, enum.Enum, and uuid.UUID. For comparison, the standard library serializes str, int, float, bool or None by default.

>>> import ormsgpack, datetime, uuid
>>> ormsgpack.packb(
        {uuid.UUID("7202d115-7ff3-4c81-a7c1-2a1f067b1ece"): [1, 2, 3]},
        option=ormsgpack.OPT_NON_STR_KEYS,
    )
>>> ormsgpack.packb(
        {datetime.datetime(1970, 1, 1, 0, 0, 0): [1, 2, 3]},
        option=ormsgpack.OPT_NON_STR_KEYS | ormsgpack.OPT_NAIVE_UTC,
    )

These types are generally serialized how they would be as values, e.g., datetime.datetime is still an RFC 3339 string and respects options affecting it.

This option has the risk of creating duplicate keys. This is because non-str objects may serialize to the same str as an existing key, e.g., {"1970-01-01T00:00:00+00:00": true, datetime.datetime(1970, 1, 1, 0, 0, 0): false}. The last key to be inserted to the dict will be serialized last and a msgpack deserializer will presumably take the last occurrence of a key (in the above, false). The first value will be lost.

OPT_OMIT_MICROSECONDS

Do not serialize the microsecond field on datetime.datetime and datetime.time instances.

>>> import ormsgpack, datetime
>>> ormsgpack.packb(
        datetime.datetime(1970, 1, 1, 0, 0, 0, 1),
    )
>>> ormsgpack.packb(
        datetime.datetime(1970, 1, 1, 0, 0, 0, 1),
        option=ormsgpack.OPT_OMIT_MICROSECONDS,
    )
OPT_PASSTHROUGH_DATACLASS

Passthrough dataclasses.dataclass instances to default. This allows customizing their output but is much slower.

>>> import ormsgpack, dataclasses
>>>
@dataclasses.dataclass
class User:
    id: str
    name: str
    password: str

def default(obj):
    if isinstance(obj, User):
        return {"id": obj.id, "name": obj.name}
    raise TypeError

>>> ormsgpack.packb(User("3b1", "asd", "zxc"))
b'\x83\xa2id\xa33b1\xa4name\xa3asd\xa8password\xa3zxc'
>>> ormsgpack.packb(User("3b1", "asd", "zxc"), option=ormsgpack.OPT_PASSTHROUGH_DATACLASS)
TypeError: Type is not msgpack serializable: User
>>> ormsgpack.packb(
        User("3b1", "asd", "zxc"),
        option=ormsgpack.OPT_PASSTHROUGH_DATACLASS,
        default=default,
    )
b'\x82\xa2id\xa33b1\xa4name\xa3asd'
OPT_PASSTHROUGH_DATETIME

Passthrough datetime.datetime, datetime.date, and datetime.time instances to default. This allows serializing datetimes to a custom format, e.g., HTTP dates:

>>> import ormsgpack, datetime
>>>
def default(obj):
    if isinstance(obj, datetime.datetime):
        return obj.strftime("%a, %d %b %Y %H:%M:%S GMT")
    raise TypeError

>>> ormsgpack.packb({"created_at": datetime.datetime(1970, 1, 1)})
b'\x81\xaacreated_at\xb31970-01-01T00:00:00'
>>> ormsgpack.packb({"created_at": datetime.datetime(1970, 1, 1)}, option=ormsgpack.OPT_PASSTHROUGH_DATETIME)
TypeError: Type is not msgpack serializable: datetime.datetime
>>> ormsgpack.packb(
        {"created_at": datetime.datetime(1970, 1, 1)},
        option=ormsgpack.OPT_PASSTHROUGH_DATETIME,
        default=default,
    )
b'\x81\xaacreated_at\xbdThu, 01 Jan 1970 00:00:00 GMT'

This does not affect datetimes in dict keys if using OPT_NON_STR_KEYS.

OPT_PASSTHROUGH_SUBCLASS

Passthrough subclasses of builtin types to default.

>>> import ormsgpack
>>>
class Secret(str):
    pass

def default(obj):
    if isinstance(obj, Secret):
        return "******"
    raise TypeError

>>> ormsgpack.packb(Secret("zxc"))
b'\xa3zxc'
>>> ormsgpack.packb(Secret("zxc"), option=ormsgpack.OPT_PASSTHROUGH_SUBCLASS)
TypeError: Type is not msgpack serializable: Secret
>>> ormsgpack.packb(Secret("zxc"), option=ormsgpack.OPT_PASSTHROUGH_SUBCLASS, default=default)
b'\xa6******'

This does not affect serializing subclasses as dict keys if using OPT_NON_STR_KEYS.

OPT_SERIALIZE_NUMPY

Serialize numpy.ndarray instances. For more, see numpy.

OPT_SERIALIZE_PYDANTIC

Serialize pydantic.BaseModel instances. Right now it ignores the config (str transformations), support might be added later.

OPT_UTC_Z

Serialize a UTC timezone on datetime.datetime instances as Z instead of +00:00.

>>> import ormsgpack, datetime
>>> ormsgpack.packb(
        datetime.datetime(1970, 1, 1, 0, 0, 0, tzinfo=datetime.timezone.utc),
    )
b'"1970-01-01T00:00:00+00:00"'
>>> ormsgpack.packb(
        datetime.datetime(1970, 1, 1, 0, 0, 0, tzinfo=datetime.timezone.utc),
        option=ormsgpack.OPT_UTC_Z
    )
b'"1970-01-01T00:00:00Z"'

Deserialize

def unpackb(__obj: Union[bytes, bytearray, memoryview], / , option=None) -> Any: ...

unpackb() deserializes msgpack to Python objects. It deserializes to dict, list, int, float, str, bool, bytes and None objects.

bytes, bytearray, memoryview input are accepted.

ormsgpack maintains a cache of map keys for the duration of the process. This causes a net reduction in memory usage by avoiding duplicate strings. The keys must be at most 64 bytes to be cached and 512 entries are stored.

The global interpreter lock (GIL) is held for the duration of the call.

It raises MsgpackDecodeError if given an invalid type or invalid msgpack.

MsgpackDecodeError is a subclass of ValueError.

option

unpackb() supports the OPT_NON_STR_KEYS option, that is similar to original msgpack's strict_map_keys=False. Be aware that this option is considered unsafe and disabled by default in msgpack due to possibility of HashDoS.

Types

dataclass

ormsgpack serializes instances of dataclasses.dataclass natively. It serializes instances 40-50x as fast as other libraries and avoids a severe slowdown seen in other libraries compared to serializing dict.

It is supported to pass all variants of dataclasses, including dataclasses using __slots__, frozen dataclasses, those with optional or default attributes, and subclasses. There is a performance benefit to not using __slots__.

Dataclasses are serialized as maps, with every attribute serialized and in the order given on class definition:

>>> import dataclasses, ormsgpack, typing

@dataclasses.dataclass
class Member:
    id: int
    active: bool = dataclasses.field(default=False)

@dataclasses.dataclass
class Object:
    id: int
    name: str
    members: typing.List[Member]

>>> ormsgpack.packb(Object(1, "a", [Member(1, True), Member(2)]))
b'\x83\xa2id\x01\xa4name\xa1a\xa7members\x92\x82\xa2id\x01\xa6active\xc3\x82\xa2id\x02\xa6active\xc2'

Users may wish to control how dataclass instances are serialized, e.g., to not serialize an attribute or to change the name of an attribute when serialized. ormsgpack may implement support using the metadata mapping on field attributes, e.g., field(metadata={"json_serialize": False}), if use cases are clear.

Performance

alt text

--------------------------------------------------------------------------------- benchmark 'dataclass': 2 tests --------------------------------------------------------------------------------
Name (time in ms)                 Min                 Max                Mean            StdDev              Median               IQR            Outliers       OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_dataclass_ormsgpack       3.4248 (1.0)        7.7949 (1.0)        3.6266 (1.0)      0.3293 (1.0)        3.5815 (1.0)      0.0310 (1.0)          4;34  275.7434 (1.0)         240           1
test_dataclass_msgpack       140.2774 (40.96)    143.6087 (18.42)    141.3847 (38.99)    1.0038 (3.05)     141.1823 (39.42)    0.7304 (23.60)         2;1    7.0729 (0.03)          8           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

datetime

ormsgpack serializes datetime.datetime objects to RFC 3339 format, e.g., "1970-01-01T00:00:00+00:00". This is a subset of ISO 8601 and compatible with isoformat() in the standard library.

>>> import ormsgpack, datetime, zoneinfo
>>> ormsgpack.packb(
    datetime.datetime(2018, 12, 1, 2, 3, 4, 9, tzinfo=zoneinfo.ZoneInfo('Australia/Adelaide'))
)
>>> ormsgpack.unpackb(_)
"2018-12-01T02:03:04.000009+10:30"
>>> ormsgpack.packb(
    datetime.datetime.fromtimestamp(4123518902).replace(tzinfo=datetime.timezone.utc)
)
>>> ormsgpack.unpackb(_)
"2100-09-01T21:55:02+00:00"
>>> ormsgpack.packb(
    datetime.datetime.fromtimestamp(4123518902)
)
>>> ormsgpack.unpackb(_)
"2100-09-01T21:55:02"

datetime.datetime supports instances with a tzinfo that is None, datetime.timezone.utc, a timezone instance from the python3.9+ zoneinfo module, or a timezone instance from the third-party pendulum, pytz, or dateutil/arrow libraries.

datetime.time objects must not have a tzinfo.

>>> import ormsgpack, datetime
>>> ormsgpack.packb(datetime.time(12, 0, 15, 290))
>>> ormsgpack.unpackb(_)
"12:00:15.000290"

datetime.date objects will always serialize.

>>> import ormsgpack, datetime
>>> ormsgpack.packb(datetime.date(1900, 1, 2))
>>> ormsgpack.unpackb(_)
"1900-01-02"

Errors with tzinfo result in MsgpackEncodeError being raised.

It is faster to have ormsgpack serialize datetime objects than to do so before calling packb(). If using an unsupported type such as pendulum.datetime, use default.

To disable serialization of datetime objects specify the option ormsgpack.OPT_PASSTHROUGH_DATETIME.

To use "Z" suffix instead of "+00:00" to indicate UTC ("Zulu") time, use the option ormsgpack.OPT_UTC_Z.

To assume datetimes without timezone are UTC, se the option ormsgpack.OPT_NAIVE_UTC.

enum

ormsgpack serializes enums natively. Options apply to their values.

>>> import enum, datetime, ormsgpack
>>>
class DatetimeEnum(enum.Enum):
    EPOCH = datetime.datetime(1970, 1, 1, 0, 0, 0)
>>> ormsgpack.packb(DatetimeEnum.EPOCH)
>>> ormsgpack.unpackb(_)
"1970-01-01T00:00:00"
>>> ormsgpack.packb(DatetimeEnum.EPOCH, option=ormsgpack.OPT_NAIVE_UTC)
>>> ormsgpack.unpackb(_)
"1970-01-01T00:00:00+00:00"

Enums with members that are not supported types can be serialized using default:

>>> import enum, ormsgpack
>>>
class Custom:
    def __init__(self, val):
        self.val = val

def default(obj):
    if isinstance(obj, Custom):
        return obj.val
    raise TypeError

class CustomEnum(enum.Enum):
    ONE = Custom(1)

>>> ormsgpack.packb(CustomEnum.ONE, default=default)
>>> ormsgpack.unpackb(_)
1

float

ormsgpack serializes and deserializes double precision floats with no loss of precision and consistent rounding.

int

ormsgpack serializes and deserializes 64-bit integers by default. The range supported is a signed 64-bit integer's minimum (-9223372036854775807) to an unsigned 64-bit integer's maximum (18446744073709551615).

numpy

ormsgpack natively serializes numpy.ndarray and individual numpy.float64, numpy.float32, numpy.int64, numpy.int32, numpy.int8, numpy.uint64, numpy.uint32, and numpy.uint8 instances. Arrays may have a dtype of numpy.bool, numpy.float32, numpy.float64, numpy.int32, numpy.int64, numpy.uint32, numpy.uint64, numpy.uintp, or numpy.intp. ormsgpack is faster than all compared libraries at serializing numpy instances. Serializing numpy data requires specifying option=ormsgpack.OPT_SERIALIZE_NUMPY.

>>> import ormsgpack, numpy
>>> ormsgpack.packb(
        numpy.array([[1, 2, 3], [4, 5, 6]]),
        option=ormsgpack.OPT_SERIALIZE_NUMPY,
)
>>> ormsgpack.unpackb(_)
[[1,2,3],[4,5,6]]

The array must be a contiguous C array (C_CONTIGUOUS) and one of the supported datatypes.

If an array is not a contiguous C array or contains an supported datatype, ormsgpack falls through to default. In default, obj.tolist() can be specified. If an array is malformed, which is not expected, ormsgpack.MsgpackEncodeError is raised.

Performance

alt text alt text alt text alt text alt text

---------------------------------------------------------------------------------- benchmark 'numpy float64': 2 tests ---------------------------------------------------------------------------------
Name (time in ms)                      Min                 Max                Mean             StdDev              Median                IQR            Outliers      OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_numpy_ormsgpack[float64]      77.9625 (1.0)       85.2507 (1.0)       79.0326 (1.0)       1.9043 (1.0)       78.5505 (1.0)       0.7408 (1.0)           1;1  12.6530 (1.0)          13           1
test_numpy_msgpack[float64]       511.5176 (6.56)     606.9395 (7.12)     559.0017 (7.07)     44.0661 (23.14)    572.5499 (7.29)     81.2972 (109.75)        3;0   1.7889 (0.14)          5           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


------------------------------------------------------------------------------------- benchmark 'numpy int32': 2 tests -------------------------------------------------------------------------------------
Name (time in ms)                      Min                   Max                  Mean             StdDev                Median                IQR            Outliers     OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_numpy_ormsgpack[int32]       197.8751 (1.0)        210.3111 (1.0)        201.1033 (1.0)       5.1886 (1.0)        198.8518 (1.0)       3.8297 (1.0)           1;1  4.9726 (1.0)           5           1
test_numpy_msgpack[int32]       1,363.8515 (6.89)     1,505.4747 (7.16)     1,428.2127 (7.10)     53.4176 (10.30)    1,425.3516 (7.17)     72.8064 (19.01)         2;0  0.7002 (0.14)          5           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


-------------------------------------------------------------------------------- benchmark 'numpy int8': 2 tests ---------------------------------------------------------------------------------
Name (time in ms)                   Min                 Max                Mean            StdDev              Median                IQR            Outliers     OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_numpy_ormsgpack[int8]     107.8013 (1.0)      113.7336 (1.0)      109.0364 (1.0)      1.7805 (1.0)      108.3574 (1.0)       0.4066 (1.0)           1;2  9.1712 (1.0)          10           1
test_numpy_msgpack[int8]       685.4149 (6.36)     703.2958 (6.18)     693.2396 (6.36)     7.9572 (4.47)     691.5435 (6.38)     14.4142 (35.45)         1;0  1.4425 (0.16)          5           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


------------------------------------------------------------------------------------- benchmark 'numpy npbool': 2 tests --------------------------------------------------------------------------------------
Name (time in ms)                       Min                   Max                  Mean             StdDev                Median                IQR            Outliers      OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_numpy_ormsgpack[npbool]        87.9005 (1.0)         89.5460 (1.0)         88.7928 (1.0)       0.5098 (1.0)         88.8508 (1.0)       0.6609 (1.0)           4;0  11.2622 (1.0)          12           1
test_numpy_msgpack[npbool]       1,095.0599 (12.46)    1,176.3442 (13.14)    1,120.5916 (12.62)    32.9993 (64.73)    1,110.4216 (12.50)    38.4189 (58.13)         1;0   0.8924 (0.08)          5           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


--------------------------------------------------------------------------------- benchmark 'numpy uint8': 2 tests ---------------------------------------------------------------------------------
Name (time in ms)                    Min                 Max                Mean             StdDev              Median                IQR            Outliers     OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_numpy_ormsgpack[uint8]     133.1743 (1.0)      134.7246 (1.0)      134.2793 (1.0)       0.4946 (1.0)      134.3120 (1.0)       0.4492 (1.0)           1;1  7.4472 (1.0)           8           1
test_numpy_msgpack[uint8]       727.1393 (5.46)     824.8247 (6.12)     775.7032 (5.78)     34.9887 (70.73)    775.9595 (5.78)     36.2824 (80.78)         2;0  1.2892 (0.17)          5           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

uuid

ormsgpack serializes uuid.UUID instances to RFC 4122 format, e.g., "f81d4fae-7dec-11d0-a765-00a0c91e6bf6".

>>> import ormsgpack, uuid
>>> ormsgpack.packb(uuid.UUID('f81d4fae-7dec-11d0-a765-00a0c91e6bf6'))
>>> ormsgpack.unpackb(_)
"f81d4fae-7dec-11d0-a765-00a0c91e6bf6"
>>> ormsgpack.packb(uuid.uuid5(uuid.NAMESPACE_DNS, "python.org"))
>>> ormsgpack.unpackb(_)
"886313e1-3b8a-5372-9b90-0c9aee199e5d"

Pydantic

alt text ormsgpack serializes pydantic.BaseModel instances natively. Currently it ignores pydantic.BaseModel.Config.

Performance

-------------------------------------------------------------------------------- benchmark 'pydantic': 2 tests ---------------------------------------------------------------------------------
Name (time in ms)                Min                 Max                Mean            StdDev              Median               IQR            Outliers       OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_pydantic_ormsgpack       4.3918 (1.0)       12.6521 (1.0)        4.8550 (1.0)      1.1455 (3.98)       4.6101 (1.0)      0.0662 (1.0)         11;24  205.9727 (1.0)         204           1
test_pydantic_msgpack       124.5500 (28.36)    125.5427 (9.92)     125.0582 (25.76)    0.2877 (1.0)      125.0855 (27.13)    0.2543 (3.84)          2;0    7.9963 (0.04)          8           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Latency

Graphs

alt text alt text alt text alt text alt text alt text alt text alt text

Data

----------------------------------------------------------------------------- benchmark 'canada packb': 2 tests ------------------------------------------------------------------------------
Name (time in ms)                   Min                Max              Mean            StdDev            Median               IQR            Outliers       OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_ormsgpack_packb[canada]     3.5302 (1.0)       3.8939 (1.0)      3.7319 (1.0)      0.0563 (1.0)      3.7395 (1.0)      0.0484 (1.0)         56;22  267.9571 (1.0)         241           1
test_msgpack_packb[canada]       8.8642 (2.51)     14.0432 (3.61)     9.3660 (2.51)     0.5649 (10.03)    9.2983 (2.49)     0.0982 (2.03)         3;11  106.7691 (0.40)        106           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


------------------------------------------------------------------------------- benchmark 'canada unpackb': 2 tests --------------------------------------------------------------------------------
Name (time in ms)                      Min                Max               Mean             StdDev             Median                IQR            Outliers      OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_msgpack_unpackb[canada]       10.1176 (1.0)      62.0466 (1.18)     33.4806 (1.0)      18.8279 (1.0)      46.6582 (1.0)      38.5921 (1.02)         30;0  29.8680 (1.0)          67           1
test_ormsgpack_unpackb[canada]     11.3992 (1.13)     52.6587 (1.0)      34.1842 (1.02)     18.9461 (1.01)     47.6456 (1.02)     37.8024 (1.0)           8;0  29.2533 (0.98)         20           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


----------------------------------------------------------------------------- benchmark 'citm_catalog packb': 2 tests -----------------------------------------------------------------------------
Name (time in ms)                         Min               Max              Mean            StdDev            Median               IQR            Outliers       OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_ormsgpack_packb[citm_catalog]     1.8024 (1.0)      2.1259 (1.0)      1.9487 (1.0)      0.0346 (1.0)      1.9525 (1.0)      0.0219 (1.0)         79;60  513.1650 (1.0)         454           1
test_msgpack_packb[citm_catalog]       3.4195 (1.90)     3.8128 (1.79)     3.6928 (1.90)     0.0535 (1.55)     3.7009 (1.90)     0.0250 (1.14)        47;49  270.7958 (0.53)        257           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


------------------------------------------------------------------------------ benchmark 'citm_catalog unpackb': 2 tests ------------------------------------------------------------------------------
Name (time in ms)                           Min                Max               Mean             StdDev            Median               IQR            Outliers      OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_ormsgpack_unpackb[citm_catalog]     5.6986 (1.0)      46.1843 (1.0)      14.2491 (1.0)      15.9791 (1.0)      6.1051 (1.0)      0.3074 (1.0)           5;5  70.1798 (1.0)          23           1
test_msgpack_unpackb[citm_catalog]       7.2600 (1.27)     56.6642 (1.23)     16.4095 (1.15)     16.3257 (1.02)     7.7364 (1.27)     0.4944 (1.61)        28;29  60.9404 (0.87)        125           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


----------------------------------------------------------------------------------- benchmark 'github packb': 2 tests -----------------------------------------------------------------------------------
Name (time in us)                     Min                 Max                Mean            StdDev              Median               IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_ormsgpack_packb[github]      73.0000 (1.0)      215.9000 (1.0)       80.4826 (1.0)      4.8889 (1.0)       80.3000 (1.0)      1.1000 (1.83)     866;1118       12.4250 (1.0)        6196           1
test_msgpack_packb[github]       103.8000 (1.42)     220.8000 (1.02)     112.8049 (1.40)     4.9686 (1.02)     113.0000 (1.41)     0.6000 (1.0)     1306;1560        8.8649 (0.71)       7028           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


----------------------------------------------------------------------------------- benchmark 'github unpackb': 2 tests -----------------------------------------------------------------------------------
Name (time in us)                       Min                 Max                Mean            StdDev              Median               IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_ormsgpack_unpackb[github]     201.3000 (1.0)      318.5000 (1.0)      219.0861 (1.0)      6.7340 (1.0)      219.1000 (1.0)      1.2000 (1.0)       483;721        4.5644 (1.0)        3488           1
test_msgpack_unpackb[github]       289.8000 (1.44)     436.0000 (1.37)     314.9631 (1.44)     9.4130 (1.40)     315.1000 (1.44)     2.3000 (1.92)      341;557        3.1750 (0.70)       2477           1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------------- benchmark 'twitter packb': 2 tests ---------------------------------------------------------------------------------------
Name (time in us)                        Min                   Max                  Mean             StdDev                Median                IQR            Outliers         OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_ormsgpack_packb[twitter]       820.7000 (1.0)      2,945.2000 (2.03)       889.3791 (1.0)      78.4139 (2.43)       884.2000 (1.0)      12.5250 (1.0)          4;76  1,124.3799 (1.0)         809           1
test_msgpack_packb[twitter]       1,209.3000 (1.47)     1,451.2000 (1.0)      1,301.3615 (1.46)     32.2147 (1.0)      1,306.7000 (1.48)     14.1000 (1.13)      118;138    768.4260 (0.68)        592           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


------------------------------------------------------------------------------ benchmark 'twitter unpackb': 2 tests -----------------------------------------------------------------------------
Name (time in ms)                      Min                Max              Mean            StdDev            Median               IQR            Outliers       OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_ormsgpack_unpackb[twitter]     2.7097 (1.0)      41.1530 (1.0)      3.2721 (1.0)      3.5860 (1.03)     2.8868 (1.0)      0.0614 (1.32)         4;38  305.6098 (1.0)         314           1
test_msgpack_unpackb[twitter]       3.8079 (1.41)     42.0617 (1.02)     4.4459 (1.36)     3.4893 (1.0)      4.1097 (1.42)     0.0465 (1.0)          2;54  224.9267 (0.74)        228           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Reproducing

The above was measured using Python 3.7.9 on Azure Linux VM (x86_64) with ormsgpack 0.2.1 and msgpack 1.0.2.

The latency results can be reproduced using ./scripts/benchmark.sh and graphs using pytest --benchmark-histogram benchmarks/bench_*.

Questions

Why can't I install it from PyPI?

Probably pip needs to be upgraded. pip added support for manylinux2014 in 2019.

Will it deserialize to dataclasses, UUIDs, decimals, etc or support object_hook?

No. This requires a schema specifying what types are expected and how to handle errors etc. This is addressed by data validation libraries a level above this.

Will it support PyPy?

If someone implements it well.

Packaging

To package ormsgpack requires Rust on the nightly channel and the maturin build tool. maturin can be installed from PyPI or packaged as well. This is the simplest and recommended way of installing from source, assuming rustup is available from a package manager:

rustup default nightly
pip wheel --no-binary=ormsgpack ormsgpack

This is an example of building a wheel using the repository as source, rustup installed from upstream, and a pinned version of Rust:

pip install maturin
curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain nightly-2021-05-25 --profile minimal -y
maturin build --no-sdist --release --strip --manylinux off
ls -1 target/wheels

Problems with the Rust nightly channel may require pinning a version. nightly-2021-05-25 is known to be ok.

ormsgpack is tested for amd64 and aarch64 on Linux, macOS, and Windows. It may not work on 32-bit targets. It has recommended RUSTFLAGS specified in .cargo/config so it is recommended to either not set RUSTFLAGS or include these options.

There are no runtime dependencies other than libc.

License

orjson was written by ijl <[email protected]>, copyright 2018 - 2021, licensed under both the Apache 2 and MIT licenses.

ormsgpack was forked from orjson and is maintained by Aviram Hassan <[email protected]>, licensed same as orjson.

Comments
  • Passthrough int and tuple

    Passthrough int and tuple

    We have developped some functionality for ormsgpack that is currently not supported, but you might want to support:

    This PR adds:

    • Optional passthrough of Python integers, such that also ints larger than 64 bits can be serialized using a default function. Currently they can not be serialized in any way, and by adding this option, the library doesn't get more complicated, but serialization of larger integers is allowed. (Updated tests accordingly)
    • Optional passthrough of Python tuples, such that they can also be deserialized as tuples. Again, no complications for the library, but just leaving it to the default option. (Updated tests accordingly)
    • An OPT_NO_OPT constant, for convenience of the user (with value 0).
    • Fixed typing of the option parameter, since in fact only ints are allowed, and None-values give errors.

    Please let me know if you would like to add (part of) this functionality.

    opened by TariqTNO 8
  • Fix: Drop python 3.6 and support 3.11

    Fix: Drop python 3.6 and support 3.11

    Changelog:
    • Add .idea to ignored files.
    • Removed duplicate of Programming Language :: Python :: 3.10.
    • Add information about Python 3.11 support.
    • Add Python 3.11 to release.yml in github actions.
    • Add support for maturin 0.13.
    • Fix Clippy warnings and errors.
    • Update ahash, pyo3 and pyo3-build-config.
    • Change timezone which has different hours.
    • Fix Github actions pip update.

    #128

    opened by onegreyonewhite 7
  • BIGINT passthrough semantics

    BIGINT passthrough semantics

    The new BIGINT_PASSTHROUGH in 1.1.0 seems to be a bit confusing. Originally I thought it uses passthrough for the ints ormsgpack simply rejected before but it turns out that it also passes through u64 which previously was efficiently converted to the uint64 msgpack format. I think the library should use msgpack formats if possible and the passthrough should extend that functionality instead of changing it. So I propose that passthrough would happen only if the int can't fit into either int64 or uint64, which means it can't be converted to a native msgpack type and needs to be encoded.

    opened by pejter 5
  • Python 3.11 builds

    Python 3.11 builds

    Python 3.11 released!

    If nothing blocks it, please, post wheels for python 3.11. I built it on the master branch and everything was built and works successfully.

    opened by onegreyonewhite 3
  • Broken typing in latest (0.3.2)

    Broken typing in latest (0.3.2)

    I saw you updated maturin in #23 so it can generate type stubs for you but there was no release build triggered and as such the latest version on PyPI still has broken types, because it's missing the "py.typed" file.

    I don't know if you'd prefer to include the "py.typed" file or remove the .pyi stub and let maturin generate everything, but either way it'd be nice to release a fixed version. If you let me know which you'd prefer I can make a quick PR.

    opened by pejter 3
  • Bump serde from 1.0.137 to 1.0.151

    Bump serde from 1.0.137 to 1.0.151

    Bumps serde from 1.0.137 to 1.0.151.

    Release notes

    Sourced from serde's releases.

    v1.0.151

    • Update serde::{ser,de}::StdError to re-export core::error::Error when serde is built with feature="std" off and feature="unstable" on (#2344)

    v1.0.150

    • Relax some trait bounds from the Serialize impl of HashMap and BTreeMap (#2334)
    • Enable Serialize and Deserialize impls of std::sync::atomic types on more platforms (#2337, thanks @​badboy)

    v1.0.149

    • Relax some trait bounds from the Serialize impl of BinaryHeap, BTreeSet, and HashSet (#2333, thanks @​jonasbb)

    v1.0.148

    • Support remote derive for generic types that have private fields (#2327)

    v1.0.147

    • Add serde::de::value::EnumAccessDeserializer which transforms an EnumAccess into a Deserializer (#2305)

    v1.0.146

    v1.0.145

    v1.0.144

    • Change atomic ordering used by Serialize impl of atomic types to match ordering used by Debug impl of those same types (#2263, thanks @​taiki-e)

    v1.0.143

    • Invert build.rs cfgs in serde_test to produce the most modern configuration in the default case (#2253, thanks @​taiki-e)

    v1.0.142

    • Add keywords to crates.io metadata

    v1.0.141

    • Add no-std category to crates.io metadata

    v1.0.140

    • Invert serde_derive cfgs to convenience non-Cargo builds on a modern toolchain (#2251, thanks @​taiki-e)

    v1.0.139

    • Add new constructor function for all IntoDeserializer impls (#2246)

    v1.0.138

    • Documentation improvements
    Commits
    • 44bf363 Release 1.0.151
    • f261184 Merge pull request #2344 from dtolnay/coreerror
    • df40f80 Make StdError identical to core::error::Error on feature="unstable"
    • e7060ba Merge pull request #2342 from atouchet/badges
    • d98f0ee Update build status badge
    • 4f157a8 Prevent build.rs rerunning unnecessarily on all source changes
    • d493649 Release 1.0.150
    • 0e947e6 Merge pull request #2338 from serde-rs/atomic
    • 9249dab Deduplicate atomic_impl macro calls
    • 7440e56 Deduplicate atomic_impl macro implementations
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump serde from 1.0.137 to 1.0.150

    Bump serde from 1.0.137 to 1.0.150

    Bumps serde from 1.0.137 to 1.0.150.

    Release notes

    Sourced from serde's releases.

    v1.0.150

    • Relax some trait bounds from the Serialize impl of HashMap and BTreeMap (#2334)
    • Enable Serialize and Deserialize impls of std::sync::atomic types on more platforms (#2337, thanks @​badboy)

    v1.0.149

    • Relax some trait bounds from the Serialize impl of BinaryHeap, BTreeSet, and HashSet (#2333, thanks @​jonasbb)

    v1.0.148

    • Support remote derive for generic types that have private fields (#2327)

    v1.0.147

    • Add serde::de::value::EnumAccessDeserializer which transforms an EnumAccess into a Deserializer (#2305)

    v1.0.146

    v1.0.145

    v1.0.144

    • Change atomic ordering used by Serialize impl of atomic types to match ordering used by Debug impl of those same types (#2263, thanks @​taiki-e)

    v1.0.143

    • Invert build.rs cfgs in serde_test to produce the most modern configuration in the default case (#2253, thanks @​taiki-e)

    v1.0.142

    • Add keywords to crates.io metadata

    v1.0.141

    • Add no-std category to crates.io metadata

    v1.0.140

    • Invert serde_derive cfgs to convenience non-Cargo builds on a modern toolchain (#2251, thanks @​taiki-e)

    v1.0.139

    • Add new constructor function for all IntoDeserializer impls (#2246)

    v1.0.138

    • Documentation improvements
    Commits
    • d493649 Release 1.0.150
    • 0e947e6 Merge pull request #2338 from serde-rs/atomic
    • 9249dab Deduplicate atomic_impl macro calls
    • 7440e56 Deduplicate atomic_impl macro implementations
    • 0d79306 Update atomic_impl macros to have same input syntax in all cfgs
    • 37faaf2 Mention target_has_atomic stabilization
    • 650358f Replace obsolete comment about target_has_atomic support
    • 6159ead Invert use_target_has_atomic cfg
    • 692ac99 Format PR 2337 with rustfmt
    • 86161ce Adjust spacing in some macro matchers
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump serde from 1.0.137 to 1.0.149

    Bump serde from 1.0.137 to 1.0.149

    Bumps serde from 1.0.137 to 1.0.149.

    Release notes

    Sourced from serde's releases.

    v1.0.149

    • Relax some trait bounds from the Serialize impl of BinaryHeap, BTreeSet, and HashSet (#2333, thanks @​jonasbb)

    v1.0.148

    • Support remote derive for generic types that have private fields (#2327)

    v1.0.147

    • Add serde::de::value::EnumAccessDeserializer which transforms an EnumAccess into a Deserializer (#2305)

    v1.0.146

    v1.0.145

    v1.0.144

    • Change atomic ordering used by Serialize impl of atomic types to match ordering used by Debug impl of those same types (#2263, thanks @​taiki-e)

    v1.0.143

    • Invert build.rs cfgs in serde_test to produce the most modern configuration in the default case (#2253, thanks @​taiki-e)

    v1.0.142

    • Add keywords to crates.io metadata

    v1.0.141

    • Add no-std category to crates.io metadata

    v1.0.140

    • Invert serde_derive cfgs to convenience non-Cargo builds on a modern toolchain (#2251, thanks @​taiki-e)

    v1.0.139

    • Add new constructor function for all IntoDeserializer impls (#2246)

    v1.0.138

    • Documentation improvements
    Commits
    • 0353354 Release 1.0.149
    • 34ae042 Merge pull request #2333 from jonasbb/remove-trait-bounds
    • cc128fe Remove some Serialize trait bounds
    • 7766103 Release 1.0.148
    • 30f7c71 Merge pull request #2331 from dtolnay/remote
    • 50354c2 Improve error message on remote derive duplicate generics
    • c4f67e6 Add ui test of duplicate generics in remote derive
    • 0daafe4 Merge pull request #2330 from dtolnay/remote
    • 3702191 Fix Into conversion involving generic remote derive with getter
    • 7328b34 Add test of generic remote derive with getter
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump serde from 1.0.137 to 1.0.148

    Bump serde from 1.0.137 to 1.0.148

    Bumps serde from 1.0.137 to 1.0.148.

    Release notes

    Sourced from serde's releases.

    v1.0.148

    • Support remote derive for generic types that have private fields (#2327)

    v1.0.147

    • Add serde::de::value::EnumAccessDeserializer which transforms an EnumAccess into a Deserializer (#2305)

    v1.0.146

    v1.0.145

    v1.0.144

    • Change atomic ordering used by Serialize impl of atomic types to match ordering used by Debug impl of those same types (#2263, thanks @​taiki-e)

    v1.0.143

    • Invert build.rs cfgs in serde_test to produce the most modern configuration in the default case (#2253, thanks @​taiki-e)

    v1.0.142

    • Add keywords to crates.io metadata

    v1.0.141

    • Add no-std category to crates.io metadata

    v1.0.140

    • Invert serde_derive cfgs to convenience non-Cargo builds on a modern toolchain (#2251, thanks @​taiki-e)

    v1.0.139

    • Add new constructor function for all IntoDeserializer impls (#2246)

    v1.0.138

    • Documentation improvements
    Commits
    • 7766103 Release 1.0.148
    • 30f7c71 Merge pull request #2331 from dtolnay/remote
    • 50354c2 Improve error message on remote derive duplicate generics
    • c4f67e6 Add ui test of duplicate generics in remote derive
    • 0daafe4 Merge pull request #2330 from dtolnay/remote
    • 3702191 Fix Into conversion involving generic remote derive with getter
    • 7328b34 Add test of generic remote derive with getter
    • fabbd2b Merge pull request #2329 from dtolnay/safety
    • 6814f97 Revert Buf::as_str safety change from PR 2319
    • 4ea403c Merge pull request #2328 from dtolnay/remote
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump once_cell from 1.12.0 to 1.16.0

    Bump once_cell from 1.12.0 to 1.16.0

    Bumps once_cell from 1.12.0 to 1.16.0.

    Changelog

    Sourced from once_cell's changelog.

    1.16.0

    • Add no_std implementation based on critical-section, #195.
    • Deprecate atomic-polyfill feature (use the new critical-section instead)

    1.15.0

    • Increase minimal supported Rust version to 1.56.0.
    • Implement UnwindSafe even if the std feature is disabled.

    1.14.0

    • Add extension to unsync and sync Lazy mut API:
      • force_mut
      • get_mut

    1.13.1

    • Make implementation compliant with strict provenance.
    • Upgrade atomic-polyfill to 1.0

    1.13.0

    • Add Lazy::get, similar to OnceCell::get.

    1.12.1

    • Remove incorrect debug_assert.
    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump serde from 1.0.137 to 1.0.147

    Bump serde from 1.0.137 to 1.0.147

    Bumps serde from 1.0.137 to 1.0.147.

    Release notes

    Sourced from serde's releases.

    v1.0.147

    • Add serde::de::value::EnumAccessDeserializer which transforms an EnumAccess into a Deserializer (#2305)

    v1.0.146

    v1.0.145

    v1.0.144

    • Change atomic ordering used by Serialize impl of atomic types to match ordering used by Debug impl of those same types (#2263, thanks @​taiki-e)

    v1.0.143

    • Invert build.rs cfgs in serde_test to produce the most modern configuration in the default case (#2253, thanks @​taiki-e)

    v1.0.142

    • Add keywords to crates.io metadata

    v1.0.141

    • Add no-std category to crates.io metadata

    v1.0.140

    • Invert serde_derive cfgs to convenience non-Cargo builds on a modern toolchain (#2251, thanks @​taiki-e)

    v1.0.139

    • Add new constructor function for all IntoDeserializer impls (#2246)

    v1.0.138

    • Documentation improvements
    Commits
    • f415092 Release 1.0.147
    • 6d00971 Merge pull request #2305 from serde-rs/enumaccessdeserializer
    • 354b48f Add EnumAccessDeserializer to turn EnumAccess into a Deserializer
    • 3fd8e52 Release 1.0.146
    • 142dce0 Touch up PR 2303
    • 6aed101 Merge pull request #2303 from tage64/master
    • e2ccfd9 Remove bad deserialization from sequence to internally tagged newtype variant...
    • a07d794 Update test_suite/tests/test_annotations.rs
    • 90d28fc Serialize and deserialize a tagged newtype variant over unit () as if it was ...
    • 55cf0ac Merge pull request #2297 from serde-rs/output
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump serde from 1.0.137 to 1.0.152

    Bump serde from 1.0.137 to 1.0.152

    Bumps serde from 1.0.137 to 1.0.152.

    Release notes

    Sourced from serde's releases.

    v1.0.152

    • Documentation improvements

    v1.0.151

    • Update serde::{ser,de}::StdError to re-export core::error::Error when serde is built with feature="std" off and feature="unstable" on (#2344)

    v1.0.150

    • Relax some trait bounds from the Serialize impl of HashMap and BTreeMap (#2334)
    • Enable Serialize and Deserialize impls of std::sync::atomic types on more platforms (#2337, thanks @​badboy)

    v1.0.149

    • Relax some trait bounds from the Serialize impl of BinaryHeap, BTreeSet, and HashSet (#2333, thanks @​jonasbb)

    v1.0.148

    • Support remote derive for generic types that have private fields (#2327)

    v1.0.147

    • Add serde::de::value::EnumAccessDeserializer which transforms an EnumAccess into a Deserializer (#2305)

    v1.0.146

    v1.0.145

    v1.0.144

    • Change atomic ordering used by Serialize impl of atomic types to match ordering used by Debug impl of those same types (#2263, thanks @​taiki-e)

    v1.0.143

    • Invert build.rs cfgs in serde_test to produce the most modern configuration in the default case (#2253, thanks @​taiki-e)

    v1.0.142

    • Add keywords to crates.io metadata

    v1.0.141

    • Add no-std category to crates.io metadata

    v1.0.140

    • Invert serde_derive cfgs to convenience non-Cargo builds on a modern toolchain (#2251, thanks @​taiki-e)

    v1.0.139

    • Add new constructor function for all IntoDeserializer impls (#2246)

    v1.0.138

    • Documentation improvements
    Commits
    • ccf9c6f Release 1.0.152
    • b25d0ea Link to Hjson data format
    • 4f4557f Link to bencode data format
    • bf400d6 Link to serde_tokenstream data format
    • 4d2e36d Wrap flexbuffers bullet point to 80 columns
    • df6310e Merge pull request #2347 from dtolnay/docsrs
    • 938ab5d Replace docs.serde.rs links with intra-rustdoc links
    • ef5a0de Point documentation links to docs.rs instead of docs.serde.rs
    • 5d186c7 Opt out -Zrustdoc-scrape-examples on docs.rs
    • 44bf363 Release 1.0.151
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • Bump ryu from 1.0.10 to 1.0.12

    Bump ryu from 1.0.10 to 1.0.12

    Bumps ryu from 1.0.10 to 1.0.12.

    Release notes

    Sourced from ryu's releases.

    1.0.12

    • Documentation improvements

    1.0.11

    • Add categories and keywords to crates.io metadata
    Commits
    • 0c6d5f6 Release 1.0.12
    • 55da97f Update build status badge
    • f5be563 Time out workflows after 45 minutes
    • 3bb780b Add a funding file
    • 48f3afb Resolve unnecessary_cast clippy lint
    • ee53dc9 Remove default package.readme metadata from Cargo.toml
    • 1fd9d37 GitHub Workflows security hardening
    • ca9bc4f Release 1.0.11
    • ccafc0b Update keywords in crates.io metadata
    • 9470700 Add no-std category to crates.io metadata
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • Bump itoa from 1.0.2 to 1.0.5

    Bump itoa from 1.0.2 to 1.0.5

    Bumps itoa from 1.0.2 to 1.0.5.

    Release notes

    Sourced from itoa's releases.

    1.0.5

    • Documentation improvements

    1.0.4

    • Add no-panic feature to confirm no panicking codepaths (#38)

    1.0.3

    • Add categories and keywords to crates.io metadata
    Commits
    • ef4faed Release 1.0.5
    • b74302f Update build status badge
    • e035699 Time out workflows after 45 minutes
    • faae88d Release 1.0.4
    • 29aa8d7 Merge pull request #38 from dtolnay/nopanic
    • 91ba6e8 Add no-panic feature to confirm no panicking codepaths
    • 228add3 Remove default package.readme metadata from Cargo.toml
    • 8c4881a GitHub Workflows security hardening
    • ccd581e Release 1.0.3
    • c9010e1 Update keywords in crates.io metadata
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • Project status

    Project status

    Hi everyone, I'm not actively maintaining this repository. I will be reviewing/accepting incoming PRs, but no planned maintenance or improvements will be made by me in the near future. If anyone wants to become a maintainer, ping me over aviramyhassan on gmail.

    opened by aviramha 0
  • Bump rmp-serde from 1.1.0 to 1.1.1

    Bump rmp-serde from 1.1.0 to 1.1.1

    Bumps rmp-serde from 1.1.0 to 1.1.1.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • Bump ahash from 0.7.6 to 0.8.2

    Bump ahash from 0.7.6 to 0.8.2

    Bumps ahash from 0.7.6 to 0.8.2.

    Release notes

    Sourced from ahash's releases.

    v0.8.2

    Patch release. Reverts a change that was causing compilation problems on WASM and on certain no-std environments. tkaitchuck/aHash#145

    Version 0.8.1

    • Fix compiler error on wasm with default features.
    • Significant improvements to speed on unusual architectures (Things other than: x86_64, aarch64, mips64, powerpc64, riscv64gc, or s390x)
    • Improved documentation
    • Added support for into_keys and into_values to AHashMap
    • Fix AVR support by using atomic-polyfill
    • Simplify the way that flags work

    Version 0.8.0

    • Improve the API from 0.7 and may require code changes for some applications.
    • Removes the build time detection of the availability of a runtime random number generator and instead relies on a cargo feature runtime-rng which is on by default.
    • There are no changes to the hashing algorithm
    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
Releases(1.2.4)
  • 1.2.4(Nov 16, 2022)

  • 1.2.3(Jun 26, 2022)

  • 1.2.2(Apr 19, 2022)

  • 1.2.1(Mar 1, 2022)

  • 1.2.0(Feb 14, 2022)

    1.2.0 - 14/2/2022

    Changed

    • Extended int passthrough to support u64. by @pejter

    Misc

    • Updated README to include new options. by @ThomasTNO
    • Updated dependencies
    • Renamed in numpy.rs from_parent to to_children to fix new lint rules
    Source code(tar.gz)
    Source code(zip)
  • 1.1.0(Jan 8, 2022)

    1.1.0 - 8/1/2022

    Added

    • Add optional passthrough for tuples. by @TariqTNO
    • Add optional passthrough for ints, that do not fit into an i64. by @TariqTNO

    Changed

    • opt parameter can be None.

    Misc

    • Updated dependencies.
    • Dropped 3.6 CI/CD.
    • Added macOS universal builds (M1)
    Source code(tar.gz)
    Source code(zip)
  • 1.0.3(Dec 18, 2021)

  • 1.0.2(Oct 26, 2021)

  • 1.0.1(Oct 13, 2021)

    1.0.1 - 13/10/2021

    Fixed

    • Decrement refcount for numpy PyArrayInterface. by @ilj.
    • Fix serialization of dataclass inheriting from abc.ABC and using __slots__. by @ilj

    Changed

    • Updated dependencies.
    • find_str_kind test for 4-byte before latin1. by @ilj
    Source code(tar.gz)
    Source code(zip)
  • 1.0.0(Aug 31, 2021)

    1.0.0 - 31/8/2021

    Changed

    • Aligned to orjson's flags and features of SIMD. Didn't include the stable compilation part as seems unnecessary.

    Misc

    • Bumped serde, pyo3.
    • Fixed pyproject.toml to work with newest maturin version.
    Source code(tar.gz)
    Source code(zip)
  • 0.3.6(Aug 24, 2021)

  • 0.3.5(Aug 5, 2021)

  • 0.3.4(Jul 27, 2021)

  • 0.3.3(Jul 23, 2021)

    0.3.3 - 23/7/2021

    Misc

    • Refactored adding objects to the module, creating a __all__ object similar to the way PyO3 creates. This solves an issue with upgrading to new maturin version.
    • Changed < Py3.7 implementation to use automatic range inclusion.
    • Added test to validate correct Python method flags are used on declare.
    • Changed to use PyO3 configurations instead of our own. PR #25 by @pejter.
    Source code(tar.gz)
    Source code(zip)
  • 0.3.2(Jul 13, 2021)

    0.3.2 - 13/7/2021

    Fixed

    • Fix memory leak serializing datetime.datetime with tzinfo. (Copied from orjson)

    Changed

    • Update dependencies, PyO3 -> 0.14.1.

    Misc

    • Setup dependabot.
    Source code(tar.gz)
    Source code(zip)
  • 0.3.1(Jun 25, 2021)

    Changed

    • packb of maps and sequences is now almost 2x faster as it leverages known size. PR #18 by @ijl.

    Misc

    • Added scripts/bench_target.py and scripts/profile.sh for easily benchmarking and profiling. Works only on Linux. PR #17 by @ijl.

    Re-release as cargo.toml wasn't updated hence not uploaded to PyPI.

    Source code(tar.gz)
    Source code(zip)
  • 0.3.0(Jun 13, 2021)

    Added

    • unpackb now accepts keyword argument option with argument OPT_NON_STR_KEYS. This option will let ormsgpack unpack dictionaries with non-str keys. Be aware that this option is considered unsafe and disabled by default in msgpack due to possibility of HashDoS.
    • packb now is able to pack dictionaries with tuples as keys. unpackb is able to unpack such dictionaries. Both requires OPT_NON_STR_KEYS.

    Misc

    • Grouped benchmarks in a pattern that should make more sense.
    • Added pydantic docs to README.md
    • Added graphs and benchmark results.
    Source code(tar.gz)
    Source code(zip)
  • 0.2.1(Jun 12, 2021)

    Fixed

    • Depth limit is now enforced for ormsgpack.unpackb - function should be safe for use now.

    Removed

    • Removed OPT_SERIALIZE_UUID from ormsgpack.pyi as it doesn't exist.

    Misc

    • Added scripts/test.sh for running tests.
    • Added benchmarks, modified scripts to match new layout.
    Source code(tar.gz)
    Source code(zip)
  • 0.2.0(Jun 10, 2021)

    Added

    • Add support for serializing pydantic's BaseModel instances using ormsgpack.OPT_SERIALIZE_PYDANTIC.

    Fixed

    • orjson.packb with option argument as ormsgpack.OPT_NON_STR_KEYS serializes bytes key into tuple of integers instead of using bin type. This also resulted in assymetrical packb/unpackb.

    Misc

    • Added --no-index to pip install ormsgpack to avoid installing from PyPI on CI.
    Source code(tar.gz)
    Source code(zip)
  • 0.1.0(Jun 9, 2021)

Owner
Aviram Hassan
Backend Team Leader, Python/Rust/C.
Aviram Hassan
A high performance python technical analysis library written in Rust and the Numpy C API.

Panther A efficient, high-performance python technical analysis library written in Rust using PyO3 and rust-numpy. Indicators ATR CMF SMA EMA RSI MACD

Greg 210 Dec 22, 2022
Robust and Fast tokenizations alignment library for Rust and Python

Robust and Fast tokenizations alignment library for Rust and Python

Yohei Tamura 14 Dec 10, 2022
Rust numeric library with R, MATLAB & Python syntax

Peroxide Rust numeric library contains linear algebra, numerical analysis, statistics and machine learning tools with R, MATLAB, Python like macros. W

Tae Geun Kim 351 Dec 29, 2022
Rust crate to create Anki decks. Based on the python library genanki

genanki-rs: A Rust Crate for Generating Anki Decks With genanki-rs you can easily generate decks for the popular open source flashcard platform Anki.

Yannick Funk 63 Dec 23, 2022
This repository features a simple Kalman filter and RTS smoother (KFS) implementation in Rust by using the ndarray library.

Kalman filter and RTS smoother in Rust (ndarray) This repository features a simple Kalman filter and RTS smoother (KFS) implementation in Rust by usin

SPDEs 3 Dec 1, 2022
Toy library for neural networks in Rust using Vulkan compute shaders

descent Toy library for neural networks in Rust using Vulkan compute shaders. Features Multi-dimensional arrays backed by Vulkan device memory Use Rus

Simon Brown 71 Dec 16, 2022
Locality Sensitive Hashing in Rust with Python bindings

lsh-rs (Locality Sensitive Hashing) Locality sensitive hashing can help retrieving Approximate Nearest Neighbors in sub-linear time. For more informat

Ritchie Vink 65 Jan 2, 2023
Python package to compute levensthein distance in rust

Contents Introduction Installation Usage License Introduction Rust implementation of levensthein distance (https://en.wikipedia.org/wiki/Levenshtein_d

Thibault Blanc 2 Feb 21, 2022
Pyxirr - Rust-powered collection of financial functions for Python.

PyXIRR Rust-powered collection of financial functions. PyXIRR stands for "Python XIRR" (for historical reasons), but contains many other financial fun

Alexander Volkovsky 82 Jan 2, 2023
Rust-port of spotify/annoy as a wrapper for Approximate Nearest Neighbors in C++/Python optimized for memory usage.

Rust-port of spotify/annoy as a wrapper for Approximate Nearest Neighbors in C++/Python optimized for memory usage.

Arthur·Thomas 13 Mar 10, 2022
Rust-port of spotify/annoy as a wrapper for Approximate Nearest Neighbors in C++/Python optimized for memory usage.

Fareast This library is a rust port of spotify/annoy , currently only index serving is supported. It also provides FFI bindings for jvm, dotnet and da

Arthur·Thomas 13 Mar 10, 2022
Python+Rust implementation of the Probabilistic Principal Component Analysis model

Probabilistic Principal Component Analysis (PPCA) model This project implements a PPCA model implemented in Rust for Python using pyO3 and maturin. In

FindHotel 11 Dec 16, 2022
A real-time implementation of "Ray Tracing in One Weekend" using nannou and rust-gpu.

Real-time Ray Tracing with nannou & rust-gpu An attempt at a real-time implementation of "Ray Tracing in One Weekend" by Peter Shirley. This was a per

null 89 Dec 23, 2022
HNSW ANN from the paper "Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs"

hnsw Hierarchical Navigable Small World Graph for fast ANN search Enable the serde feature to serialize and deserialize HNSW. Tips A good default for

Rust Computer Vision 93 Dec 30, 2022
Simple neural network library for classification written in Rust.

Cogent A note I continue working on GPU stuff, I've made some interesting things there, but ultimately it made me realise this is far too monumental a

Jonathan Woollett-Light 41 Dec 25, 2022
RustFFT is a high-performance FFT library written in pure Rust.

RustFFT is a high-performance FFT library written in pure Rust. It can compute FFTs of any size, including prime-number sizes, in O(nlogn) time.

Elliott Mahler 411 Jan 9, 2023
l2 is a fast, Pytorch-style Tensor+Autograd library written in Rust

l2 • ?? A Pytorch-style Tensor+Autograd library written in Rust Installation • Contributing • Authors • License • Acknowledgements Made by Bilal Khan

Bilal Khan 163 Dec 25, 2022
Reinforcement learning library written in Rust

REnforce Reinforcement library written in Rust This library is still in early stages, and the API has not yet been finalized. The documentation can be

Niven Achenjang 20 Jun 14, 2022
🚀 efficient approximate nearest neighbor search algorithm collections library written in Rust 🦀 .

?? efficient approximate nearest neighbor search algorithm collections library written in Rust ?? .

Hora-Search 2.3k Jan 3, 2023