Traits for inspecting memory usage of Rust types

Related tags

Miscellaneous memuse
Overview

memuse

This crate contains traits for measuring the dynamic memory usage of Rust types.

About

Memory-tracking is a common activity in large applications, particularly ones that receive data from a network and store it in memory. By monitoring how much memory is used by different areas of the application, memory pressure can be alleviated by ignoring new packets, or implementing random drop logic for DoS mitigation.

Measuring memory use on the stack is easy, with std::mem::size_of and friends. Measuring memory allocated on the heap is more tricky. Applications can use a custom global allocator to track the memory usage of different areas. This isn't an option for reusable library code however, and the nearest alternative (using custom allocators for individual types) is currently only an experimental feature in nightly Rust (allocator_api).

This crate takes a different approach: it provides traits that library authors can use to expose dynamic memory usage information on their types. By composing these implementations, we gain the ability to query the amount of heap-allocated memory in use by specific instances of types at any point in time, without any changes to the way in which these types are constructed.

Minimum Supported Rust Version

Rust 1.51 or newer.

In the future, we reserve the right to change MSRV (i.e. MSRV is out-of-scope for this crate's SemVer guarantees), however when we do it will be accompanied by a minor version bump.

License

Licensed under either of

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Comments
  • Add `DynamicUsage::dynamic_usage_bounds` method

    Add `DynamicUsage::dynamic_usage_bounds` method

    We can't always precisely know the amount of heap-allocated memory for external types. The new method enables the caller to establish bounds on the actual dynamic memory usage, while DynamicUsage::dynamic_usage now returns a "best estimate" in these cases.

    opened by str4d 0
  • Look at `deepsize`

    Look at `deepsize`

    While looking up the history of the heapsize crate, I found another crate implementing memory usage introspection: https://github.com/Aeledfyr/deepsize

    opened by str4d 0
  • Look at `loupe`

    Look at `loupe`

    While looking up the history of the heapsize crate, I found another more recent crate implementing memory usage introspection: https://github.com/wasmerio/loupe

    opened by str4d 0
  • Import ideas from Servo's `malloc_size_of`

    Import ideas from Servo's `malloc_size_of`

    This crate currently implements a trait-based measurement approach, where trait impls return the amount of memory that their types requested be allocated.

    malloc_size_of is an internal component of Servo (that they built to replace their heapsize crate). The primary difference is that the trait takes an allocator argument, so that the allocator can be queried directly for the size of the allocated memory. This prevents under-counting for small or unaligned allocations (in that the allocator might allocate more memory than is requested). However, for some types it requires an API that not all allocators support, and in the absence of that API, it uses the same calculation approach as heapsize and memuse.

    We should look over malloc_size_of for useful ideas. In particular, I'd like to make it possible for callers to get "real allocation" results if they have access to an allocator of the correct type, but not require it (as I think the calculation approach can be made accurate enough that for large types the simpler API suffices).

    opened by str4d 0
  • Import ideas from `heapsize` crate

    Import ideas from `heapsize` crate

    I searched for existing crates prior to writing this one, and somehow completely missed the heapsize crate from the Servo devs. It takes a very similar approach to this crate, in that it defines a trait that individual library developers would implement for their own types. It is also used by a small but

    However, the crate has been unmaintained since 2019, the Servo devs having abandoned this approach in favour of their internal malloc_size_of crate. Had I discovered heapsize, I probably would have tried extending it, but now that memuse exists, we should take the opportunity to experiment and figure out a design we like (which may or may not stay close to heapsize).

    In any case, we should look over the heapsize crate for useful ideas. In particular, it impls its traits for several more stdlib structs than we currently do.

    opened by str4d 0
  • Add impls for BTreeMap and BTreeSet

    Add impls for BTreeMap and BTreeSet

    We can't determine the number of nodes in the B-tree from the public API, so we estimate the memory usage as the midpoint between the lower and upper bounds.

    opened by str4d 0
Owner
null
GitHub
Community showcase and examples of Actix ecosystem usage.

Actix Examples Curated examples using the Actix ecosystem. Community Showcase Merino: Web service for Firefox Suggest lemmy: A federated alternative t

Actix 2.9k Dec 31, 2022
Annoyed that Rust has many string types? Well it doesn't have to

generic-str The one true string type in Rust! This project intends to be a proof-of-concept for an idea I had a few months back. There is lots of unsa

Conrad Ludgate 40 Apr 9, 2022
Use explicit container types with Scrypto! Leverage the Rust compiler's type checking to increase security and productivity when developing Radix blueprints.

Scrypto Static Types Use explicit container types with Scrypto! Leverage the Rust compiler's type checking to increase security and productivity when

null 7 Aug 5, 2022
Rust types for the OASIS Common Alerting Protocol (CAP)

Rust types for the OASIS Common Alerting Protocol (CAP)

Will Glynn 2 Jun 6, 2022
A rust program to try and detect some types of Hardware Keyloggers.

Hardware Keylogger Detection Warning: Certain Types of Hardware keyloggers can not be detected by this program, Passive Hardware Keyloggers are imposs

null 4 Dec 5, 2022
The most primitive and the fastest implementation of a fixed-size last-in-first-out stack on stack in Rust, for Copy-implementing types

This is the simplest and the fastest (faster than Vec!) implementation of a last-in-first-out stack data structure, on stack, when stack elements are

Yegor Bugayenko 10 Jun 18, 2023
A list of known SS58 account types as an enum.

A list of known SS58 account types as an enum.

Parity Technologies 39 Dec 14, 2022
🪣 Types for a `Vec`'s raw parts

raw-parts A wrapper around the decomposed parts of a Vec<T>. This struct contains the Vec's internal pointer, length, and allocated capacity. RawParts

Artichoke Ruby 3 Sep 1, 2022
An unsafe botched job that doesn't rely on types being 'static lifetime.

An unsafe botched job that doesn't rely on types being 'static lifetime. Will panic if provided a 0 field struct. I will fix this when I figure out how.

twhite 0 Feb 4, 2022
A repository full of manually generated hand curated JSON files, which contain the API Types that the Discord API returns.

Discord API Types A repository full of manually generated hand curated JSON files, which contain the API Types that the Discord API returns. Also did

Unofficial Discord Documentation 1 Sep 16, 2022
Option and Either types with variants known at compile time.

Const Either Some types to allow deciding at compile time if an option contains a value or which variant from the either type is active. This might be

null 1 May 5, 2022
Application that simulates a large grid of Pokémon types fighting each other.

poke-fighting-rust Rust project that simulates a grid of Pokémon fighting with each other. Each Pokémon type is a pixel on the grid and is represented

Ólafur Waage 11 Dec 19, 2022
secmem-proc is a crate designed to harden a process against low-privileged attackers running on the same system trying to obtain secret memory contents of the current process.

secmem-proc is a crate designed to harden a process against low-privileged attackers running on the same system trying to obtain secret memory contents of the current process. More specifically, the crate disables core dumps and tries to disable tracing on unix-like OSes.

null 3 Dec 19, 2022
Allocate memory at compile time!

const-alloc Link to the docs! Allocate memory at compile time! Currently, in stable rust there is no way to dynamically allocate or deallocate memory

Sp00ph 1 Feb 5, 2022
🦀🚀🔥 A blazingly fast and memory-efficient implementation of `if err != nil` 🔥🚀🦀

?????? A blazingly fast and memory-efficient implementation of `if err != nil` ??????

Federico Damián Schonborn 6 Dec 30, 2022
🌋 A very lightweight wrapper around the Vulkan Memory Allocator 🦀

?? vk-mem-alloc-rs A very lightweight wrapper around the Vulkan Memory Allocator ?? [dependencies] vk-mem-alloc = "0.1.1" Simple Vulkan Memory Allocat

Project KML 13 Nov 8, 2022
Hide memory artifacts using ROP and hardware breakpoints.

Description This tool is a simple PoC of how to hide memory artifacts using a ROP chain in combination with hardware breakpoints. The ROP chain will c

Kurosh Dabbagh Escalante 87 Dec 14, 2022
Executable memory allocator with support for dual mapping and W^X protection

jit-allocator A simple memory allocator for executable code. Use JitAllocator type to allocate/release memory and virtual_memory module functions to e

playX 5 Jul 5, 2023
Memory.lol - a tiny web service that provides historical information about social media accounts

memory.lol Overview This project is a tiny web service that provides historical information about social media accounts. It can currently be used to l

Travis Brown 317 Jul 12, 2023