global allocator that provides hooks for tracking allocation events

Overview

tracking-allocator

A GlobalAlloc-compatible allocator implementation that provides the ability to track allocation events.

examples

As allocators are specialized bits of code, we've included an example in the examples/ folder to show how to use tracking_allocator, rather than putting abbreviated snippets in the README. It is extensively documented, and explains the finer points of using this crate, and what can be acheived with it.

The actual Rust-level documentation is present, and should hopefully be clear and concise, but the example is meant to be how you learn to use the crate, with the Rust-level documentation as a rote "what's that type signature again?" style of reference.

When running the example, you should end up seeing output similar to this:

allocation -> addr=0x55e882b744f0 size=80 group_id=Some(0) tags=None
deallocation -> addr=0x55e882b74490
allocation -> addr=0x55e882b74550 size=12 group_id=Some(1) tags=None
allocation -> addr=0x55e882b74570 size=96 group_id=Some(1) tags=None
deallocation -> addr=0x55e882b74550
deallocation -> addr=0x55e882b74570
Comments
  • Use size of wrapped allocation when calling `AllocationTracker` methods.

    Use size of wrapped allocation when calling `AllocationTracker` methods.

    Right now, we explicitly use the wrapped object's layout for reporting, which is technically correct from a "show me the allocation as it was requested". However, for workloads with many smaller allocations, we could be using a significantly higher amount of memory by virtue of our wrapper header.

    Given that the intent is to report allocations faithfully, and as precisely as we reasonably can, we should likely switch to using the size of the wrapped allocation overall.

    opened by tobz 1
  • Write loom test for token registry update behavior.

    Write loom test for token registry update behavior.

    We changed the token registry update logic to avoid using arc-swap as it involved allocations on the read path which could lead to reentrant brokenness. However, the code we switched to is a lot lower-level and more unsafe and gnarly.

    Luckily, it really just depends on normal atomics so it's likely a ripe candidate to test with loom for correctness.

    opened by tobz 1
  • Embed allocation information directly into the allocation itself.

    Embed allocation information directly into the allocation itself.

    (Based on an idea originally crafted by @jswrenn in his fork: https://github.com/jswrenn/tracking-allocator/blob/wild-idea/src/allocator.rs)

    A major problem of the current design is that implementors must go to fairly extreme lengths to correlate deallocations with allocations, including the hoops that must be jumped through, in general, to avoid reentrant (de)allocations in the tracker implementation.

    This PR does two major things:

    • avoids reentrancy in AllocationTracker::allocated/AllocationTracker::deallocated by temporarily short-circuiting the logic to call it when we're already calling it
    • adds the group ID as a field that comes before the requested allocation layout so it can be read in the deallocation path

    Effectively, implementors can now more easily (de)allocate during tracker calls if necessary, and they can fully correlate a deallocation with the original allocation, in terms of group ID. Doing so before would have required literally tracking allocations by pointer address, which was... basically intractable.

    Additionally, we've added AllocationRegistry::untracked which allows (de)allocating outside of the normal tracker codepaths as there may be a desire to allocate after tracking has been enabled, but not have it be attributed to any group.

    opened by tobz 0
  • Prevent infinitely recurring allocation tracking.

    Prevent infinitely recurring allocation tracking.

    Implements the suggestion of using a thread-local variable to prevent infinite recursion when the user's AllocationTracker, itself, allocates.

    Instead of a separate variable, this PR reuses CURRENT_ALLOCATION_TOKEN. In brief, Allocator<A>::{alloc,dealloc} now set CURRENT_ALLOCATION_TOKEN to None, call AllocationTracker::{allocated,deallocated}, then, finally, restore the value of CURRENT_ALLOCATION_TOKEN.

    This approach is very lightweight: the overhead is, essentially, just two additional writes to an &mut Option<AllocationGroupId>. There's no detectable performance regression.

    opened by jswrenn 0
  • When `tracing-compat`, also require `tracing-subscriber/std` feature.

    When `tracing-compat`, also require `tracing-subscriber/std` feature.

    This feature is required to call .extensions_mut on a SpanRef.

    Without this change, I was getting a compile error trying to build tracking-allocator as a dependency.

    opened by jswrenn 0
  • Add `AllocationGroupId`.

    Add `AllocationGroupId`.

    This PR allows users to correlate between the group_id of allocation events and AllocationGroupToken. Specifically:

    • A new AllocationGroupToken::id method turns an AllocationGroupToken into an AllocationGroupId.
    • The group_id parameter of AllocationTracker::allocated is now a AllocationGroupId.

    Unlike AllocationGroupToken, the AllocationGroupId type may be freely copied, compared and hashed.

    opened by jswrenn 0
  • Consider using a drop guard when enabling tracking.

    Consider using a drop guard when enabling tracking.

    In certain cases, leaving tracking enabled before main exits can lead to reentrancy of the tracking infrastructure as cleanup routines as triggered. As a concrete example, if println! is used in the allocation tracker implementation, this causes a panic at program exit:

    • implicit cleanup routine for stdio runs at end of main
    • this routine takes a lock and mutably borrows some internal state from a RefCell
    • a deallocation happens, which drops us into allocation tracking
    • as we println! in the allocation tracker, this drops us back into the print infrastructure
    • the locks guarding stdout/stderr are reentrant, so we still able to take them
    • the code again tries to mutably borrow some internal state, but alas, it is already borrowed
    • gasp! le panic.

    While printing directly in the allocation tracker implementation is not advised, this still represents a general issue where we likely shouldn't be tracking anything past actual program code, which using a drop guard should achieve: we can clear the tracker implementation right before any runtime boilerplate cleanup runs.

    opened by tobz 0
  • Write test to ensure that dropping an allocation group guard doesn't panic if already exited.

    Write test to ensure that dropping an allocation group guard doesn't panic if already exited.

    We recently fixed a bug where dropping the allocation group guard for an already exited group lead to trying to transition to idle when the group was already idle, which caused a panic as it is not correct behavior.

    We need to write a test to back up that change and ensure we don't regress in the future.

    opened by tobz 0
Owner
Toby Lawrence
Software Engineer at @DataDog working on @vectordotdev. Author of @metrics-rs.
Toby Lawrence
The Solid-State Register Allocator

The Solid-State Register Allocator A simple, extremely fast, reverse linear scan register allocator. See the detailed write-up for an in-depth explana

Matt Keeter 66 Dec 13, 2022
Fast, concurrent, arena-based allocator with drop support

blink-alloc Blink-alloc is extremely fast allocator based on the common idea of allocating linearly zipping a cursor through memory chunk and reset ev

Zakarum 55 Mar 5, 2023
Executable memory allocator with support for dual mapping and W^X protection

jit-allocator A simple memory allocator for executable code. Use JitAllocator type to allocate/release memory and virtual_memory module functions to e

playX 5 Jul 5, 2023
A realtime flight tracking program for our Software Engineering 300 class at ERAU

Flight Tracking ERAU SE300 Description Software that allows for weather and plane tracking to facilitate the user in looking at plane paths. Many peop

null 18 Sep 29, 2022
Simple async library for triggering IFTTT events using webhooks.

IFTTT Webhook A simple Rust async library for triggering IFTTT events using webhooks. Installation Installation can be performed using cargo add: carg

Leo Dutra 1 Mar 11, 2021
🥅 Dead simple webhook worker for Sentry to output events in a Discord channel

?? Sentry Webhook Dead simple webhook worker for Sentry to output events in a Discord channel Why? This is just a simple Rust HTTP service to do so, t

Noel 5 Nov 7, 2022
Recognize gestures by touch events

Gesture Recognizer This crate provides abstract API to recognize and handle simple gestures. At now three type of gestures are supported: Move by one

ilya sheprut 4 Jul 24, 2021
Swayidle alternative to handle wayland idle notifications, sleep and lock events in Rust with Lua scripting based configuration language

swayidle-rs This is intended as a replacement of sway's idle management daemon. I use it as a tool to understand rust message passing and state manage

Reza Jelveh 8 Nov 27, 2023
This crate provides a convenient macro that allows you to generate type wrappers that promise to always uphold arbitrary invariants that you specified.

prae This crate provides a convenient macro that allows you to generate type wrappers that promise to always uphold arbitrary invariants that you spec

null 96 Dec 4, 2022
todo-or-die provides procedural macros that act as checked reminders.

todo-or-die provides procedural macros that act as checked reminders.

David Pedersen 552 Dec 24, 2022
Provides two APIs for easily cancelling futures, with the option to fallback to a timeout cancellation

tokio-context Provides two different methods for cancelling futures with a provided handle for cancelling all related futures, with a fallback timeout

Peter Farr 18 Dec 27, 2022
`fugit` provides a comprehensive library of `Duration` and `Instant` for the handling of time in embedded systems, doing all it can at compile time.

fugit fugit provides a comprehensive library of Duration and Instant for the handling of time in embedded systems, doing all it can at compile time. T

Emil Fresk 40 Oct 2, 2022
cooptex provides deadlock-free Mutexes.

cooptex provides deadlock-free Mutexes. The [CoopMutex::lock] method wraps the [std::sync::Mutex] return value with a Result that will request

Shelby Doolittle 12 Oct 10, 2022
Provides a Suricata Eve output for Kafka with Suricate Eve plugin

Suricata Eve Kafka Output Plugin for Suricata 6.0.x This plugin provides a Suricata Eve output for Kafka. Base on suricata-redis-output: https://githu

Center 7 Dec 15, 2022
Provides a wrapper to deserialize clap app using serde.

clap-serde Provides a wrapper to deserialize clap app using serde. API Reference toml const CLAP_TOML: &'static str = r#" name = "app_clap_serde" vers

null 17 Jan 9, 2023
A (mostly) drop-in replacement for Rust's Result that provides backtrace support

Errant A (mostly) drop-in replacement for Rust's Result that provides backtrace support. Please note that Errant is still very early in development an

Joshua Barretto 17 Dec 26, 2022
This repository provides an emulator for iterated prisoner's dilemma.

Iterated Prisoner's Dilemma Emulator Name This repository provides an emulator for iterated prisoner's dilemma. Description You can run the program by

tison 3 Jan 28, 2022
Memory.lol - a tiny web service that provides historical information about social media accounts

memory.lol Overview This project is a tiny web service that provides historical information about social media accounts. It can currently be used to l

Travis Brown 317 Jul 12, 2023
Hooks for Yew, inspired by streamich/react-use and alibaba/hooks.

Yew Hooks Hooks for Yew, inspired by streamich/react-use and alibaba/hooks. Hooks State use_toggle - tracks state of counterparts. use_bool_toggle - t

Jet Li 116 Jan 4, 2023
Easy c̵̰͠r̵̛̠ö̴̪s̶̩̒s̵̭̀-t̶̲͝h̶̯̚r̵̺͐e̷̖̽ḁ̴̍d̶̖̔ ȓ̵͙ė̶͎ḟ̴͙e̸̖͛r̶̖͗ë̶̱́ṉ̵̒ĉ̷̥e̷͚̍ s̷̹͌h̷̲̉a̵̭͋r̷̫̊ḭ̵̊n̷̬͂g̵̦̃ f̶̻̊ơ̵̜ṟ̸̈́ R̵̞̋ù̵̺s̷̖̅ţ̸͗!̸̼͋

Rust S̵̓i̸̓n̵̉ I̴n̴f̶e̸r̵n̷a̴l mutability! Howdy, friendly Rust developer! Ever had a value get m̵̯̅ð̶͊v̴̮̾ê̴̼͘d away right under your nose just when

null 294 Dec 23, 2022