A memory visualizer in Rust (ptrace + userfaultfd)

Overview

mevi

A memory visualizer for Linux 5.7+

Made for this video: https://www.youtube.com/watch?v=DpnXaNkM9_M

Prerequisite

The vm.unprivileged_userfaultfd sysctl needs to be switched to 1:

$ sudo sysctl -w vm.unprivileged_userfaultfd=1

Doing this effectively "softens" your system to some attacks, so only do this in a VM or if you're reckless, but also, it seems less awful than running mevi + tracees as root. (No, giving the mevi binary CAP_PTRACE isn't enough).

You can technically run a bunch of apps with only user faults, but some fairly basic stuff like cat /hosts will fail with EFAULT without it, so, I'm not making it easy to go that route - if you really know what you're doing you can figure out where to pass the "user faults only" flag.

Usage

Install the mevi executable:

$ just install

(Or, without just, look into the Justfile for the cargo invocation)

To build & serve the frontend, you'll need to install support for the wasm target via rustup:

$ rustup target add wasm32-unknown-unknown

And to have trunk installed.

$ just serve

Open the frontend in your browser: http://localhost:8080

From another terminal, start the program you want to trace via mevi:

$ mevi PROGRAM ARGS

The frontend should connect to http://localhost:5001/stream.

If you're running this on a remote server, you'll need to forward both ports, with SSH for example:

ssh -L 5001:localhost:5001 -L 8080:localhost:8080 your-remote-host

License

This project is primarily distributed under the terms of both the MIT license and the Apache License (Version 2.0).

See LICENSE-APACHE and LICENSE-MIT for details.

FAQ / Troubleshooting

I get EPERM at some point

Did you skip past that sysctl note above?

The RSS numbers don't match up with htop/btop/procmaps etc.

mevi only tracks private+anonymous memory mappings. The discrepancy probably comes from mapped files, and to a lesser extent, shared memory.

I have a tiny program and everything goes by way too fast.

Try sleeping in your loops! Computers go fast noawadays and mevi tries not to slow your program down.

I have a multi-threaded program and it's all wrong

Yeah, sorry about that. userfaultfd events don't have all the info we need, and ptrace observes events out-of-order, so the view of multi-threaded programs gets out-of-sync with the kernel.

Can I run this on a big program?

Sure, Firefox works, with a non-snap version, and with sandbox disabled, like so (THIS IS DANGEROUS, THE SANDBOX IS THERE FOR A REASON).

First let's make sure you don't have firefox running in the background:

$ pkill firefox
# you can do it several times, until `pidof firefox` returns nothing

Then:

$ RUST_LOG=error RUST_BACKTRACE=1 MOZ_DISABLE_CONTENT_SANDBOX=1 MOZ_DISABLE_GMP_SANDBOX=1 MOZ_DISABLE_RDD_SANDBOX=1 MOZ_DISABLE_SOCKET_PROCESS_SANDBOX=1 mevi /usr/lib/firefox/firefox

Does this show backtraces?

No, but you can do that in your fork.

Does this allow travelling back in time?

No, but you can do that in your fork.

Does this have yet another, secret third feature?

Clearly not, but again, you can do that in your fork. This is a research project, I will not be maintaining it beyond "have it run in its current form".

If you want to spin this out into its own product, more power to you, but I'll have already moved on.

Why isn't this published on crates.io?

It's not a library and it's not usable without the frontend anyway. One day stable cargo will let us build wasm artifacts and ship them with the resulting binary, but that day is not today.

Why isn't this using eBPF?

I wanted to see how far I could take ptrace + userfaultfd. I'm interested in exploring eBPF later.

You might also like...
k-mer counter in Rust using the rust-bio and rayon crates

krust is a k-mer counter written in Rust and run from the command line that will output canonical k-mers and their frequency across the records in a f

Experimental Rust tool for generating FFI definitions allowing many other languages to call Rust code

Diplomat is an experimental Rust tool for generating FFI definitions allowing many other languages to call Rust code. With Diplomat, you can simply define Rust APIs to be exposed over FFI and get high-level C, C++, and JavaScript bindings automatically!

Aws-sdk-rust - AWS SDK for the Rust Programming Language

The AWS SDK for Rust This repo contains the new AWS SDK for Rust (the SDK) and its public roadmap. Please Note: The SDK is currently released as a dev

Rust + Yew + Axum + Tauri, full-stack Rust development for Desktop apps.

rust-yew-axum-tauri-desktop template Rust + Yew + Axum + Tauri, full-stack Rust development for Desktop apps. Crates frontend: Yew frontend app for de

A lightning fast version of tmux-fingers written in Rust, copy/pasting tmux like vimium/vimperator
A lightning fast version of tmux-fingers written in Rust, copy/pasting tmux like vimium/vimperator

tmux-thumbs A lightning fast version of tmux-fingers written in Rust for copy pasting with vimium/vimperator like hints. Usage Press ( prefix + Space

A command-line tool collection to assist development written in RUST

dtool dtool is a command-line tool collection to assist development Table of Contents Description Usage Tips Installation Description Now dtool suppor

Rust mid-level IR Abstract Interpreter

MIRAI MIRAI is an abstract interpreter for the Rust compiler's mid-level intermediate representation (MIR). It is intended to become a widely used sta

Migrate C code to Rust
Migrate C code to Rust

C2Rust helps you migrate C99-compliant code to Rust. The translator (or transpiler) produces unsafe Rust code that closely mirrors the input C code. T

C to Rust translator

Corrode: Automatic semantics-preserving translation from C to Rust This program reads a C source file and prints an equivalent module in Rust syntax.

Comments
  • Error build serve

    Error build serve

    just serve trunk serve --release crates/mevi-frontend/index.html sh: line 1: trunk: command not found error: Recipeservefailed on line 10 with exit code 127

    opened by RomanAverin 4
  • cannot find value `_UFFDIO_WRITEPROTECT` in this scope

    cannot find value `_UFFDIO_WRITEPROTECT` in this scope

    I try to build on Ubuntu 2004, and I've this error during the build of userfaultfd-rs crate:

      error[E0425]: cannot find value `_UFFDIO_WRITEPROTECT` in this scope
      --> src/linux5_7.rs:20:74
       |
    20 |     1 << _UFFDIO_WAKE | 1 << _UFFDIO_COPY | 1 << _UFFDIO_ZEROPAGE | 1 << _UFFDIO_WRITEPROTECT;
       |                                                                          ^^^^^^^^^^^^^^^^^^^^ help: a constant with a similar name exists: `UFFDIO_WRITEPROTECT`
    ...
    38 | pub const UFFDIO_WRITEPROTECT: u32 = 0xc018aa06;
       | ------------------------------------------------ similarly named constant `UFFDIO_WRITEPROTECT` defined here
    

    I try several kernel headers versions after 5.7 but none works.

    opened by musitdev 2
  • mevi panics when ran on sandboxed chromium

    mevi panics when ran on sandboxed chromium

    Command : target/release/mevi chromium Result :

    2023-03-23T14:52:11.818086Z  WARN mevi::tracer: a thread is changing the brk for the process, we should handle that
    [26942] => [1] mapping 7fdf7a100000..7fdf7a102000 (8 KiB) with NotResident
    The application panicked (crashed).
    Message:  called `Option::unwrap()` on a `None` value
    Location: crates/mevi/src/tracer.rs:200
    

    'hack' to bypass the issue ? :

    diff --git a/crates/mevi/src/tracer.rs b/crates/mevi/src/tracer.rs
    index 4518cb1..dd0dbc4 100644
    --- a/crates/mevi/src/tracer.rs
    +++ b/crates/mevi/src/tracer.rs
    @@ -195,7 +195,9 @@ impl Tracer {
                                             "{tid} => {for_tid} mapping {range:x?} ({}) with {state:?}",
                                             formatter(range.end - range.start)
                                         );
    -                                    let target = self.tracees.get(&for_tid).unwrap();
    +                                    let target = self.tracees.get(&for_tid);
    +                                    if target.is_some() {
    +                                    let target = target.unwrap();
                                         match &target.kind {
                                             TraceeKind::Fresh => unreachable!(),
                                             TraceeKind::Process { uffd, .. } => {
    @@ -213,6 +215,7 @@ impl Tracer {
                                                 panic!("thread {for_tid} of process {pid} mapping memory should show up in the parent");
                                             }
                                         }
    +                                }
    
                                         let ev = MeviEvent::TraceeEvent(
                                             for_tid,
    
    wontfix 
    opened by msyyces8x95 1
  • Include `rustup target add wasm32-unknown-unknown` in the setup instructions

    Include `rustup target add wasm32-unknown-unknown` in the setup instructions

    Running just serve without the wasm target installed resulted in pages of errors, it would be nice if this was handled automatically, or listed in the prerequisites.

    opened by froody 0
Owner
amos
hi, I'm amos! πŸ¦€ I make articles & videos about how computers work πŸ»β€β„ cool bear's less cool counterpart ✨ be kind
amos
This crate allows writing a struct in Rust and have it derive a struct of arrays layed out in memory according to the arrow format.

Arrow2-derive - derive for Arrow2 This crate allows writing a struct in Rust and have it derive a struct of arrays layed out in memory according to th

Jorge Leitao 29 Dec 27, 2022
A rust library that makes reading and writing memory of the Dolphin emulator easier.

dolphin-memory-rs A crate for reading from and writing to the emulated memory of Dolphin in rust. A lot of internals here are directly based on aldela

Madison Barry 4 Jul 19, 2022
Translate C++/Rust type into C type with the same memory layout

clayout, translate C++/Rust type into C type with the same memory layout. Generally, clayout is used together with bpftrace. clayout is developed on d

盏一 11 Nov 17, 2022
A memory efficient immutable string type that can store up to 24* bytes on the stack

compact_str A memory efficient immutable string type that can store up to 24* bytes on the stack. * 12 bytes for 32-bit architectures About A CompactS

Parker Timmerman 342 Jan 2, 2023
bustd is a lightweight process killer daemon for out-of-memory scenarios for Linux!

bustd: Available memory or bust! bustd is a lightweight process killer daemon for out-of-memory scenarios for Linux! Features Small memory usage! bust

Pop!_OS 8 Oct 6, 2022
A memory profiler for Linux.

Bytehound - a memory profiler for Linux Features Can be used to analyze memory leaks, see where exactly the memory is being consumed, identify tempora

Koute 3.3k Dec 25, 2022
messloc is a drop in replacement for malloc that can transparently recover from memory fragmentation without any changes to application code.

messloc is a drop in replacement for malloc that can transparently recover from memory fragmentation without any changes to application code. Goals Al

null 11 Dec 10, 2022
Allocscope - a memory tracking tool

allocscope a memory tracking tool allocscope is a tool for tracking down where the most egregiously large allocations are occurring in a C, C++ or Rus

Matt Kimball 489 Feb 22, 2023
A memory-based evasion technique which makes shellcode invisible from process start to end.

phantom A memory-based evasion technique which makes shellcode invisible from process start to end. Motivation ShellGhost Offensive Edition, and rust!

B1-TEAM 5 Sep 15, 2023
High-performance QEMU memory and instruction tracing

Cannoli Cannoli is a high-performance tracing engine for qemu-user. It can record a trace of both PCs executed, as well as memory operations. It consi

Margin Research 412 Oct 18, 2023