The following problem occurred today, it didn't exist before:

thread '' panicked at 'not implemented: 'memalign' is unimplemented!', preload/src/api.rs:831:5

When attempting to load my application with libmemory_profiler.so in LD_PRELOAD, it immediately segfaults.

Built with:

rustc 1.36.0-nightly (6afcb5628 2019-05-19)

Captured from GDB:

gdb --args env LD_PRELOAD=/usr/local/lib/libmemory_profiler.so ./build/myapp

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7881e75 in memory_profiler::initialize () at preload/src/lib.rs:1239
1239	fn initialize() {
(gdb) bt
#0  0x00007ffff7881e75 in memory_profiler::initialize () at preload/src/lib.rs:1239
#1  0x00007ffff7883cc9 in memory_profiler::allocate (size=32, is_calloc=<optimized out>) at preload/src/lib.rs:1417
#2  malloc (size=32) at preload/src/lib.rs:1459
#3  0x00007ffff7883df6 in memory_profiler::allocate (size=32, is_calloc=<optimized out>) at preload/src/lib.rs:1429
#4  malloc (size=32) at preload/src/lib.rs:1459
...

Hi

We have troubles when analysing gathered data. Gathered data is about 9GB. We tried to squeeze the data but it failed. Then we tried to load big data and it failed as well

./bytehound strip --output X Y.dat
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', /tmp/koute/memory-profiler/cli-core/src/squeeze.rs:217:100
none: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Aborted

when loading big data:

./bytehound server Y.dat
[2022-12-01T15:11:46Z INFO  server_core] Trying to load "Y.dat"...
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', cli-core/src/loader.rs:864:68
stack backtrace:
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Aborted

Did you encounter such problem? Do you know how we should proceed with that?

I am profiling an application which uses obscene amounts of memory, but is not, strictly speaking, leaking memory: if I kill the app, everything is cleanly deallocated.

Still, I'd love to know what takes all the memory. I think what I need is to take a look at the live allocations at some specific point in time, and get a famegraph. This'll give me essentially a profile for a heap snapshot at a point in time.

Can I already get this with bytehound? I've seen flamegraph for leaked memory, and the graph of the total live allocations, but neither is quite what I am looking for.

On debian10 with nigthly rust

thread '<unnamed>' panicked at 'cannot access a TLS value during or after it is destroyed: AccessError', src/libcore/result.rs:999:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:71
             at src/libstd/sys_common/backtrace.rs:59
             at src/libstd/panicking.rs:197
   2: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:211
             at src/libstd/panicking.rs:474
   3: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:381
   4: rust_begin_unwind
             at src/libstd/panicking.rs:308
   5: core::panicking::panic_fmt
             at src/libcore/panicking.rs:85
   6: core::result::unwrap_failed
             at /rustc/37ff5d388f8c004ca248adb635f1cc84d347eda0/src/libcore/macros.rs:18
   7: memory_profiler::unwind::grab
             at /root/.cargo/git/checkouts/not-perf-e01bfa01482c86ed/9739e8b/nwind/src/local_unwinding.rs:0
             at preload/src/unwind.rs:166
   8: calloc
             at preload/src/lib.rs:1423
             at preload/src/lib.rs:1470
   9: g_malloc0
  10: g_slice_free_chain_with_offset
  11: g_queue_free
  12: __nptl_deallocate_tsd.part.8
  13: start_thread
  14: clone
Aborted (core dumped)

Failed to profile rust application due to error

thread '<unnamed>' panicked at 'not implemented: 'aligned_alloc' is unimplemented!', preload/src/api.rs:907:5
stack backtrace:
   0:     0x7f9661783cea - std::backtrace_rs::backtrace::libunwind::trace::h972caad916e73545
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   1:     0x7f9661783cea - std::backtrace_rs::backtrace::trace_unsynchronized::he59049878fe5a05d
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x7f9661783cea - std::sys_common::backtrace::_print_fmt::he4a91f9bcfad9b40
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/std/src/sys_common/backtrace.rs:66:5
   3:     0x7f9661783cea - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h51433dc001920472
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/std/src/sys_common/backtrace.rs:45:22
   4:     0x7f966172d10c - core::fmt::write::hc9dbd37d69b2c204
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/core/src/fmt/mod.rs:1198:17
   5:     0x7f9661762b14 - std::io::Write::write_fmt::h6b2550ce8adb9e04
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/std/src/io/mod.rs:1672:15
   6:     0x7f9661784af5 - std::sys_common::backtrace::_print::h006829bd22a5a4ee
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/std/src/sys_common/backtrace.rs:48:5
   7:     0x7f9661784af5 - std::sys_common::backtrace::print::h0f4d319136ab4456
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/std/src/sys_common/backtrace.rs:35:9
   8:     0x7f9661784af5 - std::panicking::default_hook::{{closure}}::h5b3cdff51fbe7401
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/std/src/panicking.rs:295:22
   9:     0x7f9661785075 - std::panicking::default_hook::hdc1d8baf28b4ffd7
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/std/src/panicking.rs:314:9
  10:     0x7f9661785075 - std::panicking::rust_panic_with_hook::h80e138cc00203db9
                               at /rustc/d68e7ebc38cb42b8b237392b28045edeec761503/library/std/src/panicking.rs:698:17
  11:     0x7f96617f9221 - nwind_ret_trampoline_start
                               at /home/user/.cargo/git/checkouts/not-perf-af1a46759dd83df9/18bd8d3/nwind/src/arch/amd64_trampoline.s:17
  12:                0x0 - <unknown>

used command

LD_PRELOAD=./libbytehound.so <rust_application> <args>

Tried with local build from the latest master, and with pre-built versions Target OS: Ubuntu 20.04

Please let me know if any option is missing to run tool properly, or rust applications are not support for now. Thanks in advance.

1. My environment is Ubuntu 18.04
2. Yarn has been installed already
3. Rust has been changed to nightly version
4. I run "cargo build --release -p memory-profiler" suceessfully and libmemory_profiler.so has been generate already

When I run "cargo build --release -p memory-profiler-cli", it failed and the error log as below:

davidwang@system-MS-7918:~/Tools/memory-profiler$ cargo build --release -p memory-profiler-cli Compiling server-core v0.1.0 (/home/davidwang/Tools/memory-profiler/server-core) 00h00m00s 0/0: : **error: failed to run custom build command for server-core v0.1.0** (/home/davidwang/Tools/memory-profiler/server-core)

Caused by: process didn't exit successfully: /home/davidwang/Tools/memory-profiler/target/release/build/server-core-5dfb1577abf53aa3/build-script-build (exit code: 101) --- stderr ERROR: [Errno 2] No such file or directory: 'install' thread 'main' panicked at 'Failed to install the dependencies for the WebUI; child process exited with error code Some(1)! You might want to try to run 'rm -Rf ~/.cache/yarn' and try again.', server-core/build.rs:61:21 stack backtrace: 0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39 1: std::sys_common::backtrace::_print at src/libstd/sys_common/backtrace.rs:71 2: std::panicking::default_hook::{{closure}} at src/libstd/sys_common/backtrace.rs:59 at src/libstd/panicking.rs:197 3: std::panicking::default_hook at src/libstd/panicking.rs:211 4: std::panicking::rust_panic_with_hook at src/libstd/panicking.rs:474 5: std::panicking::continue_panic_fmt at src/libstd/panicking.rs:381 6: std::panicking::begin_panic_fmt at src/libstd/panicking.rs:336 7: semalock::Semalock::with at server-core/build.rs:61 at /home/davidwang/.cargo/registry/src/github.com-1ecc6299db9ec823/semalock-0.2.0/src/lib.rs:99 at /rustc/8869ee03d7f258e1b76a11c6fbb01b5708a9f504/src/libcore/result.rs:639 at /home/davidwang/.cargo/registry/src/github.com-1ecc6299db9ec823/semalock-0.2.0/src/lib.rs:96 8: build_script_build::main at server-core/build.rs:46 9: std::rt::lang_start::{{closure}} at /rustc/8869ee03d7f258e1b76a11c6fbb01b5708a9f504/src/libstd/rt.rs:64 10: std::panicking::try::do_call at src/libstd/rt.rs:49 at src/libstd/panicking.rs:293 11: __rust_maybe_catch_panic at src/libpanic_unwind/lib.rs:85 12: std::rt::lang_start_internal at src/libstd/panicking.rs:272 at src/libstd/panic.rs:388 at src/libstd/rt.rs:48 13: main 14: __libc_start_main 15: _start

It seems to find install dir failed, but where should the dir be?

Hello, maybe the most things for this are already in place and it's easy to do. Currently manual usage of the heaptrack GUI is needed to see the memory flamegraph. Inferno could be used to directly generate an interactive SVG by exporting a perf-compatible text format (the text format can even be without collapsed stacks, as there is the inferno-collapse-perf command for that).

Hi,

Currently (on 1.59-nightly), bytehound doesn't compile because of the stabilization of thread_local_const_init feature:

error[E0635]: unknown feature `thread_local_const_init`
 --> preload/src/lib.rs:1:12
  |
1 | #![feature(thread_local_const_init)]
  |            ^^^^^^^^^^^^^^^^^^^^^^^

For more information about this error, try `rustc --explain E0635`.
error: could not compile `bytehound-preload` due to previous error

The feature line should be removed from the code from now on I think.

Cheers, Gerry

Adds plots to allocations so that it's possible to tell if given backtrace is the reason for potentially visible memory leak in overview page. Improves greatly feedback regarding if backtrace is a leak or not, as one sees the trend of allocations from this backtrace - are they increasing? or maybe it was just one spike at the end of trace?

The plot will be visible when "Group by backtraces" is selected.

PR code is a little wonky though... but it seems to work well.

I was thinking about using this project to compare the memory usage of a Go application that I have rewritten in Rust, but when testing the memory-profiler with a simple go example it generated no result.

Heres the go example I've tested:

package main

import "fmt"
import "os"
import "strconv"

func fib(n uint64) uint64 {
	switch n {
	case 0:
		return 0
	case 1:
		return 1
	default:
		return fib(n-1) + fib(n-2)
	}
}

func main() {
	input, _ := strconv.Atoi(os.Args[1])
	n := uint64(input)
	fmt.Println(fib(n))
}

And running:

go build test.go
LD_PRELOAD=/home/jonathas/Repositories/memory-profiler/target/release/libmemory_profiler.so ./test 40

Didn't generate any dat file at all. I have even tried adding a make or new calls to the fib func in order to introduce more allocations but I still got no results.

thread '<unnamed>' panicked at 'index out of bounds: the len is 256 but the index is 18446744073709551615', /home/user/.cargo/git/checkouts/not-perf-af1a46759dd83df9/911723c/nwind/src/local_unwinding.rs:455:26
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Aborted

I got the error(unfortunately I can't attach the code), there is some problem with index calculations.

I can only say that we are using memory mapping of a huge file (150 GB), while I do not access this memory, everything is ok, I believe that the problems begin immediately when accessing(but that's just a hypothesis).

Linux version 5.15.0-1026-aws gcc (Ubuntu ~20.04.1) 9.4.0, x86_64

stack backtrace:
   0: rust_begin_unwind
             at /rustc/d0dc9efff14ac0a1eeceffd1e605e37eeb8362a0/library/std/src/panicking.rs:575:5
   1: core::panicking::panic_fmt
             at /rustc/d0dc9efff14ac0a1eeceffd1e605e37eeb8362a0/library/core/src/panicking.rs:64:14
   2: core::panicking::panic_bounds_check
             at /rustc/d0dc9efff14ac0a1eeceffd1e605e37eeb8362a0/library/core/src/panicking.rs:147:5
   3: nwind::local_unwinding::ShadowStack::push
             at /home/user/.cargo/git/checkouts/not-perf-af1a46759dd83df9/911723c/nwind/src/local_unwinding.rs:455:26
   4: nwind::local_unwinding::LocalAddressSpace::unwind_through_fresh_frames
             at /home/user/.cargo/git/checkouts/not-perf-af1a46759dd83df9/911723c/nwind/src/local_unwinding.rs:905:27
   5: bytehound::unwind::grab_with_unwind_state
             at /home/user/bytehound/preload/src/unwind.rs:393:31
   6: nwind_ret_trampoline_start
             at /home/user/cargo/git/checkouts/not-perf-af1a46759dd83df9/911723c/nwind/src/arch/amd64_trampoline.s:17

I seeamd64_trampoline here, is it ok(I use x86)?

I did not find the maximum size for the output file in the environment variables, I would like to split it into files, for example, 5GB each and send them to remote storage. Maybe this is possible and I missed something?

Hi! in the backtraces, i see graphs with plots in green and red color. where green is area is said to be 'global' and red is said to be 'matched'.

1. What exactly global and matched means?
2. Also what should i interpret from the following graph for a backtrace:

some details about the scenario: About the above graph, the first part where we can see memory consumption increasing continuously is the duration where we were running lock and unlock of a component in our system for some 200 iterations. The later part, which is flat(and in red), is the duration when iterations had completed but profiling continued for some time.

I had a debugging usecase where the executable would either never finish or get OOM-killed. So I guess the output could be somewhat incomplete/messed up. The output was very large (34GB) so I stripped it (bytehound strip --threshold 10 ...). That bytehound server would crash because it didn't find some backtraces. This PR fixes that and instead just prints a warning.

Hi, great project! I successfully compiled the project, but when executing this statement：

LD_PRELOAD=./libbytehound.so ./bytehound serverv

i got: ./bytehound: symbol lookup error: ./libbytehound.so: undefined symbol: memfd_create

and every program is like this Did I do something wrong？

Information about env： In docker container (uname -r = 5.10.104-linuxkit) already install gcc g++

Dearest Maintainer,

Bytehound has been a joy to use. Thank you for this. I found the "Case study: Memory leak analysis" after searching for how to use bytehound. I have found the example scripting to be amazingly helpful. My ask would to to make that flow in to a page.

"by backtrace" which gives the chart and then you can select the groups one at a time.

A page for leaked until the end as well. maybe that is a filter on by backtrace.

Last question I have is what does "memory lost to fragmentation" really mean? what does one do with this information?

Either way! thanks for reading this. I looked at the ui folder and saw it was react. I am very lost in that world but one day might take the time.

Thanks for the amazing software. It has been very helpful.

Becker