Process killer daemon for out-of-memory scenarios

Related tags

System tools linux rust daemon out-of-memory oomkiller
Overview

bustd: Available memory or bust!

bustd is a lightweight process killer daemon for out-of-memory scenarios for Linux!

Features

Small memory usage!

bustd seems to use less memory than some other lean daemons such as earlyoom:

$ ps -F -C bustd
UID          PID    PPID  C    SZ   RSS PSR STIME TTY          TIME CMD
vrmiguel  353609  187407  5   151     8   2 01:20 pts/2    00:00:00 target/x86_64-unknown-linux-musl/release/bustd -V -n

$ ps -F -C earlyoom
UID          PID    PPID  C    SZ   RSS PSR STIME TTY          TIME CMD
vrmiguel  350497    9498  0   597   688   6 01:12 pts/1    00:00:00 ./earlyoom/

¹: RSS stands for resident set size and represents the portion of RAM occupied by a process.

²: Compared when bustd was in this commit and earlyoom in this one. bustd compiled with musl libc and earlyoom with glibc through GCC 11.1. Different configurations would likely change these figures.

Small CPU usage

Much like earlyoom and nohang, bustd uses adaptive sleep times during its memory polling. Unlike these two, however, bustd does not read from /proc/meminfo, instead opting for the sysinfo syscall.

This approach has its up- and downsides. The amount of free RAM that sysinfo reads does not account for cached memory, while MemAvailable in /proc/meminfo does.

The sysinfo syscall is one order of magnitude faster, at least according to this kernel patch (granted, from 2015).

As bustd can't solely rely on the free RAM readings of sysinfo, we check for memory stress through Pressure Stall Information.

bustd will try to lock all pages mapped into its address space

Much like earlyoom, bustd uses mlockall to avoid being sent to swap, which allows the daemon to remain responsive even when the system memory is under heavy load and susceptible to thrashing.

Checks for Pressure Stall Information

The Linux kernel, since version 4.20 (and built with CONFIG_PSI=y), presents canonical new pressure metrics for memory, CPU, and IO. In the words of Facebook Incubator:

PSI stats are like barometers that provide fair warning of impending resource 
shortages, enabling you to take more proactive, granular, and nuanced steps 
when resources start becoming scarce.

More specifically, bustd checks for how long, in microseconds, processes have stalled in the last 10 seconds. By default, bustd will kill a process when processes have stalled for 25 microseconds in the last ten seconds.

Building

Requirements:

git clone https://github.com/vrmiguel/bustd
cd bustd && cargo run --release

The -n, --no-daemon flag is useful for running bustd through an init system such as systemd.

Prebuilt binaries

Binaries are generated at every commit through GitHub Actions

TODO

  • Allow for customization of the critical scenario (PSI cutoff)
  • Command-line argument for disabling daemonization (useful for runnning bustd as a systemd service)
  • Command-line argument to enable killing the entire process group, not just the chosen process itself
  • Allow the user to setup a list of software that bustd should never kill
    • In progress! A list of glob patterns of processes that bustd will not kill.
    • Example: firefox | systemd-* | rustc
  • Notification sending and general notification customization settings
Comments
  • Security advisories for time and chrono crates used by procfs

    Security advisories for time and chrono crates used by procfs

    Something to be wary of:

    # cargo audit
    Fetching advisory database from `https://github.com/RustSec/advisory-db.git`
      Loaded 372 security advisories (from /home/mmstick/.cargo/advisory-db)
    Updating crates.io index
    Scanning Cargo.lock for vulnerabilities (35 crate dependencies)
Crate:         chrono
Version:       0.4.19
Title:         Potential segfault in `localtime_r` invocations
Date:          2020-11-10
ID:            RUSTSEC-2020-0159
URL:           https://rustsec.org/advisories/RUSTSEC-2020-0159
Solution:      No safe upgrade is available!
Dependency tree: 
chrono 0.4.19
└── procfs 0.11.0
    └── bustd 0.1.0

Crate:         time
Version:       0.1.44
Title:         Potential segfault in the time crate
Date:          2020-11-18
ID:            RUSTSEC-2020-0071
URL:           https://rustsec.org/advisories/RUSTSEC-2020-0071
Solution:      Upgrade to >=0.2.23
Dependency tree: 
time 0.1.44

error: 2 vulnerabilities found!
    opened by mmstick 6
  • Updated CLI Argument Parsing

    Updated CLI Argument Parsing

    Transitioned from crate argh to clap. Created identical argument-parsing framework, with some additional elements, such as short and long flags for all items, instead of the previous only short flags. Currently cargo check passes with no warnings and the help page that is printed is fairly pretty.

    opened by JamesDevlin5 3
  • Unable to compile on Arch Linux

    Unable to compile on Arch Linux

    I'm trying to compile bustd on Arch, but I'm getting this:

    $ cargo build --release
   Compiling proc-macro2 v1.0.27
   Compiling libc v0.2.97
   Compiling unicode-xid v0.2.2
   Compiling syn v1.0.73
   Compiling unicode-segmentation v1.8.0
   Compiling argh_shared v0.1.5
   Compiling cc v1.0.68
   Compiling boxfnonce v0.1.1
   Compiling cfg-if v1.0.0
   Compiling heck v0.3.3
   Compiling quote v1.0.9
   Compiling daemonize v0.4.1
   Compiling bustd v0.1.0 (/home/gui/bustd)
   Compiling argh_derive v0.1.5
   Compiling no-panic v0.1.15
   Compiling argh v0.1.5
error: linking with `cc` failed: exit status: 1
  |
  = note: "cc" "-m64" "/home/gui/bustd/target/release/deps/bustd-29598bf87e19bc74.bustd.4072317c-cgu.0.rcgu.o" "-Wl,--as-needed" "-L" "/home/gui/bustd/target/release/deps" "-L" "/home/gui/bustd/target/release/build/bustd-6bc53eed358101fc/out" "-L" "/usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "-Wl,--whole-archive" "-lhelper" "-Wl,--no-whole-archive" "-Wl,--start-group" "-Wl,--end-group" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-0c7e43ac4e512046.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L" "/usr/lib64/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/gui/bustd/target/release/deps/bustd-29598bf87e19bc74" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs"
  = note: /usr/bin/ld: /home/gui/bustd/target/release/deps/bustd-29598bf87e19bc74.bustd.4072317c-cgu.0.rcgu.o: in function `<bustd::memory::mem_info::sys_info::__NoPanic as core::ops::drop::Drop>::drop':
          bustd.4072317c-cgu.0:(.text._ZN86_$LT$bustd..memory..mem_info..sys_info..__NoPanic$u20$as$u20$core..ops..drop..Drop$GT$4drop17h281314e3600cb451E+0x3): undefined reference to `
          
          ERROR[no-panic]: detected panic in function `sys_info`
          '
          /usr/bin/ld: /home/gui/bustd/target/release/deps/bustd-29598bf87e19bc74.bustd.4072317c-cgu.0.rcgu.o: in function `<bustd::memory::mem_lock::_mlockall_wrapper::__NoPanic as core::ops::drop::Drop>::drop':
          bustd.4072317c-cgu.0:(.text._ZN95_$LT$bustd..memory..mem_lock.._mlockall_wrapper..__NoPanic$u20$as$u20$core..ops..drop..Drop$GT$4drop17h8622ff938e405c5dE+0x3): undefined reference to `
          
          ERROR[no-panic]: detected panic in function `_mlockall_wrapper`
          '
          /usr/bin/ld: /home/gui/bustd/target/release/deps/bustd-29598bf87e19bc74.bustd.4072317c-cgu.0.rcgu.o: in function `<bustd::utils::effective_user_id::__NoPanic as core::ops::drop::Drop>::drop':
          bustd.4072317c-cgu.0:(.text._ZN84_$LT$bustd..utils..effective_user_id..__NoPanic$u20$as$u20$core..ops..drop..Drop$GT$4drop17hb0dd8c74268564e8E+0x3): undefined reference to `
          
          ERROR[no-panic]: detected panic in function `effective_user_id`
          '
          /usr/bin/ld: /home/gui/bustd/target/release/deps/bustd-29598bf87e19bc74.bustd.4072317c-cgu.0.rcgu.o: in function `<bustd::utils::running_as_sudo::__NoPanic as core::ops::drop::Drop>::drop':
          bustd.4072317c-cgu.0:(.text._ZN82_$LT$bustd..utils..running_as_sudo..__NoPanic$u20$as$u20$core..ops..drop..Drop$GT$4drop17h8cd11b1c65f8f688E+0x3): undefined reference to `
          
          ERROR[no-panic]: detected panic in function `running_as_sudo`
          '
          /usr/bin/ld: /home/gui/bustd/target/release/deps/bustd-29598bf87e19bc74.bustd.4072317c-cgu.0.rcgu.o: in function `<bustd::utils::page_size::__NoPanic as core::ops::drop::Drop>::drop':
          bustd.4072317c-cgu.0:(.text._ZN76_$LT$bustd..utils..page_size..__NoPanic$u20$as$u20$core..ops..drop..Drop$GT$4drop17h3810a645249e8a35E+0x3): undefined reference to `
          
          ERROR[no-panic]: detected panic in function `page_size`
          '
          collect2: error: ld returned 1 exit status
          
  = help: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)

error: could not compile `bustd` due to previous error

    Any tips? Thanks.

    • rustc 1.56.1 (Arch Linux rust 1:1.56.1-3)
    • cc (GCC) 11.1.0
    • linux 5.10.78-1-lts
    opened by guihkx 3

  • "Starting daemonization process!" but no process is started

    After running cargo run --release it prints message from title, but running ps -F -C bustd returns empty result, ie. no bustd is running.

    If I run it with -n option it exits with IO error No such file, so I guess it doesn't actually work. I'm on 4.15 kernel, is that the problem?

    Thanks

    opened by Boza-s6 2
  • Notification support

    Notification support

    Should this be achieved by making this service a DBus service offering signals that a user session service can watch and give notify-rust notifications with?

    opened by mmstick 2
  • Updated CLI Argument Parsing (New)

    Updated CLI Argument Parsing (New)

    Re-implemented the argument parsing module, now using the nightly version of clap, and a derive-attribute pattern instead of the previous builder pattern, as suggested by @Xunjin in the previous pull request. The help page is colorized and very clean looking. Some small changes have yet to be fully implemented:

    • The glob attribute on the cli arguments struct is commented out
    • The unkillables cli argument is not split by tilde

    Other than that, everything looks pretty good!

    opened by JamesDevlin5 4
  • Make free ram threshold configurable

    Make free ram threshold configurable

    Is it a good idea to make https://github.com/vrmiguel/bustd/blob/22470c4b351527ff75e0c4e71a55f0c675a21614/src/monitor.rs#L64 configurable ?

    I personally have a pc that's very constrained on resource so I'm using bustd with that setting changed to 5, or else the killing of process happens earlier that I want. But my pc is old so I don't know how frequent this use-case is.

    One obvious problem of making it configurable , is if the user sets a value like 100% it might theoretically kill all his session? maybe there should a be an acceptable range.

    opened by sigmaSd 4
Releases(v0.1.1)
Owner
Vinícius Miguel
barely passed the Turing test
Vinícius Miguel
GitHub
`haproxy_autconfd` is a daemon that automatically assembles a HAProxy config and restarts HAProxy if the config changes

haproxy_autconfd Welcome to haproxy_autconfd ?? haproxy_autconfd is a daemon that automatically assembles a HAProxy config and restarts HAProxy if the

null 0 Nov 9, 2021
LeftHK - A hotkey daemon written in Rust

LeftHK LeftHK - A hotkey daemon written in Rust THIS IS BETA SOFTWARE The configuration file should be created in ~/.config/lefthk/ and called config.

null 15 Oct 12, 2022
bustd is a lightweight process killer daemon for out-of-memory scenarios for Linux!

bustd: Available memory or bust! bustd is a lightweight process killer daemon for out-of-memory scenarios for Linux! Features Small memory usage! bust

Pop!_OS 8 Oct 6, 2022
Easy c̵̰͠r̵̛̠ö̴̪s̶̩̒s̵̭̀-t̶̲͝h̶̯̚r̵̺͐e̷̖̽ḁ̴̍d̶̖̔ ȓ̵͙ė̶͎ḟ̴͙e̸̖͛r̶̖͗ë̶̱́ṉ̵̒ĉ̷̥e̷͚̍ s̷̹͌h̷̲̉a̵̭͋r̷̫̊ḭ̵̊n̷̬͂g̵̦̃ f̶̻̊ơ̵̜ṟ̸̈́ R̵̞̋ù̵̺s̷̖̅ţ̸͗!̸̼͋

Rust S̵̓i̸̓n̵̉ I̴n̴f̶e̸r̵n̷a̴l mutability! Howdy, friendly Rust developer! Ever had a value get m̵̯̅ð̶͊v̴̮̾ê̴̼͘d away right under your nose just when

null 294 Dec 23, 2022
secmem-proc is a crate designed to harden a process against low-privileged attackers running on the same system trying to obtain secret memory contents of the current process.

secmem-proc is a crate designed to harden a process against low-privileged attackers running on the same system trying to obtain secret memory contents of the current process. More specifically, the crate disables core dumps and tries to disable tracing on unix-like OSes.

null 3 Dec 19, 2022
DNS resolver for split-horizon scenarios

polyresolver is a resolver for split-horizon scenarios polyresolver is used to root domain names to different nameservers for the purposes of resolvin

Erik Hollensbe 7 Aug 23, 2022
A brand-new multi-scenarios smart contract compiler framework

The Smart Intermediate Representation The Smart Intermediate Representation(short for IR) project is a new compiler framework intended for smart contr

AntChainOpenLabs 62 Jan 2, 2024
figure out who holds the nfts that came out of a candy machine.

Candy Holders This is far from finished, but can: find tokens with a given update authority find holders of those tokens Neither the Rust or Node APIs

ExiledApes 24 Dec 29, 2022
A Rust on-site channel benchmarking helper. Inter-Process (async / busy) & Intra-Process (async single threaded / async multi threaded)

On-Site Rust Channel Benchmarking Helper Deploy on server to determine which public crates are the fastest for communicating in different architecture

null 23 Jul 9, 2024
This crate allows writing a struct in Rust and have it derive a struct of arrays layed out in memory according to the arrow format.

Arrow2-derive - derive for Arrow2 This crate allows writing a struct in Rust and have it derive a struct of arrays layed out in memory according to th

Jorge Leitao 29 Dec 27, 2022
It's implemented by laying out the elements in memory contiguously like alloc::vec::Vec

A Vec<T: ?Sized> It's implemented by laying out the elements in memory contiguously like alloc::vec::Vec Layout A Vechonk is 4 usize long. It owns a s

nils 4 Sep 15, 2022
Provides a mechanism to lay out data into GPU buffers according to WGSL's memory layout rules

Provides a mechanism to lay out data into GPU buffers ensuring WGSL's memory layout requirements are met. Features supports all WGSL host-shareable ty

Teodor Tanasoaia 69 Dec 30, 2022
MiniDump a process in memory with rust

safetydump Rust in-memory MiniDump implementation. Features ntdll!NtGetNextProcess to obtain a handle for the desired ProcessId as opposed to kernel32

null 26 Oct 11, 2022
A cross-platform and safe Rust API to create and manage memory mappings in the virtual address space of the calling process.

mmap-rs A cross-platform and safe Rust API to create and manage memory mappings in the virtual address space of the calling process. This crate can be

S.J.R. van Schaik 19 Oct 23, 2022
A CLI tool which can help you automatically kill process of your choice. Useful for freeing up memory and CPU usage!

Quickiller There are always programs such as chrome that keep eating up your resources even when closed! The only way to prevent this is to kill all o

Codingsquirrel 1 Dec 8, 2021
Memory usage monitor for process trees

gotta-watch-em-all Executes a process with given arguments and monitors, logs when memory usage grows to a new peak. Example: cargo run -- cargo -- bu

Aaron Friel 8 Dec 8, 2022
A process memory reader and debugger for Windows (x86_64)

Winreader Winreader is a process memory reader and debugger for Windows, implemented and developed in the Rust language, using the official Microsoft

null 31 Feb 15, 2023
A simple code that will load a shellcode directly into RAM memory in a new process

「 ?? 」About RustSCLoader RustSCLoader is a simple code that has the intention of loading a shellcode directly into RAM memory in a new process that wi

null 5 May 15, 2023
A memory-based evasion technique which makes shellcode invisible from process start to end.

phantom A memory-based evasion technique which makes shellcode invisible from process start to end. Motivation ShellGhost Offensive Edition, and rust!

B1-TEAM 5 Sep 15, 2023
A library that allows for the arbitrary inspection and manipulation of the memory and code of a process on a Linux system.

raminspect raminspect is a crate that allows for the inspection and manipulation of the memory and code of a running process on a Linux system. It pro

Liam Germain 24 Sep 26, 2023