File logging

This PR is served for #39.

Implementation (Optional)

Following TODO placeholder left in the source code.

I have

• [x] run cargo fmt;
• [x] run cargo clippy;
• [x] run cargo testand all tests pass;
• [x] linked to the originating issue (if applicable).

Contact Details

No response

What happened?

@MatteoNardi:

Turns out the issue is not related to the kernel version, but to Archlinux. I reproduced the issue with another Archlinux machine running 5.15, which is the same version of my development Ubuntu 22.04 machine.

Creating a file with the path /tmp/1/2/3/4/myfile emits an event with path /1/2/3/4/myfile, so I suppose it has something to do with some special behavior of the /tmp folder. For some reason, the kernel struct dentry's chain of d_parent we traverse for building the path, ends prematurely.

Relevant log output

#### Kernel/OS


[vagrant@archlinux ~]$ uname -a
Linux archlinux 5.17.1-arch1-1 #1 SMP PREEMPT Mon, 28 Mar 2022 20:55:33 +0000 x86_64 GNU/Linux

Output

test tests::unlink_file ... FAILED                           

failures:                     

---- tests::file_name stdout ----                            
[INFO  bpf_common::trace_pipe] Logging events from /sys/kernel/debug/tracing/trace_pipe                                    
69581201284 1120 deleted /file_name_1                                                                                      
69581219467 1120 created /file_name_1                                                                                      
69581228087 1120 open /tmp/file_name_1 (33345)                                                                             

69581219467 1120 created /file_name_1 (3/4)                                                                                
- pid: 1120 (OK)              
- timestamp: 69581190947 - 69581233481 (OK)                                                                                
- event type: FsEvent :: FileCreated (OK)                                                                                  
- filename: (FAIL)                                           
  |    found: StringArray { data: "/file_name_1" }                                                                         
  | expected: StringArray { data: "/tmp/file_name_1" }                                                                     

thread 'tests::file_name' panicked at 'No event found matching results', /home/matteo/projects/pulsar-experiments/bpf_common/src/test_runner.rs:155:13   
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace                                              

---- tests::unlink_file stdout ----                          
[INFO  bpf_common::trace_pipe] Logging events from /sys/kernel/debug/tracing/trace_pipe                                    
70211704597 1120 deleted /unlink_file                                                                                      

70211704597 1120 deleted /unlink_file (3/4)                                                                                
- pid: 1120 (OK)              
- timestamp: 70211693859 - 70211718252 (OK)                                                                                
- event type: FsEvent :: FileDeleted (OK)                                                                                  
- filename: (FAIL)                                           
  |    found: StringArray { data: "/unlink_file" }                                                                         
  | expected: StringArray { data: "/tmp/unlink_file" }                                                                     

thread 'tests::unlink_file' panicked at 'No event found matching results', /home/matteo/projects/pulsar-experiments/bpf_common/src/test_runner.rs:155:13 


failures:                     
    tests::file_name                                         
    tests::unlink_file                                       

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

Readme update / Installer

• Readme revision
• Introduce the install script
• Change few doc sections in the source code
• Fix doc broken links (some structures need to be public to be linked in the doc, but fields and constructors remain private)

Closes #70

I have

• [x] run cargo fmt;
• [x] run cargo clippy;
• [x] run cargo testand all tests pass;
• [x] linked to the originating issue (if applicable).

Feature

Since most use-cases are based on container applications, Pulsar should be able to filter and to track only container processes. Pulsar is already able to monitor and track container applications. It would be nice to filter them based on the container ID.

Possible solution

Previous Pulsar versions attempted to do this using cgroups. Also Tracee uses cgroups (https://github.com/aquasecurity/tracee/issues/255).

Original issue author: @hdtrinh

This PR is a tentative refactor of the test suite. Tests are split in a different binary, which can be invoked with sudo by using a xtask.

Fixes https://github.com/Exein-io/pulsar/issues/3

Contact Details

No response

What happened?

Test pulsar on debian/bullseye64, it failed.

Relevant log output

vagrant@bullseye:~$ curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/Exein-io/pulsar/main/pulsar-install.sh | sh
info: downloading files
info: installing files
info: generating configuration
info: basic rules
info: cleaning
info: installation complete
vagrant@bullseye:~$ uname -a
Linux bullseye 5.10.0-21-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21) x86_64 GNU/Linux
vagrant@bullseye:~$ sudo pulsard
[2023-02-13T11:17:38Z INFO  pulsar::pulsard::daemon] Starting module process-monitor
[2023-02-13T11:17:38Z INFO  pulsar::pulsard::daemon] Starting module file-system-monitor
[2023-02-13T11:17:38Z WARN  bpf_common::feature_autodetect::lsm] LSM not supported: eBPF LSM programs disabled
[2023-02-13T11:17:38Z ERROR pulsar::pulsard::module_manager] Module error in file-system-monitor: failed program load kprobe security_path_mknod
    
    Caused by:
        0: the BPF_PROG_LOAD syscall failed. Verifier output: number of funcs in func_info doesn't match number of subprogs
           verification time 25 usec
           stack depth 0+0
           processed 0 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0
           
        1: Invalid argument (os error 22)
[2023-02-13T11:17:38Z INFO  pulsar::pulsard::daemon] Starting module network-monitor
[2023-02-13T11:17:38Z WARN  bpf_common::feature_autodetect::lsm] LSM not supported: eBPF LSM programs disabled
[2023-02-13T11:17:38Z INFO  pulsar::pulsard::daemon] Starting module logger
[2023-02-13T11:17:38Z INFO  pulsar::pulsard::daemon] Starting module rules-engine
[2023-02-13T11:17:39Z INFO  pulsar::pulsard::daemon] Starting module desktop-notifier

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

Make sending large eBPF events more memory efficient

This fixes #73 by removing variable-length fields from the eBPF event payloads and adding them to a dynamic buffer, sent after the event.

• eBPF buffer management code has been added to bpf-common/include/buffer.bpf.h
• variable length fields have been replaced by indexes (start + len) of the buffer
  • struct buffer_index is used on the eBPF side
  • struct BufferIndex is used on the Rust side
• Converting a C type to Event::Payload, now requires also the buffer. The IntoPayload trait was introduced.
• We use bytes::Bytes for buffer management, this results in less data copying.
  • There's a copy when reading the PerfEvent (unavoidable with aya APIs)
  • There's a copy when converting the C-like struct to event::Payload
    • Ideally we could remove this by using Bytes inside event::Payload
  • We no longer make a copy of big buffers which are processed before being converted to Payload. For example the UDP message buffer, which is used for intercepting DNS, is copied only once, from kernel to user space. Previously it was copied 2 times.

This was the simplest solution I could envision for #73.

I have

• [x] run cargo fmt;
• [x] run cargo clippy;
• [x] run cargo testand all tests pass;
• [x] linked to the originating issue (if applicable).

The bpf-helper bpf_d_path, available since kernel 5.10, would be a better alternative to our path extraction code, as it would remove the MAX_COMPONENTS limitation.

It only works with BTF enabled eBPF programs (kfuncs would be ok, kprobes would not) and a subset of LSM hooks. See https://github.com/torvalds/linux/blob/76f598ba7d8e2bfb4855b5298caedd5af0c374a8/kernel/trace/bpf_trace.c#L936-L947

We should research if we can make use of it.

Currently process-monitor monitors all system processes (which is necessary). Setting whitelist, whitelist_children or targets in the configuration affects only the other modules, which generate events only for those specific target processes (which is the desired behaviour).

We should implement a filter on process-monitor events that makes us able to distinguish only target processes (and children) events. Ultimately, all process should be monitored, but only target process events should be shared over the event bus.

Contact Details

No response

What happened?

parse_version_signature error on WSL2

Relevant log output

[2023-02-13T11:10:36Z WARN  bpf_common::program] Error identifying kernel version Invalid version_signature format: 5.15.79.1-microsoft-standard-WSL2. Assuming kernel 5.0.4

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

Improve LSM autodetection by trying to load an example LSM program.

• [x] split bpf-builder to its own crate because bpf-common would have needed to depend on itself at build-time.
• [x] added lsm_supported to BpfContext in order to avoid repeating the time-consuming check.
• [x] add a configuration field to pulsar for forcing the use of kprobes/lsm

I have

• [x] run cargo fmt;
• [x] run cargo clippy;
• [x] run cargo testand all tests pass;
• [x] linked to the originating issue (if applicable).

Started to work on #97:

• Added user and group id on fork event (using bpf_get_current_uid_gid())
• Added hooks to intercept credentials change:
  • On LSM-supported kernels,task_fix_set*id calls are used
  • On kernel >= 5.10, we fall back on security_task_fix_set*id
  • On older kernels we fall back to commit_creds kprobe
• Testing on architest virtual machines led me to discover some regression to to previous pull requests. I've fixed them and all tests are passing now.

To completely fix #97, the only item left is to:

• update process-tracker with the user id
• add a mapping in process-tracker from user-id to username
• put those into header on event generation

Credential changes syscalls

A small explanation on credentials-related system calls. User is changed with the setuid syscall, which invokes the LSM task_fix_setuid and calls commit_creds. The same for group changes: setgid leads to a call to task_fix_setgid and commit_creds.

I have

• [x] run cargo fmt;
• [x] run cargo clippy;
• [x] run cargo testand all tests pass;
• [x] linked to the originating issue (if applicable).

Rust 1.70 now includes OnceCell, which should make our dependency on lazy_static unneeded. We should port our code to it.

Possible reasons not to it: Rust 1.70 just came out and could take some time before being available on projects sticking to an older version (I'm thinking about Yocto)

Now in the readme we point directly to the install script in the main branch. this can be inconsistent with the files in latest, so it's better to have a dedicated script for every tag

I find the modules start task function a bit strange, expecially the:

loop {
    tokio::select! {
    r = shutdown.recv() => return r,
    _ = rx_config.changed() => ...
    msg = receiver.recv() => ...
}

at the current state, this is an obligated pattern modules should do.

Another reason is that for developers is simpler to work with objects as it feels more natural.

My proposal is to have something similar to this for the module trait:

// module name
pub struct Name2(pub String);

// module semver
pub struct Version2(pub i32);

// needed by the upper level to decide if allocate a receiver or not
pub enum OnEvent<'a> {
    Nothing,
    Do(EventHandler<'a>),
}

impl<'a> OnEvent<'a> {
    pub fn execute<F: 'a + FnMut() -> Result<(), Box<dyn std::error::Error>>>(f: F) -> Self {
        Self::Do(EventHandler(Box::new(f)))
    }
}

pub struct EventHandler<'a>(Box<dyn FnMut() -> Result<(), Box<dyn std::error::Error>> + 'a>);

// context similar to the current to send events and errors
pub struct Cx2 {}

// raw module configuration
pub struct ModuleConfig2 {}

// event
#[derive(Debug)]
pub struct Event2 {}

// module trait
pub trait Module2: Sized {
    type Config: for<'a> TryFrom<&'a ModuleConfig2>;

    fn start(config: &Self::Config, cx: &Cx2) -> (Name2, Version2, Self);
    
    // default for producer only modules
    fn on_event<'a>(
        &'a mut self,
        config: &'a Self::Config,
        event: &'a Event2,
        cx: &'a Cx2,
    ) -> OnEvent<'a> {
        OnEvent::Nothing
    }

    fn on_cfg_change(
        &mut self,
        config: &Self::Config,
        cx: &Cx2,
    ) -> Result<(), Box<dyn std::error::Error>> {
        Ok(())
    }

    // default normal drop
    fn graceful_stop(self, cx: &Cx2) {}
}

and move the loop/select block to the upper level into the ModuleManager as it's the same for every module.

The following lines are previews of how modules should look with new traits:

mod producer1 {
    use pdk::*;

    struct MyProducerModuleConfig {
        file_path: String,
    }

    impl TryFrom<&ModuleConfig2> for MyProducerModuleConfig {
        type Error = String;

        fn try_from(value: &ModuleConfig2) -> std::result::Result<Self, Self::Error> {
            todo!()
        }
    }

    struct MyProducerModule {
        ebpf_probe: i32, // in reality Program type
    }

    impl Module2 for MyProducerModule {
        type Config = MyProducerModuleConfig;

        fn start(config: &Self::Config, cx: &Cx2) -> (Name2, Version2, Self) {
            let ebpf_probe = 0; // program_start(...)
            (
                Name2("producer".to_string()),
                Version2(666),
                Self { ebpf_probe },
            )
        }
    }
}

mod consumer1 {
    use pdk::*;

    struct MyConsumerModuleConfig {
        file_path: String,
    }

    impl TryFrom<&ModuleConfig2> for MyConsumerModuleConfig {
        type Error = String;

        fn try_from(value: &ModuleConfig2) -> std::result::Result<Self, Self::Error> {
            todo!()
        }
    }

    struct MyConsumerModule {
        file_writer: i32,
    }

    impl Module2 for MyConsumerModule {
        type Config = MyConsumerModuleConfig;

        fn start(config: &Self::Config, cx: &Cx2) -> (Name2, Version2, Self) {
            let file_writer = 0;
            (
                Name2("consumer".to_string()),
                Version2(666),
                Self { file_writer },
            )
        }

        fn on_event<'a>(
            &'a mut self,
            config: &'a Self::Config,
            event: &'a Event2,
            cx: &'a Cx2,
        ) -> OnEvent<'a> {
            OnEvent::execute(move || {
                // self.file_writer.write(event);
                todo!()
            })
        }

        fn graceful_stop(self, cx: &Cx2) {
            // file_writer.flush()
            // file_writer.close()
        }
    }
}

mod consumer2 {
    use pdk::*;

    struct MyEngineConfig {
        option1: bool,
    }

    impl TryFrom<&ModuleConfig2> for MyEngineConfig {
        type Error = String;

        fn try_from(value: &ModuleConfig2) -> std::result::Result<Self, Self::Error> {
            todo!()
        }
    }

    struct MyEngine {
        engine: i32,
    }

    impl Module2 for MyEngine {
        type Config = MyEngineConfig;

        fn start(config: &Self::Config, cx: &Cx2) -> (Name2, Version2, Self) {
            let engine = 0; //  = PulsarEngine::new(&config.rules_path, cx.get_sender())?;
            (
                Name2("consumer2".to_string()),
                Version2(666),
                Self { engine },
            )
        }

        fn on_cfg_change(&mut self, config: &Self::Config, cx: &Cx2)-> Result<(), Box<dyn std::error::Error>> {
            //  self.engine = PulsarEngine::new(&config.rules_path, ctx.get_sender())?;
            Ok(())
        }

        fn on_event<'a>(
            &'a mut self,
            config: &'a Self::Config,
            event: &'a Event2,
            cx: &'a Cx2,
        ) -> OnEvent<'a> {
            OnEvent::execute(move || {
                // println!("event consumed: {event:?}");
                // engine.process(&event);
                Ok(())
            })
        }
    }
}

Disclaimer: this code is not complete, but it should resemble what we will get in the end, signatures can be polished. it should compile apart from possible errors adapting to this issue.

Note: maybe the type erasure on the handle function is not needed, even the EventHandler shim with the execute function, but it needs a full test.

Github workflow should be cleaned, notable things:

• use cross even for x86 build in build.yaml for coherency with release workflow
• improve release workflow using a matrix. steps should be sequential or we need to add a temporary storage where every parallel tasks should store the artifact before the final release creation with file uploads