fennec is an artifact collection tool written in Rust to be used during incident response on *nix based systems. fennec allows you to write a configuration file that contains how to collect artifacts.
Features
🌟
🦀
A single statically compiled binary
🔬
Execute any osquery SQL query
💻
Execute system commands
📚
Parse any text file using regex
🧰
Ability to collect system logs and files
🧱
Return data in structured manner
🃏
Support multiple output formats (JSONL, CSV and KJSON)
🤸♀️
Flexible configuration file
💾
Directly write to ZIP file to safe space
⚡
Very fast!
Tests
🧪
OS Details
Architecture
Success?
Details
Ubuntu 20.04.3 LTS
x86_64
✅
Ubuntu 19.04
x86_64
✅
Ubuntu 18.04.6 LTS
x86_64
✅
Ubuntu 17.04
x86_64
✅
Ubuntu 16.04.7 LTS
x86_64
✅
Ubuntu 15.10
x86_64
✅
Ubuntu 14.04.6 LTS
x86_64
✅
Ubuntu 13.04
x86_64
✅
Ubuntu 12.04.5 LTS
x86_64
✅
CentOS 8.4.2105
x86_64
✅
CentOS 7.9.2009
x86_64
✅
CentOS 6.10
x86_64
✅
CentOS 5.11
x86_64
❌
osquery requires libc >= 2.12
Ubuntu 20.04
aarch64
✅
MacOS Monterey v12.0.1
x86_64
✅
Usage
✍
fennec 0.1.0
AbdulRhman Alfaifi <[email protected]>
Aritfact collection tool for*nix systems
USAGE:
fennec_x86_64-unknown-linux-gnu [OPTIONS]
OPTIONS:
-c, --config <FILE> Sets a custom config file
-f, --log-file <FILE> Sets the log file name [default: fennec.log]
-h, --help Print help information
-l, --log-level <LEVEL> Sets the log level [default: info] [possible values: trace,
debug, info, error]
-o, --output<FILE> Sets output file name [default: ABDULRHMAN-PC.zip]
--osquery-path <PATH> Sets osquery path, if osquery is embedded it will be writen to
this path otherwise the path will be used to spawn osquery
instance [default: ./osqueryd]
--output-format <FORMAT> Sets output format [default: jsonl] [possible values: jsonl,
csv, kjson]
-q, --quiet Do not print logs to stdout
--show-config Show the embedded configuration file
-V, --version Print version information
-c, --config : Use the specified configuration file instead of the embedded configuration
-f, --log-file : Change the default name for the log file (default: fennec.log)
-h, --help : Print help message
-l, --log-level : Change the default log level (default: info)
-o, --output : Change the default output file name for the zip file (default: {HOSTNAME}.zip, where hostname is the runtime evaluated machine hostname)
--osquery-path : Path to osquery executable, This value will be used based on these conditions:
If osquery binary is embedded into fennec then extract it and dump it to --osquery-path
If osquery is not embedded into fennec then use the osquery binary in the path --osquery-path
--output-format : Choose the output format, Supported formats:
jsonl : A new line separated JSON objects (default)
csv: Comma separated values
kjson: Use this format if you want to upload the resulting file to Kuiper analysis platform.
-q, --quiet : Do not print logs to stdout
--show-config : Print the embedded configuration then exit
-V, --version : Print fennec version then exit
Compile with dependencies
👨💻
fennec depends on osquery to run the artifacts with the type query. The directory called deps contains the file that will be embedded into the binary depending on the target OS and architecture, Before compiling follow the below steps:
Modify the configuration file deps//config.yaml as needed
You can also use the precompiled binaries in the release section.
Examples
🚀
Default Configurations
The following is an example ran on Ubuntu 20 with the same configurations in this repo:
Using Fennec with Kuiper
To output data to Kuiper supported format execute Fennec with the following argument:
sudo ./fennec --output-format kjson
or add the following to the args section in the configuration:
args:
- "--output-format"
- "kjson"
recompile then execute:
sudo ./fennec
then upload the resulting zip file to Kuiper, the following is an example:
Configuration
🔨
By default the configuration in the path deps/config.yaml will be embedded into the executable during compilation. The configuration is in YAML format and have two sections:
Args
contains a list of arguments to be passed to the executable as command line arguments, the following is an example for the args section that will set the output format to jsonl and the log file name to fennec.log:
The command line arguments will be used in the following priorities:
arguments passed to the executable
arguments in the configuration file
default arguments
Artifacts
Contains a list of artifacts to be collected. Each artifact contains the following fields:
name: the name of the artifact, the results of the artifact will be written to a file with this name
type: the type of the artifact, the supported artifacts are:
query
collection
command
parse
description (optional): contain description about the artifact
quires OR paths OR commands: quires if the artifact type is query and it contains a list of osquery SQL queries. paths if the artifact type is collection OR parse and it contains a list of paths. commands if the artifact type is command and it contains a list commands. These names are for the sake of readability ,you can use any of them in any artifact type.
regex: this field is only used if the artifact type parse is used, this field contains regex to parse text file
maps (optional): contains a list of mappers to modify key names and format values, check the maps section for more details
Artifact Types: Query
Execute osquery SQL queries. The following example artifact to retrieve all users on the system:
artifacts:
- name: userstype: querydescription: "List all local users"queries:
- 'select * from groups join user_groups using (gid) join users using (uid)'
...
Artifact Types: Collection
This artifact type collect files/folders specified in the field paths. The following is an example of this artifact type that collect system logs:
This artifact type provides the ability to parse text files using regex and return the data it in structured format. The example bellow parse nginx access logs and return the results in structured format:
This configuration will read the files in the path /var/log/nginx/access.* line by line and run the regex to extract fields. This artifact also check if the file is in gzip format which is used to compress old logs to save space and decompresses them and parses them. The regex should be in named captures format as documented in the rust regex library. The following is an example nginx access record before and after parsing:
This optional field can be used to change result field names and run post processing called modifiers on the field value. The below example will show the results for parsing nginx access record without maps:
To change the field name time to @timestamp we add the following maps configuration to the artifact configurations:
[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}) - (?P[^ ]+) \[(?P[0-9]{2}/[a-zA-Z]{3}/[0-9]{4}:[0-9]{2}:[0-9]{2}:[0-9]{2} \+[0-9]{4})\] "(?P[A-Z]+)?[ ]?(?P.*?)[ ]?(HTTP/(?P[0-9\.]+))?" (?P[0-9]{3}) (?P[0-9]+) "(?P.*?)" "(?P.*?)"'
maps:
- from: time #change field name from
to: '@timestamp' # to this name">
artifcats:
- name: nginx_accesstype: parsedescription: "Nginx access logs"paths:
- /var/log/nginx/access.*regex: '(?P[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}) - (?P[^ ]+) \[(?P[0-9]{2}/[a-zA-Z]{3}/[0-9]{4}:[0-9]{2}:[0-9]{2}:[0-9]{2} \+[0-9]{4})\] "(?P[A-Z]+)?[ ]?(?P.*?)[ ]?(HTTP/(?P[0-9\.]+))?" (?P[0-9]{3}) (?P[0-9]+) "(?P.*?)" "(?P.*?)"'maps:
- from: time #change field name fromto: '@timestamp'# to this name
After running the collection tool with the configuration on the same nginx access log we get the following output:
modifiers provides post processing on field value of the artifact results. For example reformatting date and time. Continuing on the example above we can change the date and time format in the field @timestamp to the format %Y-%m-%d %H:%M:%S. We can add the following to the artifact configurations to accomplish that:
Trying to build the static binary with the provided command fails.
$ RUSTFLAGS="-C target-feature=+crt-static" cargo build --release --target x86_64-unknown-linux-gnu
[...]
error: linking with `cc` failed: exit status: 1
|
= note: "cc" "-m64" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/fennec-d0207c07e2fe808f.fennec.c1597b5f-cgu.0.rcgu.o" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/fennec-d0207c07e2fe808f.fennec.c1597b5f-cgu.1.rcgu.o" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/fennec-d0207c07e2fe808f.fennec.c1597b5f-cgu.10.rcgu.o" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/fennec-d0207c07e2fe808f.fennec.c1597b5f-cgu.11.rcgu.o" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/fennec-d0207c07e2fe808f.fennec.c1597b5f-cgu.12.rcgu.o" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/fennec-d0207c07e2fe808f.fennec.c1597b5f-cgu.13.rcgu.o" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/fennec-d0207c07e2fe808f.fennec.c1597b5f-cgu.14.rcgu.o" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/fennec-d0207c07e2fe808f.fennec.c1597b5f-cgu.15.rcgu.o" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/fennec-d0207c07e2fe808f.fennec.c1597b5f-cgu.2.rcgu.o" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/fennec-d0207c07e2fe808f.fennec.c1597b5f-cgu.3.rcgu.o" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/fennec-d0207c07e2fe808f.fennec.c1597b5f-cgu.4.rcgu.o" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/fennec-d0207c07e2fe808f.fennec.c1597b5f-cgu.5.rcgu.o" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/fennec-d0207c07e2fe808f.fennec.c1597b5f-cgu.6.rcgu.o" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/fennec-d0207c07e2fe808f.fennec.c1597b5f-cgu.7.rcgu.o" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/fennec-d0207c07e2fe808f.fennec.c1597b5f-cgu.8.rcgu.o" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/fennec-d0207c07e2fe808f.fennec.c1597b5f-cgu.9.rcgu.o" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/fennec-d0207c07e2fe808f.4yiprfv4tsne6ya8.rcgu.o" "-Wl,--as-needed" "-L" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps" "-L" "/home/harrim4n/git/Fennec/target/release/deps" "-L" "/usr/lib" "-L" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libhostname-3a96cfda79cb2fe2.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libmatch_cfg-dfed388bebfa04ac.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/librust_embed-dd10f4a16930fcd2.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libinclude_flate-15797ecd2c57d251.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libinclude_flate_codegen_exports-c77c89117375e18c.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/liblibflate-372dfaf27d0d5e55.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libadler32-0f201d3f77da7d13.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/liblibflate_lz77-3a552dc88831a03e.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/librle_decode_fast-b5b993c9852f0cff.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/librust_embed_utils-859b44917c7765b5.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libwalkdir-676b4c88f8885f93.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libsame_file-b498e9a1b2aae505.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libsha2-8640ee2d06567252.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libcpufeatures-7c414830675836fe.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libopaque_debug-6b1bfd88b0562197.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libdigest-c547f8eaa72f3047.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libblock_buffer-08a085b787e9b46b.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libgeneric_array-5b109808bb92d2b4.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libtypenum-28735e42692ee586.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libnix-ece2e07b4688b0aa.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libmemoffset-6607e60557f852e3.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/liblog4rs-1e46e4f4b8237564.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/liblog_mdc-f2854a52098a572c.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libthread_id-fb83e9881d3b266e.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libhumantime-53443defbae3585b.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libtypemap-691f8bced5cedc5b.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libunsafe_any-ead8a6b3dcbc5e8e.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libtraitobject-c6ec9a9c01fa8649.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libanyhow-b867314b896794ac.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libparking_lot-73e572d72916b40b.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libparking_lot_core-3a9a0a91a813ee42.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libsmallvec-5bd5d902a4ae9784.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/liblock_api-4ff09505d67cc3da.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libscopeguard-d6e045e585b1b813.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libinstant-c820658d7c55a054.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libserde_value-a405da3880f4602b.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libordered_float-10ce90271cf5a99f.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libfnv-bc983b123c1426dd.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libarc_swap-e93f9cb83939a9d8.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libfennec-72e4b31b9b22cd86.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libserde_yaml-78cc1c0471417b85.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libdtoa-2b351bff0f9fa225.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libosquery_rs-8d10ffc7323802dc.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libthrift-b00158c7f5d2a31f.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libordered_float-6af0d5d9c0e8dce8.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libthreadpool-88dcb82f11475bbb.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libnum_cpus-c69d7baf1e0b2086.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libinteger_encoding-3ed3222e0137e5fd.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libcsv-6c69ab5e87737ccf.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libcsv_core-84e5a8ecc3c002f5.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libbstr-dda1e7f4129019c8.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libregex_automata-013daeecfbfd88c6.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libserde_json-f0cf45172c37ee2e.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libryu-6c0fa009e17ec9b1.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libitoa-036492ed62c89270.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/liblog-d126bf4683149762.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libserde-0471122b6cc17c40.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libglob-4b5805bb5dd3b4bf.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libzip-a0a8fb9f2212b249.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libthiserror-e001f9e618a33593.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libbzip2-1234bbd3e8ae1070.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libbzip2_sys-f0ca4f5ffbcbae6f.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libbyteorder-3bf0337d898a576d.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libregex-233a48ae026c5b6d.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libaho_corasick-f6420e862a10916e.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libregex_syntax-e46617c6f1f05a86.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libreadable_perms-c9fd7c7794a41a8e.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libflate2-eae0e64fbd168c8c.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libminiz_oxide-03eff55fc8ad0fc2.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libadler-6c477f5e35e4e4f0.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libcrc32fast-eed1cab3d1ca6730.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libcfg_if-b29290bb60f199f8.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libchrono-53d5d42129f9dd38.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libnum_integer-1872fb53146740c3.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libnum_traits-f796b13bf2a12399.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libtime-0196a423301af73d.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libcolored-0c0c6c080e8fdae3.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/liblazy_static-a6ca9dc7ad155658.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libclap-2b1158addc99a7ec.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libstrsim-edce8bfe43d1c1ef.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libatty-e7ed1a6d30aa6e71.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/liblibc-5a70bb2d93691053.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libtermcolor-58dae2fb28134c5b.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libtextwrap-224a7b4bd3d6b435.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libindexmap-1d31e82cd1f3ed4c.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libhashbrown-33c67aaf4e974a21.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libos_str_bytes-e57ead120d354af8.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libmemchr-44da601249921870.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libbitflags-52d0d4f1fe81a1b9.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libyaml_rust-186225488d3edbc5.rlib" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/liblinked_hash_map-4fe676da2dd350de.rlib" "-Wl,--start-group" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-4c74cbab78ec4891.rlib" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-0ef58120f7b95253.rlib" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-e35e56ad39c7e20e.rlib" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-671a9f10c55c6c87.rlib" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-ee577127549b7793.rlib" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-bed369233e55d851.rlib" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-e8504b1ed73d6c6f.rlib" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-411eeeec028606dc.rlib" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-0ddec007a0883060.rlib" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-7c5cb27d99d10614.rlib" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-6c448d94453f4d95.rlib" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-22835d1ac5e3244b.rlib" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-84878e033904a7a4.rlib" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-c0badcb9f7c5eab7.rlib" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-b4424726f33da388.rlib" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-lgcc_eh" "-lgcc" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-aa0bad4c4d134922.rlib" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-483ad457673e0f5c.rlib" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-6cfcec236d576603.rlib" "-Wl,--end-group" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-5667a4a7e2c48d47.rlib" "-Wl,-Bdynamic" "-lbz2" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L" "/home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/fennec-d0207c07e2fe808f" "-Wl,--gc-sections" "-static" "-no-pie" "-Wl,-zrelro,-znow" "-Wl,-O1" "-nodefaultlibs"
= note: /usr/bin/ld: /home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libnix-ece2e07b4688b0aa.rlib(nix-ece2e07b4688b0aa.nix.fce68b46-cgu.11.rcgu.o): in function `nix::unistd::getgrouplist':
nix.fce68b46-cgu.11:(.text._ZN3nix6unistd12getgrouplist17he1280a78b77f3550E+0x9b): warning: Using 'getgrouplist' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: /home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libnix-ece2e07b4688b0aa.rlib(nix-ece2e07b4688b0aa.nix.fce68b46-cgu.11.rcgu.o): in function `nix::unistd::initgroups':
nix.fce68b46-cgu.11:(.text._ZN3nix6unistd10initgroups17h826d7feceec2d2c0E+0x5): warning: Using 'initgroups' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: /home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libnix-ece2e07b4688b0aa.rlib(nix-ece2e07b4688b0aa.nix.fce68b46-cgu.11.rcgu.o): in function `nix::unistd::Group::from_gid':
nix.fce68b46-cgu.11:(.text._ZN3nix6unistd5Group8from_gid17hc3925ff4d7c7b54dE+0xe2): warning: Using 'getgrgid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: /home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libnix-ece2e07b4688b0aa.rlib(nix-ece2e07b4688b0aa.nix.fce68b46-cgu.11.rcgu.o): in function `nix::unistd::Group::from_name':
nix.fce68b46-cgu.11:(.text._ZN3nix6unistd5Group9from_name17had0638ce1cd8251dE+0x116): warning: Using 'getgrnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: /home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libnix-ece2e07b4688b0aa.rlib(nix-ece2e07b4688b0aa.nix.fce68b46-cgu.11.rcgu.o): in function `nix::unistd::User::from_name':
nix.fce68b46-cgu.11:(.text._ZN3nix6unistd4User9from_name17h14c72a0ecbe1950eE+0x126): warning: Using 'getpwnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: /home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/libnix-ece2e07b4688b0aa.rlib(nix-ece2e07b4688b0aa.nix.fce68b46-cgu.11.rcgu.o): in function `nix::unistd::User::from_uid':
nix.fce68b46-cgu.11:(.text._ZN3nix6unistd4User8from_uid17hf72678591dfd2ae5E+0xe2): warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: /home/harrim4n/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-4c74cbab78ec4891.rlib(std-4c74cbab78ec4891.std.f6ca25d7-cgu.0.rcgu.o): in function `<std::sys_common::net::LookupHost as core::convert::TryFrom<(&str,u16)>>::try_from':
/rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b//library/std/src/sys_common/net.rs:191: warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: attempted static link of dynamic object `/usr/lib/libbz2.so'
/usr/bin/ld: /home/harrim4n/git/Fennec/target/x86_64-unknown-linux-gnu/release/deps/liblibflate_lz77-3a552dc88831a03e.rlib(libflate_lz77-3a552dc88831a03e.libflate_lz77.467f1e72-cgu.2.rcgu.o): undefined reference to symbol '__tls_get_addr@@GLIBC_2.3'
/usr/bin/ld: /usr/lib/ld-linux-x86-64.so.2: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
= help: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
= note: use the `-l` flag to specify native libraries to link
= note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)
error: could not compile `fennec` due to previous error
While executing it on x64 linux (ubuntu) I see some errors in log:
2022-12-09 21:03:20 [fennec:422] ERROR Unable to execute osquery SQL query 'select cmdline,cwd,disk_bytes_read,disk_bytes_written,egid,euid,gid,name,nice,on_disk,parent,processes.path,md5,pgroup,pid,resident_size,root,sgid,start_time,state,suid,system_time,threads,total_size,uid,user_time,wired_size from processes JOIN hash USING (path)', ERROR: Unable to execute the query 'select cmdline,cwd,disk_bytes_read,disk_bytes_written,egid,euid,gid,name,nice,on_disk,parent,processes.path,md5,pgroup,pid,resident_size,root,sgid,start_time,state,suid,system_time,threads,total_size,uid,user_time,wired_size from processes JOIN hash USING (path)', ERROR: transport error
2022-12-09 21:03:26 [fennec:422] ERROR Unable to execute osquery SQL query 'SELECT path,md5,username,groupname,permissions FROM suid_bin JOIN hash USING (path)', ERROR: Unable to execute the query 'SELECT path,md5,username,groupname,permissions FROM suid_bin JOIN hash USING (path)', ERROR: transport error
2022-12-09 21:03:30 [fennec:422] ERROR Unable to execute osquery SQL query 'select * from groups join user_groups using (gid) join users using (uid)', ERROR: Unable to execute the query 'select * from groups join user_groups using (gid) join users using (uid)', ERROR: transport error
2022-12-09 21:03:37 [fennec:422] ERROR Unable to execute osquery SQL query 'select * from file where path like "/home/%%"', ERROR: Unable to execute the query 'select * from file where path like "/home/%%"', ERROR: transport error
2022-12-09 21:03:41 [fennec:422] ERROR Unable to execute osquery SQL query 'select * from file where path like "/root/%%"', ERROR: Unable to execute the query 'select * from file where path like "/root/%%"', ERROR: transport error
when trying to run with sudo ./target/release/fennec I get:
2022-02-13 14:54:25 [fennec:258] INFO Started 'fennec'
2022-02-13 14:54:25 [fennec:331] INFO Successfuly wrote '46276872' bytes to osquery file './osqueryd'
2022-02-13 14:54:25 [fennec:419] ERROR Unable to collect triage image, ERROR: 'Unable to create osquery instance './osqueryd', ERROR: Exec format error (os error 8)'
2022-02-13 14:54:25 [fennec:428] INFO Successfuly deleted the file 'fennec.log'
2022-02-13 14:54:25 [fennec:428] INFO Successfuly deleted the file './osqueryd'
2022-02-13 14:54:25 [fennec:439] INFO Done!
Setup is Ubuntu 21.10 (lsb_release)
Distributor ID: Ubuntu
Description: Ubuntu 21.10
Release: 21.10
Codename: impish
Running as VM (with multipass) on arm64 (mac silicon)
Does $subj mean that osquery must be installed on the subject system? (i.e. works only with osquery)
Or can it run collection regardless of it?
Cheers!
threat 'main' panicked at 'called 'Result::unwrap()' on an 'Err' value: Custom { kind: Other, error: "Large file option has not been set" }' , src/lib/mod.rs:542:87 note: run with 'RUST_BACKTRACE=1' environment variable to display a backtrace
Distributor ID: Ubuntu
Description: Ubuntu 20.04.3 LTS
Release: 20.04
Codename: focal
rustc 1.53.0
cargo 1.53.0
Linux u20 5.4.0-99-generic #112-Ubuntu SMP Wed Feb 2 17:13:12 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
ubuntu@u20:~/code/fennec.upstream$ cargo build --release
error: failed to parse manifest at /home/ubuntu/code/fennec.upstream/Cargo.toml
Caused by:
feature edition2021 is required
consider adding cargo-features = ["edition2021"] to the manifest
Once added, I get another error:
Compiling digest v0.9.0
error[E0658]: arbitrary expressions in key-value attributes are unstable
--> /home/ubuntu/.cargo/registry/src/github.com-1ecc6299db9ec823/clap-3.0.10/src/lib.rs:8:39
|
8 | #![cfg_attr(feature = "derive", doc = include_str!("../README.md"))]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: see issue #78835 https://github.com/rust-lang/rust/issues/78835 for more information
Compiling block-buffer v0.9.0
error: aborting due to previous error
For more information about this error, try rustc --explain E0658.
error: could not compile clap
The parse artifact crashes Fennec if the read file is not in UTF-8 format with the error message:
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { kind: InvalidData, message: "stream did not contain valid UTF-8" }', src/lib/mod.rs:787:78
A simple to use rust package to generate or parse Twitter snowflake IDs,generate time sortable 64 bits unique ids for distributed systems (inspired from twitter snowflake)