Rust bindings for VirusTotal/Yara

Overview

yara-rust

Build Status Crates.io Documentation

Bindings for the Yara library from VirusTotal.

More documentation can be found on the Yara's documentation.

Example

The implementation is inspired from yara-python.

const RULES: &str = r#"
    rule contains_rust {
      strings:
        $rust = "rust" nocase
      condition:
        $rust
    }
"#;

fn main() {
    let compiler = Compiler::new().unwrap();
    compiler.add_rules_str(RULES)
        .expect("Should have parsed rule");
    let rules = compiler.compile_rules()
        .expect("Should have compiled rules");
    let results = rules.scan_mem("I love Rust!".as_bytes(), 5)
        .expect("Should have scanned");
    assert!(results.iter().any(|r| r.identifier == "contains_rust"));
}

Features

  • Support from Yara v4.1.
  • Compile rules from strings or files.
  • Save and load compiled rules.
  • Scan byte arrays (&[u8]) or files.

Feature flags and Yara linking.

Look at the yara-sys crate documentation for a list of feature flags and how to link to your Yara crate.

TODO

  • Remove some unwrap on string conversions (currently this crate assume the rules, meta and namespace identifier are valid Rust's str).
  • Accept AsRef<Path> instead of &str on multiple functions.
  • Implement the scanner API.
  • Add process scanning.
  • Report the warnings to the user.

License

Licensed under either of

at your option.

Contributing

Please follow the conventional commit rules when committing to this repository.

If you add any new feature, add the corresponding unit/doc tests.

Comments
  • Add compatibility to yara 4.0

    Add compatibility to yara 4.0

    Attempt to make yara and yara-sys compatible with libyara 4.0.2

    It turns out that keeping the 3.X compatibility will be harder than I though.

    Closes #11

    opened by Hugal31 13
  • pe.number_of_signatures panics an error

    pe.number_of_signatures panics an error

    Hi,

    I used the option vendored on the toml config file and all went good but once it parses any Yara rule that contains pe.number_of_signatures , it panics as follow:

    thread 'main' panicked at 'Should have parsed rule: Compile(CompileErrors { errors: [CompileError { level: Error, filename: Some("C:\Users\user\yarafolder\test.yar"), line: 2009, message: "invalid field name "number_of_signatures"" }, CompileError

    condition:
         for any i in (0 .. pe.number_of_signatures)
    

    Toml configuration as follow:

    [dependencies] yara = "0.16.0" yara-sys ={ version = "0.16.0", features = ["bundled-4_2_3","vendored"] }

    what have I done to make not work :(?

    opened by muteb 9
  • bindgen pre-generated size_t has the wrong type on x86 and causes UB

    bindgen pre-generated size_t has the wrong type on x86 and causes UB

    When I'm compiling for windows x86, the following trivial test fails:

        #[test]
        fn check_yara_scan_mem() {
            const RULES: &str = r#"
                rule test {
                    strings:
                        $a = { 42 }
                    condition:
                        $a
                }
            "#;
    
            let rules = Compiler::new()
                .unwrap()
                .add_rules_str(RULES)
                .unwrap()
                .compile_rules()
                .unwrap();
            let matching = rules.scan_mem(&[0x00, 0x01, 0x42, 0x03], 10)
                .unwrap();
            assert_eq!(matching.len(), 1, "byte not found");
        }
    
    check_yara_scan_mem panicked at 'called `Result::unwrap()` on an `Err` value: YaraError { kind: Unknown(53) }
    

    On the line corresponding to rules_scan_mem().unwrap(). Error 53 corresponds to ERROR_CALLBACK_REQUIRED.

    In yara-rust, the call boils down to this code in internals::scan :

        let result = unsafe {
            yara_sys::yr_rules_scan_mem(
                rules,
                mem.as_ptr(),
                mem.len().try_into().unwrap(),
                flags,
                Some(scan_callback),
                &p_callback as *const Box<dyn FnMut(CallbackMsg<'a>) -> CallbackReturn> as *mut _,
                timeout,
            )
        };
    

    Some(scan_callback) is the function pointer to the callback, and p_callback is the user data that will be passed by libyara as an argument to scan_callback. (It's a pointer the actual callback closure that will be reconstructed by our scan_callback and then called, but this is not relevant here).

    The thing is: Some(scan_callback) is definitely not a function pointer, and does not have the same size as a function pointer.

    Here are the arguments that rust pushes on the stack before doing the FFI call:

    [esp + 0x1c] = 0xa (timeout)
    [esp + 0x18] = 0x1c23efc0 (&p_callback)
    [esp + 0x14] = 0xebd820 (fn scan_callback)
    [esp + 0x10] = 0x0 (tag for variant "Some")
    [esp + 0x0c] = 0x0 (flags)
    [esp + 0x08] = 0x4 (mem len]
    [esp + 0x04] = 0x019b57d4 (mem addr)
    [esp + 0x00] = 0x143b04c8 (rules)
    

    This is one stack slot more than expected by the C function yr_rules_scan_mem, because of the space needed for the Option's variant tag. Because of this, yr_rules_scan_mem sees all variables shifted by one slot:

    Screenshot_20211001_231855

    callback is now 0x0 (the tag for variant Some), user_data is now callback, and timeout amounts to the address of user_data as seconds.

    Officially, yr_rules_scan_mem is defined in rules.h as

    YR_API int yr_rules_scan_mem(
        YR_RULES* rules,
        const uint8_t* buffer,
        size_t buffer_size,
        int flags,
        YR_CALLBACK_FUNC callback,
        void* user_data,
        int timeout)
    

    where YR_CALLBACK_FUNC is

    typedef int (*YR_CALLBACK_FUNC)(
        YR_SCAN_CONTEXT* context,
        int message,
        void* message_data,
        void* user_data);
    

    Definitely no Option here. However, bindgen (I'm using the bundled bindings) has generated:

    pub type YR_CALLBACK_FUNC = ::std::option::Option<
        unsafe extern "C" fn(
            context: *mut YR_SCAN_CONTEXT,
            message: ::std::os::raw::c_int,
            message_data: *mut ::std::os::raw::c_void,
            user_data: *mut ::std::os::raw::c_void,
        ) -> ::std::os::raw::c_int,
    >;
    

    No idea where this Option comes from.

    Somehow this bug is miraculously not triggered in x64, either because the ABI is different (no stack slots, args are passed via registers), or because Option<unsafe extern "C" fn(...)> is properly optimized to be the same size as the function pointer, just like Option<NonNull<T>> is.

    For now my conclusion is that bindgen is at fault here. I'll try to pinpoint the reason, and when I find it I'll open an issue on their repo linking to this one.

    opened by Orycterope 8
  • Build failed on macos

    Build failed on macos

    hi,

    with "vendored" feature enabled, I got the following build error on Macos

    /Users/../.cargo/bin/cargo build --color=always --message-format=json-diagnostic-rendered-ansi --package .. --bin... Compiling yara-sys v0.8.0 error: failed to run custom build command for yara-sys v0.8.0 Caused by: process didn't exit successfully: /Users/../working/../target/debug/build/yara-sys-040eb9438488a67e/build-script-build (signal: 6, SIGABRT: process abort signal) --- stderr WARNING: /Users/../working/../target/debug/build/yara-sys-040eb9438488a67e/build-script-build is loading libcrypto in an unsafe way Process finished with exit code 101

    bug 
    opened by bigdogs 8
  • Compilation Error

    Compilation Error

    Hi,

    Thank you very much for creating this awesome crate. I add the yara crate to the toml and compile it but I faced this error.

    error: failed to run custom build command for `yara-sys v0.4.2`
    
    Caused by:
      process didn't exit successfully: `C:\Users\*****\Desktop\yara-rust\rsyara-scanner\target\debug\build\yara-sys-ab4856629aaf30fc\build-script-build` (exit code: 101)
      --- stdout
      cargo:rustc-link-lib=dylib=yara
    
      --- stderr
      wrapper.h:1:10: fatal error: 'yara.h' file not found
      wrapper.h:1:10: fatal error: 'yara.h' file not found, err: true
      thread 'main' panicked at 'Unable to generate bindings: ()', C:\Users\***\.cargo\registry\src\github.com-1ecc6299db9ec823\yara-sys-0.4.2\build.rs:106:14
      note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
      
    

    Is there a way to solve this issue!

    Thanks.

    opened by muteb 7
  • Support Yara 3.10, link to default Yara package dependencies

    Support Yara 3.10, link to default Yara package dependencies

    • Reworked build.rs to support macOS, Linux, and now OpenBSD
    • Static link to all dependencies needed by typical Yara package (cuckoo needs jansson, file needs magic)
    • Update docs and links to reflect Yara 3.10
    opened by pwrdwnsys 7
  • Add MacOS build

    Add MacOS build

    Fix: https://github.com/Hugal31/yara-rust/issues/40

    I only added build without tests, since I am having problems with test on MacOS: https://github.com/ikrivosheev/yara-rust/runs/3270599324?check_suite_focus=true#step:4:204

    opened by ikrivosheev 6
  • fix: bundled_3.X on windows

    fix: bundled_3.X on windows

    A proposal to address #19, tell me what you think of it.

    I've renamed bindings/yara-3.X.rs to bindings/yara-3.X-unix.rs, and generated bindings/yara-3.X-windows.rs with bindgen. Looking at the diff, you'll see that besides private internal structure layout and types, in the actually exposed API only the functions that take in a fd differ.

    I've updated the yara-sys/build.rs to use the right file at compile time.

    And finally I've added two scenarios in .tarvis-ci.yml:

    TARGET=x86_64-pc-windows-msvc   YARA_FEATURES=vendored,bundled-3_11
    TARGET=x86_64-pc-windows-msvc   YARA_FEATURES=vendored,bundled-3_8
    

    The second one is technically invalid, since it uses 3.8 bindings on a 3.11 lib built by cc (vendored), but because of backcompability it should work.

    opened by Orycterope 6
  • Add callback API into Rules

    Add callback API into Rules

    Hello @Hugal31 , sorry for my english, I will try to describe what I want to do.

    1. I kept the old api for backward compatibility
    2. Callback api provides more control over scanning
    3. I am writing an engine that will be integrated with python. To avoid converting from a Vec to PyList, I want to immediately add Rule to PyList

    If this PR is Ok, I can write Callback API for Scanner

    opened by ikrivosheev 5
  • Vendored libyara

    Vendored libyara

    Added support for building and linking vendored libyara (useful when you want to statically link libyara).

    I tested the crate's tests (with vendored feature) on Windows 10, macOS, and Ubuntu.

    lmk if there's anything else you want me to change or test

    opened by yoavk 5
  • fix: Drop struct

    fix: Drop struct

    Problem was describe here: https://github.com/Hugal31/yara-rust/issues/74. After fix drop, I got: Segmentation fault (core dumped), because context.matches was NULL and I fix it too.

    opened by ikrivosheev 4
  • fix: add base offset for scanning memory chunks

    fix: add base offset for scanning memory chunks

    This is needed to calculate the offset of a match when using iterators over blocks.

    See base at https://yara.readthedocs.io/en/v4.2.3/capi.html?highlight=YR_RULES#c.YR_MATCH

    opened by tankbusta 1
  • Add support for setting YR_MODULE_IMPORT::module_data

    Add support for setting YR_MODULE_IMPORT::module_data

    CallbackMsg::ImportModule should be extended to access YR_MODULE_IMPORT data, namely module_name and module_data(_size).

    Some modules accept additional module data, e.g. cuckoo module accepts json, which enables extended yara scanning.

    This functionality is already exposed in python bindings (https://github.com/VirusTotal/yara-python/blob/master/yara-python.c#L618) and it would be lovely to have this functionality in this awesome rust crate :)

    enhancement 
    opened by wbenny 3
  • Proc scanning test fails randomly on CI

    Proc scanning test fails randomly on CI

    The unit test rules::test::rules_scan_proc failed on a CI build with the configuration ubuntu-20.04,vendored,bindgen,nightly. I ran it again and it passed.

    May be related to #25.

    opened by Hugal31 0
  • Memory leak

    Memory leak

    I tested my application and see a huge growth of memory! Files to reproduce error: 16025.tar.gz.zip (this is simple txt files)

    Simple code to reproduce:

    use std::os::unix::fs::MetadataExt;
    use std::sync::Arc;
    use threadpool::ThreadPool;
    use yara::Compiler;
    
    const WORKERS: usize = 4;
    const RULES: &str = "..."; // path to rules
    const FILE: &str = "...";  // path to files to scan
    
    fn main() {
        let pool = ThreadPool::new(WORKERS);
        let compiler = Compiler::new().unwrap();
        let compiler = compiler.add_rules_file(RULES).unwrap();
        let rules = Arc::new(compiler.compile_rules().unwrap());
        loop {
            let mut counter = 0;
            println!("Start iter");
    
            for file in globwalk::GlobWalkerBuilder::new(FILE, "**")
                .build()
                .unwrap()
            {
                if let Ok(file) = file {
                    counter += 1;
                    if file.metadata().unwrap().size() == 0 {
                        continue;
                    }
                    let rules = rules.clone();
                    pool.execute(move || {
                        let mut scanner = rules.scanner().unwrap();
                        scanner.set_timeout(60);
                        let _ = scanner.scan_file(file.path());
                    });
                }
            };
            pool.join()
            println!("Finish iter, files={}", counter);
            std::thread::sleep(std::time::Duration::from_secs(7));
    
        }
    }
    

    What am I doing wrong?

    opened by ikrivosheev 10
  • error: could not find native static library `yara`, perhaps an -L flag is missing?

    error: could not find native static library `yara`, perhaps an -L flag is missing?

    • error message
    $ cargo build
       Compiling yara-sys v0.6.1
       Compiling yara v0.6.1
    error: could not find native static library `yara`, perhaps an -L flag is missing?
    
    • Cargo.toml
    [dependencies]
    yara = "0.6.1"
    yara-sys = { version = "0.6.1", features = ["bundled-4_1_0"] }
    
    opened by jiabochao 1
  • Windows compilation

    Windows compilation

    Hi,

    We're trying to cross-compile this little program:

    use yara::Compiler;
    
    const RULES: &str = r#"
        rule contains_rust {
          strings:
            $rust = "rust" nocase
          condition:
            $rust
    "#;
    
    fn main() {
        let mut compiler = Compiler::new().unwrap();
        compiler.add_rules_str(RULES)
            .expect("Should have parsed rule");
        let rules = compiler.compile_rules()
            .expect("Should have compiled rules");
        let results = rules.scan_mem("I love Rust!".as_bytes(), 5)
            .expect("Should have scanned");
        assert!(results.iter().any(|r| r.identifier == "contains_rust"));
    }
    
    

    to Windows from Linux by issuing:

    cargo build --bin thomas --target x86_64-pc-windows-gnu

    [package]
    name = "thomas"
    version = "0.1.0"
    authors = ["Sonny <[email protected]>"]
    edition = "2018"
    
    # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
    [[bin]]
    name = "thomas"
    path = "src/mymain.rs"
    
    [dependencies]
    yara = "0.4.3"
    yara-src = "0.1.2+3.11.0"
    

    We have set the correct inc dir: YARA_INCLUDE_DIR="../yara-3.11.0/libyara/include"

    The error we get is:

    warning: version requirement `0.1.2+3.11.0` for dependency `yara-src` includes semver metadata which will be ignored, removing the metadata is recommended to avoid confusion
       Compiling yara v0.4.3
    error[E0308]: mismatched types
       --> /volumes/SoonrRepo/Users/tclausen/.cargo/registry/src/github.com-1ecc6299db9ec823/yara-0.4.3/src/internals/compiler.rs:115:13
        |
    115 |             handle,
        |             ^^^^^^ expected `i32`, found *-ptr
        |
        = note:     expected type `i32`
                found raw pointer `*mut c_void`
    
    error[E0308]: mismatched types
       --> /volumes/SoonrRepo/Users/tclausen/.cargo/registry/src/github.com-1ecc6299db9ec823/yara-0.4.3/src/internals/scan.rs:130:13
        |
    130 |             handle,
        |             ^^^^^^ expected `i32`, found *-ptr
        |
        = note:     expected type `i32`
                found raw pointer `*mut c_void`
    
    error: aborting due to 2 previous errors
    
    For more information about this error, try `rustc --explain E0308`.
    error: could not compile `yara`
    
    To learn more, run the command again with --verbose.
    
    
    enhancement 
    opened by tclausen 7
Releases(v0.12.0)
  • v0.12.0(Oct 29, 2021)

    ⚠ BREAKING CHANGES

    • The feature bundled-4_1_2 becomes bundled-4_1_3. However, since there are no ABI changes between the two versions, it is still compatible with yara v4.1.2.

    Features

    Source code(tar.gz)
    Source code(zip)
  • v0.11.1(Oct 29, 2021)

  • v0.11.0(Oct 29, 2021)

  • v0.10.0(Sep 20, 2021)

    ⚠ BREAKING CHANGES

    • Compiler.add_rules_* functions now takes Compiler by value and return it if the rule is succesfully added.
    • Minimum Rust version is now 1.55.

    Features

    • yara-sys: vendored feature uses v4.1.2 (18b7ae4)
    • add support for yr_scanner_scan_mem_blocks (e1aa11e)

    Bug Fixes

    • prevent UB when failing to compile a rule (99f756a), closes #47
    Source code(tar.gz)
    Source code(zip)
Owner
null
Rust bindings for libinjection

libinjection-rs Rust bindings for libinjection. How to use Add libinjection to dependencies of Cargo.toml: libinjection = "0.2" Import crate: extern c

ArvanCloud 35 Sep 24, 2022
Rust bindings for the unicorn CPU emulator

unicorn-rs THIS PACKAGE IS DEPRECATED AND NO LONGER MAINTAINED. Rust bindings are now included with unicorn and will be maintained there from now on.

null 129 Oct 10, 2022
Bindings to the macOS Security.framework

macOS/iOS Security framework for Rust Documentation Bindings to the Apple's Security.framework. Allows use of TLS and Keychain from Rust. License Lice

Kornel 172 Jan 2, 2023
Detects usage of unsafe Rust in a Rust crate and its dependencies.

cargo-geiger ☢️ Looking for maintainer: https://github.com/rust-secure-code/cargo-geiger/issues/210 A program that lists statistics related to the usa

Rust Secure Code Working Group 1.1k Jan 4, 2023
Rust-verification-tools - RVT is a collection of tools/libraries to support both static and dynamic verification of Rust programs.

Rust verification tools This is a collection of tools/libraries to support both static and dynamic verification of Rust programs. We see static verifi

null 253 Dec 31, 2022
A simple password manager written in Rust

ripasso A simple password manager written in Rust. The root crate ripasso is a library for accessing and decrypting passwords stored in pass format (G

Joakim Lundborg 548 Dec 26, 2022
tcp connection hijacker, rust rewrite of shijack

rshijack tcp connection hijacker, rust rewrite of shijack from 2001. This was written for TAMUctf 2018, brick house 100. The target was a telnet serve

null 377 Jan 1, 2023
A fast, simple, recursive content discovery tool written in Rust.

A simple, fast, recursive content discovery tool written in Rust ?? Releases ✨ Example Usage ✨ Contributing ✨ Documentation ?? ?? What the heck is a f

epi 3.6k Dec 30, 2022
link is a command and control framework written in rust

link link is a command and control framework written in rust. Currently in alpha. Table of Contents Introduction Features Feedback Build Process Ackno

null 427 Dec 24, 2022
CVEs for the Rust standard library

Rust CVE Preface This is a list of CVEs for unsound APIs in the Rust standard library. These bugs break Rust's memory safety guarantee and lead to sec

Yechan Bae 26 Dec 4, 2022
Rust library for building and running BPF/eBPF modules

RedBPF A Rust eBPF toolchain. Overview The redbpf project is a collection of tools and libraries to build eBPF programs using Rust. It includes: redbp

foniod 1.5k Jan 1, 2023
Rust library for developing safe canisters.

IC Kit This library provides an alternative to ic-cdk that can help developers write canisters and unit test them in their Rust code. Install Add this

Psychedelic 26 Nov 28, 2022
MimiRust - Hacking the Windows operating system to hand us the keys to the kingdom with Rust.

MimiRust - Hacking the Windows operating system to hand us the keys to the kingdom with Rust. MimiRust is a program based on the wdigest attack vector

Thotty 0 Nov 29, 2022
simple multi-threaded port scanner written in rust

knockson simple multi-threaded port scanner written in rust Install Using AUR https://aur.archlinux.org/packages/knockson-bin/ yay -Syu knockson-bin M

Josh Münte 4 Oct 5, 2022
Rust TLS/SSL certificate expiration date from command-line checker

Rust TLS/SSL certificate expiration date from command-line checker

Jose Bovet Derpich 9 Nov 9, 2022
Lightweight slowloris (HTTP DoS) implementation in Rust.

slowlorust Lightweight slowloris (HTTP DoS) implementation in Rust. Slowloris is a denial-of-service attack program which allows an attacker to overwh

Michael Van Leeuwen 6 Sep 29, 2022
A simple port scanner built using rust-lang

A simple port scanner built using rust-lang

Krisna Pranav 1 Nov 6, 2021
Safe Rust interface to the Vulkan API.

Magma: A strictly typed Vulkan API interface. Magma is a strictly typed Rust interface for the vulkan API. This means that whenever possible, the well

null 1 Oct 11, 2022
A rust program to bruteforce ZIP, PDF and some popular hashes.

Veldora A program to bruteforce zips, pdfs and some popular hashes. This is basically a rust version of bruttle, but a lot faster. Installation: git c

Aquib 30 Dec 28, 2022