Binary Analysis Framework in Rust

Overview

Build Status Crates.io Version Documentation irc.freenode.net#rust-falcon

Welcome to Falcon

Falcon is a formal binary analysis framework in Rust.

  • Expression-based IL with strong influences from RREIL and Binary Ninja's LLIL.
  • Semantically-equivalent binary translators for 32/64-bit x86, Mips, and Mipsel.
  • Lifters for ELF and PE via goblin.
  • Fixed-point engine for data-flow analysis and abstract interpretation.
  • Performant memory models for analysis.
  • A concrete executor over Falcon IL.

Building

  • Several scripts to get you up-and-running with Falcon can be found in the scripts/ directory.
  • Dependencies are capstone and clang.
  • Falcon works out of the box with capstone4.

Questions / Support

  • We have a Gitter. This is the most reliable way to contact us.
  • You can also find me in the Binary Ninja slack under the name "endeavor".
  • There is an infrequently checked IRC channel at irc.freenode.net#rust-falcon.
Comments
  • Translator::translate_function_extended leads to Index does not exist for set entry

    Translator::translate_function_extended leads to Index does not exist for set entry

    When retrieving the Loader::program of cyberblogger from https://github.com/trailofbits/cb-multios (compiled with clang 8.0.1) an error Err("Index does not exist for set_entry") is returned from the line control_flow_graph.set_entry(block_indices[&function_address].0)?; at the end of translate_function_extended. This occurs because for the function at address 0x45008 returns from the translate_block call in the x86 decoder immediately due to a CS_ERR_OK. This leaves the translation block with no instructions which results in a bogus insertion to the block_indices at block_indices.insert(*result.0, (block_entry, block_exit)); in translate_function_extended because block entry is set to 0 and block exit is set to 0 as nothing was inserted into the overall Function CFG and the block_entry variable was never set.

    My hacky fix for an empty translation block is up at https://github.com/2over12/falcon

    I simply check if there are no instructions for a block and add a new empty CFG to the block if so. There probably should not be an empty function though, it is currently unclear to me why falcon_capstone immediately returns a CS_ERR_OK, but it also probably worth handling an empty function in some sort of sane way.

    opened by 2over12 9
  • towards a rust reversers datastructures crate

    towards a rust reversers datastructures crate

    Hello!

    So I've been reading project, really great work and I'm really excited for some of the stuff you're doing, can't wait to see more, no matter what gets decided!

    On that note, as you guessed from the title, I'm hoping it might be possible to consolidate 0-N things from this, panopticon, and a theoretical new memory interval crate that I want to write, as well as some other things.

    This is a huge, huge topic, and I likely won't hit on a lot of the points, but just getting the ball rolling is good I think, if only to see if you're interested, where you're headed with things, etc.

    If you're not interested at all, that is totally fine of course :) Just wanted to see what you think

    Generic IL/Function crate

    So, for starters (and probably most controversially), reading through your source, particularly the il module, there is so much that I think could be refactored (along with panopticon) into a generic function/il rust crate.

    I say controversial because it will likely be hard/tedious, but i do think it would be (extremely) beneficial.

    It would also require probably the deepest amount of coordination, which could be hard.

    Nevertheless, I think some prime candidates are the il, and the function objects. If we could somehow make Function<IL>, where IL is the intermediate language used, this could have really really cool benefits.

    1. It would allow all of us to try out each other's IR
    2. it would allow us to switch to another IR for a different task (perhaps one is more compact, hence faster analysis)
    3. It would allow us to reuse the function logic and definitions and methods, consolidating bugs and developer effort into a single location (this and 2 are my prime motivation)

    Its hard for me to state how great this could be if we were able to swap out IL's at will. It also just seems right from an engineering perspective, similar to backends on a compiler.

    As it stands now in both codebases, I think this modification is almost trivially possible - except - the disassembler aspect.

    But this isn't necessarily bad news!

    For almost the exact reasons in 1-3, i think it would be really cool to allow function (or whatever it ends up being) to also be generic in the disassembler, allowing a more robust disassembler implementation (like capstone), or a home grown solution like panopticon, etc.

    Again the benefits here are experimentation, can try different assembler for different IR backend, etc.

    Doing this I think will require sketching out what a generic function + a generic disassembler would look like, and what would be the most flexible, and hence requires the most cooperation and assessment of current codebases dependencies and expectations etc., but long term I think it would be really cool, it would allow all our work to be pooled together and hence we'd all benefit.

    While I think this will be the hardest part to refactor, coordinate and get right, I think it will actually have the most benefit; of course, this is just my opinion though :)

    An interval tree crate, with a second crate geared towards binary memory intervals

    I don't think this is controversial at all, and I think it would be invaluable. I want something like this already for bingrep, panopticon needs it, and i'm sure falcon could use it too.

    Basically the idea is a:

    [x..y) -> Value

    Which is a datastructure that's created after the parser pass (or whenever you want, as long as you can send it a goblin binary), and which initially gets filled up with segment/section data; which ranges, what the name of the segment is, and perhaps what "kind of data" is there. We'd figure out what we want for a segment datatype, what information we'd need, etc. And of course, if its a central crate, when we need something new, we just extend it and everyone gets the benefit.

    Similarly, and this would be the tricky part and where I want feedback, downstream users could also extend the memory ranges with their own tagging data, like [0xbeef..0xdead) -> FunctionRange, etc.

    Even if some fancy runtime extendable type doesn't work out, even if we just agree on an enum in this crate which downstream clients use, I think this would be great code reuse and benefit everyone all around.

    dynamic linker/runtime loader

    Your loader looks really awesome!!! So i've been trying to get other persons to help create a relocator crate for a while, but no one is really interested in this stuff :laughing:

    Anyway, at some future data I think panopticon wants to have this. So I've wanted to turn https://github.com/m4b/dryad into a library for quite some time. Basically I like working on that project and I'll find any excuse; also all that code going to waste would be sad.

    So I'd like to propose potentially fusing falcon's runtime loader here with dryad, or vice versa, perhaps dryad becomes a lib, or i rip out parts of it via copy paste, whatever, and then that crate is refactored to be a library which downstream consumers like falcon and panopticon (and whoever really, who knows the applications!) can use it as their runtime linking and loading system.

    Initial issue is i'll have to put the asm usage in dryad and bare functions behind feature flags, as it requires nightly, and its not nice to force that on downstream clients (which would be sad, since it's a pure rust toolchain dynamic linker that way!)

    More things haven't even thought of

    Anyway, that's my suggestion for 3 different things I think are candidate usecases to refactor out into shared dependencies for great good. I'm sure there are many other opportunities as well.

    Let me know what you're thinking; as you can tell, I'm of the persuasion we should combine all of our powers and take over the universe :angel:

    Thanks for reading this far, I know, it was a lot :)

    /cc @flanfly

    opened by m4b 6
  • ARM Translator Implementation

    ARM Translator Implementation

    This project is very, very, relevant to my interests.

    I see from the blog post you are using Capstone as the disassembler, which implements the ARM architecture. I'd like to take a stab at an implementation of ARM - at least for ARMv6 in Thumb mode (my current target).

    I will use this issue to track an attempt at adding a module in falcon/lib/translator to map the Capstone ARM instruction API to Falcon IL.

    If you've thought of this already and have ideas of where it might go wrong please chime in! I'm just familiarizing myself with the codebase now.

    opened by voteblake 6
  • 0.5.0

    0.5.0

    @emmanuel099 This adds an option to translators, allowing all unhandled instructions to be lifted to intrinsics. This is also an API-breaking change, so 0.5.0 bump.

    Anything you want to throw in/change that may be API breaking?

    opened by endeav0r 5
  • Remove docker network dependency

    Remove docker network dependency

    Building off my previous pull request, and following the intent of making it easier for people to get started with Falcon, I've added a docker.sh script which will get the dependencies needed for Docker, install Docker, then build and run the container.

    Because Docker doesn't seem to have networking available to the containers by default, I modified the Dockerfile.stretch to no longer attempt to reach out to the internet. Instead, the llvm snapshot key is downloaded on the host (in docker.sh) and Dockerfile.stretch just copies it into the container.

    In testing this, I also found that the dependencies.sh was not compatible with Debian (which doesn't have sudo installed by default), so I also updated that script as well as setup.sh. The dependencies.sh script isn't run automatically, but the first thing I did when I went into the container was to run that so I would have everything I needed to run cargo build.

    opened by anon8675309 5
  • Add arithmetic shift right expression

    Add arithmetic shift right expression

    This change is motivated by the fact, that SMT solvers have builtin support for arithmetic shift right which is more efficient than the combination of shl, sub and shr.

    opened by emmanuel099 4
  • Added scripts to get set up outside of docker

    Added scripts to get set up outside of docker

    I wanted the ability to run the code from within my VM, so I made a script to accomplish that. I also made a Vagrantfile so it was easier for me to test my changes. The setup.sh and dependencies.sh scripts can be used with Vagrant VMs, manually created VMs, physical machines, Docker images, etc. The general goal of this PR is to make it easier for people to get a Falcon environment set up.

    opened by anon8675309 3
  • how to run?

    how to run?

    Wanted to try this out (nice work btw!)

    Do you have a simple cli program that takes a binary as input and outputs whatever?

    If not, I'd suggest adding a simple prototype/reference program in lib/bin/main.rs so could be easy to try out, or perhaps something in examples/

    Keep up the good work!

    opened by m4b 3
  • Updating dependencies and fixing linting

    Updating dependencies and fixing linting

    I've used "*" so that the project can be maintained against the latest libraries. I have tested it to be working on nightly build without any issues.

    Additionally, I have used cargo clippy and cargo fix to fix linting issues.

    opened by srikwit 2
  • translator/x86: Fix rol/ror semantics

    translator/x86: Fix rol/ror semantics

    • For both, rol and ror, the value assigned to ZF was more than one bit wide
    • Replaced ZF shifting in rol by truncation (we are interested in the LSB instead of MSB)
    bug 
    opened by emmanuel099 1
  • Bump regex from 1.5.4 to 1.5.6

    Bump regex from 1.5.4 to 1.5.6

    Bumps regex from 1.5.4 to 1.5.6.

    Changelog

    Sourced from regex's changelog.

    1.5.6 (2022-05-20)

    This release includes a few bug fixes, including a bug that produced incorrect matches when a non-greedy ? operator was used.

    1.5.5 (2022-03-08)

    This releases fixes a security bug in the regex compiler. This bug permits a vector for a denial-of-service attack in cases where the regex being compiled is untrusted. There are no known problems where the regex is itself trusted, including in cases of untrusted haystacks.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Eliminate error-chain

    Eliminate error-chain

    Falcon needs a good-and-proper Error enum. This will touch almost all of the codebase.

    This is a very heavy lift, and is being forever/continually pushed to the right.

    opened by endeav0r 0
Owner
Falcon Binary Analysis Framework
Binary Program Analysis in Rust
Falcon Binary Analysis Framework
Whole program static stack analysis

cargo-call-stack Static, whole program stack analysis Other examples: Embedded CoAP / IPv4 server (source) "Hello, world!" HEADS UP: This tool relies

Jorge Aparicio 457 Dec 22, 2022
Xori is an automation-ready disassembly and static analysis library for PE32, 32+ and shellcode

Xori - Custom disassembly framework Xori is an automation-ready disassembly and static analysis library that consumes shellcode or PE binaries and pro

ENDGAME 712 Nov 28, 2022
An impish, cross-platform binary parsing crate, written in Rust

libgoblin Documentation https://docs.rs/goblin/ changelog Usage Goblin requires rustc 1.40.0. Add to your Cargo.toml [dependencies] goblin = "0.4" Fea

null 892 Dec 22, 2022
The Swiss Army Knife for Binary (In)security

binsec Swiss Army Knife for Binary (In)security binsec is a minimal static analysis utility for detecting security capabilities in ELF/PE/Mach-O execu

Alan 15 Dec 16, 2022
Rust bindings for the unicorn CPU emulator

unicorn-rs THIS PACKAGE IS DEPRECATED AND NO LONGER MAINTAINED. Rust bindings are now included with unicorn and will be maintained there from now on.

null 129 Oct 10, 2022
rd is a record/replay debugger written in rust

rd The Record & Debug Tool The Record & Debug Tool (rd) is a Rust language port of the rr-debugger/rr debugger. With rd you can record Linux program e

Sidharth Kshatriya 948 Dec 27, 2022
☢ Guerrilla (or Monkey) Patching in Rust for (unsafe) fun and profit.

Guerrilla Guerrilla (or Monkey) Patching in Rust for (unsafe) fun and profit. Provides aribtrary monkey patching in Rust. Please do not use this crate

Ryan Leckey 97 Dec 16, 2022
Binary Analysis Framework in Rust

Welcome to Falcon Falcon is a formal binary analysis framework in Rust. Expression-based IL with strong influences from RREIL and Binary Ninja's LLIL.

Falcon Binary Analysis Framework 489 Dec 18, 2022
Binary Analysis Framework in Rust

Welcome to Falcon Falcon is a formal binary analysis framework in Rust. Expression-based IL with strong influences from RREIL and Binary Ninja's LLIL.

Falcon Binary Analysis Framework 489 Dec 18, 2022
Binary coverage tool without binary modification for Windows

Summary Mesos is a tool to gather binary code coverage on all user-land Windows targets without need for source or recompilation. It also provides an

null 384 Dec 30, 2022
Binary coverage tool without binary modification for Windows

Summary Mesos is a tool to gather binary code coverage on all user-land Windows targets without need for source or recompilation. It also provides an

null 381 Dec 22, 2022
Incremental Program Analysis Framework

IncA Overview IncA is a program analysis framework. It comes with a DSL for the definition of program analyses and the runtime system evaluates progra

Tamas Szabo 70 Nov 24, 2022
A "Hello, world!" template of a Rust binary crate for the ESP-IDF framework.

Rust on ESP-IDF "Hello, World" template A "Hello, world!" template of a Rust binary crate for the ESP-IDF framework. This is the crate you get when ru

Ivan Markov 140 Jan 4, 2023
Safe Rust bindings to the DynamoRIO dynamic binary instrumentation framework.

Introduction The dynamorio-rs crate provides safe Rust bindings to the DynamoRIO dynamic binary instrumentation framework, essentially allowing you to

S.J.R. van Schaik 17 Nov 21, 2022
botwork is a single-binary, generic and open-source automation framework written in Rust for acceptance testing & RPA

botwork botwork is a single-binary, generic and open-source automation framework written in Rust for acceptance testing, acceptance test driven develo

Nitimis 8 Apr 17, 2023
Yet Another Technical Analysis library [for Rust]

YATA Yet Another Technical Analysis library YaTa implements most common technical analysis methods and indicators. It also provides you an interface t

Dmitry 197 Dec 29, 2022
a super fast scientific calculator with dimensional analysis support written in Rust 🦀

larvae a super fast scientific calculator with dimensional analysis support written in Rust ?? ?? heavily inspired from insect Usage: Command mode: $

John McAvoy 2 Jan 10, 2022
A high performance python technical analysis library written in Rust and the Numpy C API.

Panther A efficient, high-performance python technical analysis library written in Rust using PyO3 and rust-numpy. Indicators ATR CMF SMA EMA RSI MACD

Greg 210 Dec 22, 2022
Linux anti-debugging and anti-analysis rust library

DebugOff Library Linux anti-analysis Rust library The goal of this library is to make both static and dynamic (debugging) analysis more difficult. The

null 65 Jan 7, 2023
Python+Rust implementation of the Probabilistic Principal Component Analysis model

Probabilistic Principal Component Analysis (PPCA) model This project implements a PPCA model implemented in Rust for Python using pyO3 and maturin. In

FindHotel 11 Dec 16, 2022