DWARF packaging utility, written in Rust, supporting GNU extension and DWARF 5 package formats.

Related tags

Miscellaneous elf dwarf dwp split-dwarf dwarf-package dwarf-object split-debuginfo
Overview

thorin

thorin is an DWARF packaging utility for creating DWARF packages (*.dwp files) out of input DWARF objects (*.dwo files; or *.o files with .dwo sections), supporting both the pre-standard GNU extension format for DWARF packages and the standardized format introduced in DWARF 5.

Usage

thorin can read input DWARF objects from executables or can package arbitrary input dwarf objects (including DWARF objects in archive files, such as Rust rlibs)!

thorin 0.1.0
merge dwarf objects into dwarf packages

USAGE:
    thorin [OPTIONS] [--] [inputs]...

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

OPTIONS:
    -e, --exec <executables>...    Specify path to executables to read list of dwarf objects from
    -o, --output <output>          Specify path to write the dwarf package to [default: -]

ARGS:
    <inputs>...    Specify path to input dwarf objects and packages

If the input objects are of DWARF version 5 or greater, then the output package will be in DWARF 5 format. For version 4 and below, the GNU Extension format will be used for the output package.

Contributing to thorin

If you want help or mentorship, reach out to us in a GitHub issue, or ask davidtwco on the Rust Zulip instance.

thorin should always build on stable rustc. To build thorin:

$ cargo build

We use rustfmt to automatically format and style all of our code. To install and use rustfmt:

$ rustup component add rustfmt
$ cargo fmt

Filing an Issue

Think you've found a bug? File an issue! To help us understand and reproduce the issue, provide us with:

  • The (preferably minimal) test case
  • Steps to reproduce the issue using the test case
  • The expected result of following those steps
  • The actual result of following those steps

Definitely file an issue if you see an unexpected panic originating from within thorin! thorin should never panic unless it is explicitly documented to panic in the specific circumstances provided.


Name

thorin is named after Thorin Oakenshield from The Hobbit, as Thorin is a dwarf who leads other dwarves. thorin uses the gimli library (named after a dwarf from Lord of the Rings) to read DWARF format debug information, the name of which is a medieval fantasy complement to ELF, the file format for executables and object files.
You could also call this project rust-dwp, if you'd prefer that.

Author and acknowledgements

thorin is authored by David Wood of Huawei Technologies Research & Development (UK) Ltd.
In addition, thanks to the authors of object and gimli, on which this utility depends heavily; and to Philip Craig for advice and reviews during initial implementation of thorin.

License

Licensed under either of Apache License, Version 2.0 or MIT license at your option.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this crate by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Code of conduct

When contributing or interacting with this project, we ask abide the Rust Code of Conduct and ask that you do too.
Comments
  • release version 0.2.0

    release version 0.2.0

    Includes all changes since a59b8ee82:

    7e76f9d Use hashbrown cfc9c7d Reduce the number of reallocations for PackageStringTable::data 0ecb11d Avoid extra HashMap in PackageStringTable 12660a7 Change assert to debug_assert in PackageStringTable cf41f62 Handle entries with differing columns cf21bfb Update object dependency 025395b update repository links and readme

    opened by davidtwco 5
  • Is the relocation handling unnecessary?

    Is the relocation handling unnecessary?

    My understanding is that DWO files are generated such that relocations aren't needed, and they're definitely not needed for executable files.

    I tried changing DwpReader to be just EndianSlice, and the resulting DWP for my test case was unchanged.

    opened by philipc 4
  • Performance improvements

    Performance improvements

    This approximately doubles the speed for a large test case (thorin -e llvm-project/build/bin/opt -o opt.dwp). (Interestingly, both llvm-dwp and my Ubuntu's dwp fail to handle this test case.)

    Most of the time is still spend in PackageStringTable::remap_str_offsets_section. I think the main way we could further improve that is by using the strings in PackageStringTable::data as the hash keys instead of allocating a Vec for each one, but I don't know if that is possible with existing HashMap APIs. Maybe custom allocators will allow this.

    I also tried radix_trie, but it is much slower.

    opened by philipc 1
  • memmap2 on wasm

    memmap2 on wasm

    Wasm doesn't have mmap support. Rustc has a wrapper that uses Vec<u8> instead of memmap2::Mmap on wasm: https://github.com/rust-lang/rust/blob/2a9e0831d6603d87220cedd1b1293e2eb82ef55c/compiler/rustc_data_structures/src/memmap.rs#L7

    Originally posted by @bjorn3 in https://github.com/davidtwco/thorin/pull/5#discussion_r762118577

    opened by davidtwco 1
  • next_pow2

    next_pow2

    https://github.com/davidtwco/rust-dwp/blob/874db16e69573d5872f34b80372a5e71381c2601/src/main.rs#L1386 This can probably be replaced with u32::next_power_of_two.

    opened by philipc 1
  • Does number_of_columns allow for differences in entries?

    Does number_of_columns allow for differences in entries?

    https://github.com/davidtwco/rust-dwp/blob/874db16e69573d5872f34b80372a5e71381c2601/src/main.rs#L1034-L1036

    The assert here checks that all entries have the same number of columns, but what if they are different while still having the same number?

    opened by philipc 1
  • Upgrade hashbrown, indexmap, and object

    Upgrade hashbrown, indexmap, and object

    I'm working on a PR to update rust-lang/rust to hashbrown 0.12 everywhere, and thorin is the last holdout. We only need new versions of indexmap and object here, as well as upgrading the direct hashbrown dependency.

    opened by cuviper 0
  • Update `object` dependency to 0.28

    Update `object` dependency to 0.28

    This PR simply updates the object dependency from 0.27 to 0.28.

    rust_codegen_ssa now depends on 0.28 since https://github.com/rust-lang/rust/pull/90001. Whenever thorin is updated in rustc in the future, removing the use of 0.27 would deduplicate them.

    opened by lqd 0
  • release version 0.1.1 and add per-crate readme

    release version 0.1.1 and add per-crate readme

    Instead of using the same README for both the library and binary crates, use a separate specific README for each. Upgrade the version to 0.1.1, including nagisa's patches (#10).

    opened by davidtwco 0
  • Review…

    Review…

    The code looks good, at least given my understanding of DWARF which is definitely not super extensive.

    I ended up with just a couple style nits and whatnot, best reviewed commit-by-commit.

    opened by nagisa 0
  • Testing, repeated sections and diagnostics

    Testing, repeated sections and diagnostics

    Fixes #1.

    • Adds test suite, based on llvm-dwp's test suite using lit
    • Various refactorings and tidying up
    • Support for repeated sections (e.g. multiple .debug_types.dwo)
    • Confirm that each index entry has same columns
    • Record loclist/rnglist contributions in type unit indices (tested by llvm-dwp but not in the DWARF 5 standard)
    • Add incompatible package version diagnostic
    • Add duplicate split compilation unit diagnostic
    • Probably some other things I've forgotten, check the commits.
    opened by davidtwco 0
Owner
David Wood
A software engineer from Scotland, with an interest in systems software and open source software; member of the Rust compiler team.
David Wood
GitHub
Gnu-copypasta-maker - You'd just wanna interject for a moment!

GNU Copypasta Maker To compile, use make. To install, use sudo make install or if you are root make install To uninstall, use sudo make uninstall or i

Ahmet Efe AKYAZI 3 Jan 30, 2022
little brother of gnu-copypasta-maker To compile, use make.

UWU Maker little brother of gnu-copypasta-maker To compile, use make. To install, use sudo make install or if you are root make install To uninstall,

Ahmet Efe AKYAZI 1 Jan 12, 2022
A shiny new package manager written in rust explicitly for gemlock/linux and it's distributions

Gem A shiny new package manager written in rust explicitly for gemlock/linux and it's distributions. List of content How to setup Systems Ubuntu Arch

Gemlock 1 Feb 22, 2022
A fast package manager for NodeJS written in Rust.

click A fast package manager for NodeJS written in Rust. How fast? Benchmark of bun vs click clean install: Based on benchmarks done with hyperfine, c

Sam 52 Oct 10, 2023
Simple MHV6 extension used to download custom songs efficiently and effectively.

nong-downloader-extension A simple MegaHack v6 extension that helps you download NONG songs directly to GD. Compiling. Why would you want to compile??

Alpha 6 Nov 1, 2022
Rust package for constructing and analyzing phylogenies.

Rust-Phylogeny, a Rust library for phylogenetic trees Currently this library provides three algorithms: UPGMA and Neighbor-Joining for constructing a

Ragnar Groot Koerkamp 1 Nov 18, 2021
Novus - A blazingly fast and efficient package manager for windows.

Novus - A blazingly fast and efficient package manager for windows. Why Novus Swift Unlike any other package manager, Novus uses multithreaded downloads

Novus 197 Dec 18, 2022
A rust-based extension module for Halo 3: MCC PC

halo3-rs This project is a rust-based extension module for Halo 3 in the Master Chief Collection on PC. The DLL wrapper contains a collection of patch

null 2 Sep 19, 2021
cargo extension for flashing embedded rust programs via dfu based on jacobrosenthals cargo-hf2

cargo-dfu This crate provides a cargo subcommand to flash ELF binaries via dfu Most STM chips will probably work with this, although you might need to

Roman Kretschmer 0 Feb 6, 2022
SPWN extension manager.

XTND SPWN extension manager. Important Disclaimer As this will allow any module to run as builtins, with little to no limitations, use of XTND may be

Deltara 1 Dec 16, 2021
Cargo - The Rust package manager

Cargo downloads your Rust project’s dependencies and compiles your project.

The Rust Programming Language 9.5k Jan 4, 2023
💡 Use the right package manager by rust

n ?? Use the right package manager by rust ?? Inspired by ni Why ni is nice , but ni is based on Node. it is difficult to collaborate well with node v

SHEIN 3 Jul 24, 2023
basic multiple package manager

baka basic multiple package manager Docs Env baka_root_setting Windows: %USERPROFILE%/.baka/config Linux, Mac: $HOME/.baka/config baka_plugins (Just u

null 8 Dec 29, 2021
Wally is a modern package manager for Roblox projects inspired by Cargo

Wally is a package manager for Roblox inspired by Cargo (Rust) and npm (JavaScript). It brings the familiar, community-oriented world of sharing code from other communities into the Roblox ecosystem.

Uplift Games 194 Jan 3, 2023
Plow - The ontology package manager

Plow - Ontology package manager Plow is package management solution for OWL ontologies, with support for specifying dependencies between packages via

Field 33 12 Dec 21, 2022
🗃 Decoder and utility for the Flipnote Studios .ppm animation format.

?? para What? A decoder and utility for the Flipnote Studios .ppm animation format. Why this implementation over the original? This implementation is

Fuwn 11 Dec 12, 2022
Utility for recursively unzipping tuples, Options of tuples and Results of tuples.

Zipped Utility for recursively unzipping tuples, Options of tuples and Results of tuples. Install cargo add zipped Usage This crate is quiet straightf

Glacyr B.V. 2 Dec 4, 2022
Utility to quickly setup Starcraft Broodwar matches between 2 or more bots

BWAIShotgun Utility to quickly setup Starcraft Broodwar matches between 2 or more bots Be aware that all bots will be executed directly, without any l

Dennis Waldherr 5 Nov 25, 2022
This is a Discord bot written in Rust to translate to and from the Bottom Encoding Standard using bottom-rs and Serenity.

bottom-bot This is a Discord bot written in Rust to translate to and from the Bottom Encoding Standard using bottom-rs and Serenity. Ever had this pro

Bottom Software Foundation 11 Dec 10, 2022