binocle is a graphical tool to visualize binary data

[2021-10-03T21:36:28Z ERROR wgpu::backend::direct] Handling wgpu errors as fatal by default
thread 'main' panicked at 'wgpu error: Validation Error

Caused by:
    In a RenderPass
      note: encoder = `pixels_command_encoder`
    In a set_scissor_rect command
    Invalid ScissorRect parameters

', C:\Users\<user>\.cargo\registry\src\github.com-1ecc6299db9ec823\wgpu-0.10.2\src\backend\direct.rs:2160:5
stack backtrace:
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Anything I can help with to diagnose the problem?

Thanks for the repo!

I ran cargo flamegraph on the code when using the Entropy visualization. This showed one very obvious area of improvement. csscolor::parse::parse was parsing hex numbers as part of creating the colograde::magma() color gradient.

I think the easiest solution is just to make colorgrade::magma() a static and re-use it. What do you think?

Aside: Is there any work in making the UI into a separate thread from the calculations? The mouse-over is just sluggish when I change the color scheme.

Thanks for reviewing!

I've used memmap2 successfully across different platforms before and thought it might be a good solution for loading large files in binocle.

In this MR I've just replaced the file loading with an Mmap from this library. Once the Mmap is made, it can be treated as a slice, so no changes outside of Buffer needed to be made.

Do note that the call to Mmap is unsafe. As the mmap library documents state, this is to indicate that if you were to edit the file in this, or another process, while it is mapped, this would result in undefined behavior.

Edit: I've also tested that this works on macOS, but do not have a windows box handy.

❯ cargo install-update --all
    Updating registry 'https://github.com/rust-lang/crates.io-index'

Package                Installed  Latest   Needs update
binocle                v0.1.2     v0.2.0   Yes
[...]

Updating binocle
    Updating crates.io index
  Installing binocle v0.2.0
  [...]
   Compiling binocle v0.2.0
error[E0432]: unresolved imports `clap::crate_description`, `clap::crate_version`
 --> /home/mark/.cargo/registry/src/github.com-1ecc6299db9ec823/binocle-0.2.0/src/options.rs:1:12
  |
1 | use clap::{crate_description, crate_version, Parser};
  |            ^^^^^^^^^^^^^^^^^  ^^^^^^^^^^^^^ no `crate_version` in the root
  |            |
  |            no `crate_description` in the root

error: cannot find derive macro `Parser` in this scope
 --> /home/mark/.cargo/registry/src/github.com-1ecc6299db9ec823/binocle-0.2.0/src/options.rs:4:10
  |
4 | #[derive(Parser)]
  |          ^^^^^^
  |
note: `Parser` is imported here, but it is only a trait, without a derive macro
 --> /home/mark/.cargo/registry/src/github.com-1ecc6299db9ec823/binocle-0.2.0/src/options.rs:1:46
  |
1 | use clap::{crate_description, crate_version, Parser};
  |                                              ^^^^^^

error: cannot find attribute `clap` in this scope
 --> /home/mark/.cargo/registry/src/github.com-1ecc6299db9ec823/binocle-0.2.0/src/options.rs:5:3
  |
5 | #[clap(version = crate_version!(), about = crate_description!())]
  |   ^^^^
  |
  = note: `clap` is in scope, but it is a crate, not an attribute

error: cannot find attribute `clap` in this scope
 --> /home/mark/.cargo/registry/src/github.com-1ecc6299db9ec823/binocle-0.2.0/src/options.rs:8:7
  |
8 |     #[clap(
  |       ^^^^
  |
  = note: `clap` is in scope, but it is a crate, not an attribute

error[E0599]: no function or associated item named `parse` found for struct `CliOptions` in the current scope
  --> /home/mark/.cargo/registry/src/github.com-1ecc6299db9ec823/binocle-0.2.0/src/main.rs:15:31
   |
15 |     let options = CliOptions::parse();
   |                               ^^^^^ function or associated item not found in `CliOptions`
   |
  ::: /home/mark/.cargo/registry/src/github.com-1ecc6299db9ec823/binocle-0.2.0/src/options.rs:6:1
   |
6  | pub struct CliOptions {
   | --------------------- function or associated item `parse` not found for this
   |
   = help: items from traits can only be used if the trait is implemented and in scope
   = note: the following traits define an item `parse`, perhaps you need to implement one of them:
           candidate #1: `Parser`
           candidate #2: `nom::internal::Parser`
           candidate #3: `nom::sequence::Tuple`
           candidate #4: `ttf_parser::parser::FromData`
           candidate #5: `ttf_parser::parser::FromData`
           candidate #6: `ttf_parser::parser::FromSlice`
           candidate #7: `ttf_parser::ggg::lookup::LookupSubtable`
           candidate #8: `ttf_parser::ggg::layout_table::RecordListItem`

I was really exited to test it but I get a segfault every times...

Tested the debian and tarball build on Linux Mint 20.2 ( / Unbuntu bionic) x86_64 (Linux 5.4.0-88-generic) inside VirtualBox. I also attempted a custom build with cargo 1.56.0 with no better results.

Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `binocle xq'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007fe980281b40 in ?? () from /usr/lib/x86_64-linux-gnu/libvulkan_lvp.so
[Current thread is 1 (Thread 0x7fe94d7fa700 (LWP 315115))]

$ dpkg -l | grep vulkan
ii  libvulkan1:amd64           1.2.131.2-1                amd64  Vulkan loader library
ii  mesa-vulkan-drivers:amd64  21.0.3-0ubuntu0.3~20.04.3  amd64  Mesa Vulkan graphics drivers

Amazing framework. Thanks a lot to the the puffin developers. Thanks to @emilk for helping out

How to use it is described in the Readme:

cargo run --example create_test_file
cargo run --features trace -- binocle_test_file

In another terminal run:

cargo install puffin_viewer
puffin_viewer --url 127.0.0.1:8585

This fixes the perf bug with a laggy UI.

However, I suppose that the egui framework has a better method to achieve the same result.

In my approach, I store a hash of the settings and draw into the pixel buffer only when the settings have changed.

The effect is a noticeable improvement in the UI. 🙂

NOTES:

• I had to make Settings::value_range use u32 instead of f32, as f32 isn't hashable.

Desired assignee: @sharkdp

Hello, thank you for your work! But I have some problems when I run it on my OS

OS Spec: MacOS Big Sur (11.5.2 (20G95) 2,3 GHz 8-Core Intel Core i9 AMD Radeon Pro 5500M 4 GB

Rust info: rustup 1.24.3 (ce5817a94 2021-05-31)

Toolchain stable-x86_64-apple-darwin (default) rustc 1.55.0 (c8dfcfe04 2021-09-06)

I installed thry cargo install binocle

I feel free to test any updates and changes in my OS

You can see backtrace below

env RUST_BACKTRACE=full binocle test.jsc
[2021-10-07T07:40:30Z ERROR wgpu::backend::direct] Handling wgpu errors as fatal by default
thread 'main' panicked at 'wgpu error: Validation Error

Caused by:
    In a RenderPass
      note: encoder = `pixels_command_encoder`
    In a set_scissor_rect command
    Invalid ScissorRect parameters

', /Users/jilizart/.cargo/registry/src/github.com-1ecc6299db9ec823/wgpu-0.10.2/src/backend/direct.rs:2160:5
stack backtrace:
   0:        0x10298cbd4 - std::backtrace_rs::backtrace::libunwind::trace::hd9b9204470182ffc
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/../../backtrace/src/backtrace/libunwind.rs:90:5
   1:        0x10298cbd4 - std::backtrace_rs::backtrace::trace_unsynchronized::hdc48b9180be04761
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:        0x10298cbd4 - std::sys_common::backtrace::_print_fmt::h8824de041faf4112
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/sys_common/backtrace.rs:67:5
   3:        0x10298cbd4 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h8d168763f08a78b9
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/sys_common/backtrace.rs:46:22
   4:        0x1029aa3fc - core::fmt::write::h0fa46d4891ed63f1
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/core/src/fmt/mod.rs:1115:17
   5:        0x10298a40a - std::io::Write::write_fmt::hacbf9e10e17149ce
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/io/mod.rs:1665:15
   6:        0x10298e80f - std::sys_common::backtrace::_print::h26213154a400cfbe
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/sys_common/backtrace.rs:49:5
   7:        0x10298e80f - std::sys_common::backtrace::print::h0f4ab60f53d1f93b
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/sys_common/backtrace.rs:36:9
   8:        0x10298e80f - std::panicking::default_hook::{{closure}}::hee7cf38dedfa76fc
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/panicking.rs:208:50
   9:        0x10298e30d - std::panicking::default_hook::hd12252412832a45f
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/panicking.rs:225:9
  10:        0x10298ef10 - std::panicking::rust_panic_with_hook::h521fd09e07057893
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/panicking.rs:622:17
  11:        0x10298e9b5 - std::panicking::begin_panic_handler::{{closure}}::h3588a5a9cca9e8e5
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/panicking.rs:519:13
  12:        0x10298d078 - std::sys_common::backtrace::__rust_end_short_backtrace::h86522c151a195ff9
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/sys_common/backtrace.rs:141:18
  13:        0x10298e91a - rust_begin_unwind
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/panicking.rs:515:5
  14:        0x1029cef7b - std::panicking::begin_panic_fmt::hf4ebaa1b90291b0b
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/panicking.rs:457:5
  15:        0x1028a11fa - wgpu::backend::direct::default_error_handler::h6d7a2ce79deec755
  16:        0x102899ae0 - core::ops::function::Fn::call::hf7dbc75eeffb9340
  17:        0x1028a0b6d - <wgpu::backend::direct::Context as wgpu::Context>::command_encoder_end_render_pass::hfd5ba570bf76c3b6
  18:        0x10279292a - egui_wgpu_backend::RenderPass::execute::hcff90a062f822137
  19:        0x1026bc63c - binocle::gui::Gui::render::ha9e59e84680cb6ff
  20:        0x1026b6c59 - pixels::Pixels::render_with::hdb22eb48c60820be
  21:        0x1026b3d4a - binocle::event_loop::run::{{closure}}::hadc2a99b6c85b4c9
  22:        0x1026b4cd7 - <winit::platform_impl::platform::app_state::EventLoopHandler<T> as winit::platform_impl::platform::app_state::EventHandler>::handle_nonuser_event::he369832bac4dc772
  23:        0x10282e8d1 - winit::platform_impl::platform::app_state::Handler::handle_nonuser_event::h4e84ce779f567612
  24:        0x1028309a5 - winit::platform_impl::platform::app_state::AppState::cleared::h6028c81bcc2db529
  25:        0x1028331f7 - winit::platform_impl::platform::observer::control_flow_end_handler::h7a06aff7e3095726
  26:     0x7fff20454d01 - <unknown>
  27:     0x7fff20454b95 - <unknown>
  28:     0x7fff20454166 - <unknown>
  29:     0x7fff2045361c - <unknown>
  30:     0x7fff28699a83 - <unknown>
  31:     0x7fff286997e5 - <unknown>
  32:     0x7fff28699583 - <unknown>
  33:     0x7fff22c5b172 - <unknown>
  34:     0x7fff22c59945 - <unknown>
  35:     0x7fff22c4bc69 - <unknown>
  36:        0x1026b033a - winit::platform_impl::platform::event_loop::EventLoop<T>::run::h5bc5452d0bb554b0
  37:        0x1026e9283 - winit::event_loop::EventLoop<T>::run::h5e8cb8089d1588c7
  38:        0x1026eb05c - binocle::event_loop::run::h303925b133e506be
  39:        0x1026ed229 - binocle::main::h67e417241ee8838d
  40:        0x1026bc88a - std::sys_common::backtrace::__rust_begin_short_backtrace::hb4e5ebb094580cbd
  41:        0x1026e66a1 - std::rt::lang_start::{{closure}}::hd64f067cbe88cad4
  42:        0x10298f439 - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::h5adc1669b25f044b
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/core/src/ops/function.rs:259:13
  43:        0x10298f439 - std::panicking::try::do_call::h405f88a3fec16587
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/panicking.rs:401:40
  44:        0x10298f439 - std::panicking::try::h86d9af92fb3c6022
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/panicking.rs:365:19
  45:        0x10298f439 - std::panic::catch_unwind::h67150880d2b87901
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/panic.rs:434:14
  46:        0x10298f439 - std::rt::lang_start_internal::{{closure}}::h548e43d03023c928
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/rt.rs:45:48
  47:        0x10298f439 - std::panicking::try::do_call::h1383affbfec23244
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/panicking.rs:401:40
  48:        0x10298f439 - std::panicking::try::hf51149bda74e8587
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/panicking.rs:365:19
  49:        0x10298f439 - std::panic::catch_unwind::h8280208a4f5e7cf4
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/panic.rs:434:14
  50:        0x10298f439 - std::rt::lang_start_internal::h5f9b030f0a63c040
                               at /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/std/src/rt.rs:45:20
  51:        0x1026ed349 - _main
-[_MTLCommandEncoder dealloc]:131: failed assertion `Command encoder released without endEncoding'
fish: 'env RUST_BACKTRACE=full binocle…' terminated by signal SIGABRT (Abort)

Hello!

I started maintaining binocle in the Arch Linux community repository: https://archlinux.org/packages/community/x86_64/binocle/

This PR updates README.md for mentioning the installation instructions of this package on Arch Linux.

I installed with cargo install binocle

Then I tried running on an MP4 file (seems I get the same error for any file):

binocle circles1.mp4

And this is what I got:

[2021-10-04T16:49:23Z ERROR wgpu::backend::direct] Handling wgpu errors as fatal by default
thread 'main' panicked at 'wgpu error: Validation Error

Caused by:
    In a RenderPass
      note: encoder = `pixels_command_encoder`
    In a set_scissor_rect command
    Invalid ScissorRect parameters

', /Users/kamil/.cargo/registry/src/github.com-1ecc6299db9ec823/wgpu-0.10.2/src/backend/direct.rs:2160:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
-[_MTLCommandEncoder dealloc]:70: failed assertion `Command encoder released without endEncoding'
Abort trap: 6

Here is some information about my system:

rustup show
Default host: x86_64-apple-darwin
rustup home:  /Users/kamil/.rustup

stable-x86_64-apple-darwin (default)
rustc 1.55.0 (c8dfcfe04 2021-09-06)

I'm diagnosing a problem in a telescope control system with several processes that communicate using several shared memory segments backed by mmap'd files. I can reload the file by restarting binocle. It would be really nice to have binocle refresh the view on command or at some update rate.

The slider doesn't really work so well fine grained. With SHIFT it still makes +5 steps. It would be more useful to make +1 steps.

It would also be nice to use scrollwheel to do -1 and +1

Could the program do a simple (but possible computational expensive) computation to look for repeating patterns?

With as input parameters:

• offset (byte 0 to filesize)
• stride (from 1 to maybe half the filesize, or 25%)
• pattern length (span of bytes)

Result of search could be: Found pattern with length of 3 bytes, repeated 100 times with stride 20

Then width of can set equal to stride and the same kind color of pixels should show up as a vertical line.

In case of many patterns found they can be sorted by something like: pattern length * length

First I'd like to say, thanks for making this tool! It's a lot more convenient than writing a bespoke Python script every time I want to visualize a new binary in a different way.

The only complaint I have about it right now is that it assumes the user has a fairly large monitor. My laptop's built-in screen is 1366x768, so not only does the window default to a size larger than my screen, I can't shrink it to fit my screen. This means that the lower part of the controls get cut off, so I can't access the endianness setting or any other settings below it without expanding the controls really far to the left. In addition, when viewing the controls, the left size of the binary view gets cut off, meaning I have to drag the window to the right in order to try and see that part of the binary. And finally, there's some weird part in the middle of the binary that appears "stretched" for some reason, with non-square pixels, though I'm not sure if this is related--it might be a Vulkan driver issue since Mesa's support for Vulkan on Bay Trail is incomplete and that's what I'm running this on.

1366x768 is a popular resolution for inexpensive laptops, some users set their screens to lower resolutions for accessibility reasons, and others may simply want to run two instances of binocle side-by-side, so it'd be nice if binocle supported smaller window sizes.

Here are some screenshots so you can see what I mean: