Tool to draw low-resolution graphs in terminal

Overview

lowcharts

Tool to draw low-resolution graphs in terminal.

Rust codecov dependency status Release

lowcharts is meant to be used in those scenarios where we have numerical data in text files that we want to display in the terminal to do a basic analysis.

An example would be the logs of a service (webserver, database, proxy, container orchestration, etc.) where times (or sizes) of requests are logged. In an ideal world you would have those logs accessible via a kibana (or similar) or those metrics exposed to a prometheus (or similar) and graphed in a grafana dashboard (or similar). But sometimes we need to cope with non ideal worlds, and troubleshoot a service with nothing more of what we can muster in a shell terminal.

Usage

Type lowcharts --help, or lowcharts PLOT-TYPE --help for a complete list of options.

Currently six basic types of plots are supported:

Bar chart for matches in the input

Since grep -c does not aggregate counts per pattern, this is maybe my most frequent use case.

This chart is generated using lowcharts matches database.log SELECT UPDATE DELETE INSERT DROP:

Simple bar chart with lowcharts

Histogram for numerical inputs

This chart is generated using python3 -c 'import random; [print(random.normalvariate(5, 5)) for _ in range(100000)]' | lowcharts hist:

Sample histogram with lowcharts

This was inspired by data-hacks. However, for some big log files I found that project was slower of what I would like, and I found that a rust-compiled binary was better suited to my needs.

Options for specifying ranges, chart sizes and input files are supported:

lowcharts hist --max 0.5 --intervals 10 --width 50 data.txt
Samples = 50090; Min = 0.000; Max = 0.499
Average = 0.181; Variance = 0.023; STD = 0.154
each ∎ represents a count of 484
[0.000 .. 0.050] [14545] ∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
[0.050 .. 0.100] [ 6111] ∎∎∎∎∎∎∎∎∎∎∎∎
[0.100 .. 0.150] [ 4911] ∎∎∎∎∎∎∎∎∎∎
[0.150 .. 0.200] [ 4003] ∎∎∎∎∎∎∎∎
[0.200 .. 0.250] [ 3745] ∎∎∎∎∎∎∎
[0.250 .. 0.300] [ 3526] ∎∎∎∎∎∎∎
[0.300 .. 0.350] [ 3424] ∎∎∎∎∎∎∎
[0.350 .. 0.400] [ 3332] ∎∎∎∎∎∎
[0.400 .. 0.450] [ 3215] ∎∎∎∎∎∎
[0.450 .. 0.500] [ 3278] ∎∎∎∎∎∎

Above examples assume input files with a number per line. Options for figuring out where to look in the input file for values are supported by regex option. This example logs the time spent by nginx for all of 200K http responses ()

$ cat nginx*.log | lowcharts hist --regex ' 200 \d+ ([0-9.]+)' --intervals 10
Samples = 25080; Min = 0.004; Max = 0.049
Average = 0.008; Variance = 0.000; STD = 0.006
each ∎ represents a count of 228
[0.004 .. 0.009] [20569] ∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
[0.009 .. 0.013] [ 1329] ∎∎∎∎∎
[0.013 .. 0.018] [  807] ∎∎∎
[0.018 .. 0.022] [ 1412] ∎∎∎∎∎∎
[0.022 .. 0.027] [  363] ∎
[0.027 .. 0.031] [   27]
[0.031 .. 0.036] [  128]
[0.036 .. 0.040] [   22]
[0.040 .. 0.044] [  240] ∎
[0.044 .. 0.049] [  183]

Time Histogram

This chart is generated using strace -tt ls -lR * 2>&1 | lowcharts timehist --intervals 10:

Sample plot with lowcharts

Things like lowcharts timehist --regex ' 404 ' nginx.log should work in a similar way, and would give you a glimpse of when and how many 404s are being triggered in your server.

The idea is to depict the frequency of logs that match a regex (by default any log that is read by the tool). The sub-command can autodetect the most common (in my personal and biased experience) datetime/timestamp formats: rfc 3339, rfc 2822, python %(asctime)s, golang default log format, nginx, rabbitmq, strace -t (or -tt, or -ttt),ltrace,... as long as the timestamp is present in the first line in the log and the format is consistent in all the lines that contain timestamp. It is ok to have lines with no timestamp. The consistency is required because of performance reasons: the 1st log line is the only one that triggers the heuristics needed to create an specialized datetime parser on the fly.

However, if you have a format that lowcharts cannot autodetected, you can specify it via command line flag. For instance, --format '%d-%b-%Y::%H:%M:%S'. Note that, as of today, you need to leave out the timezone part of the format string (the autodetection works fine with timezones).

Split Time Histogram

This adds up the time histogram and bar chart in a single visualization.

This chart is generated using strace -tt ls -lR 2>&1 | lowcharts split-timehist open mmap close read write --intervals 10:

Sample plot with lowcharts

This graph depicts the relative frequency of search terms in time.

Common terms histogram

Useful for plotting most common terms in input lines.

This sample chart is generated using strace ls -l 2>&1 | lowcharts common-terms --lines 8 -R '(.*?)\(':

Sample plot with lowcharts

The graph depicts the 8 syscalls most used by ls -l command, along with its number of uses and sorted. In general, using lowcharts common-terms is a handy substitute to commands of the form awk ... | sort | uniq -c | sort -rn | head.

X-Y Plot

This chart is generated using cat ram-usage | lowcharts plot --height 20 --width 50:

Sample plot with lowcharts

Note that x axis is not labelled. The tool splits the input data by chunks of a fixed size and then the chart display the averages of those chunks. In other words: grouping data by time is not (yet?) supported; you can see the evolution of a metric over time, but not the speed of that evolution.

There is regex support for this type of plots.

Installing

Via release

Go over https://github.com/juan-leon/lowcharts/releases/ and download the binary you want. Decompress the file and copy the binary to your path.

Via local compilation

$ git clone https://github.com/juan-leon/lowcharts
$ cd lowcharts
$ cargo install --path .

Contributing

Feedback, ideas and pull requests are welcomed.

Issues
  • Create new plot type that combines matches and timehist plots

    Create new plot type that combines matches and timehist plots

    Every match should have its own "color", so that visually is easy to understand patterns.

    Plot would be useful to know the distribution of several kind of matches over time. The hard part (autodetecting time stamps in an efficient way) is already done.

    enhancement 
    opened by juan-leon 1
Releases(v0.4.2)
Owner
juanleon lahoz
Emacs user and systems developer. Experienced with python, golang, rust, C, C++, Java and others.
juanleon lahoz
A cross-platform graphical process/system monitor with a customizable interface and a multitude of features

A cross-platform graphical process/system monitor with a customizable interface and a multitude of features. Supports Linux, macOS, and Windows. Inspired by both gtop and gotop.

Clement Tsang 2.8k Sep 12, 2021
Tool to draw low-resolution graphs in terminal

lowcharts Tool to draw low-resolution graphs in terminal. lowcharts is meant to be used in those scenarios where we have numerical data in text files

juanleon lahoz 24 Sep 4, 2021
Manage lightweight VMs created from OCI images

krunvm krunvm is a CLI-based utility for managing lightweight VMs created from OCI images, using libkrun and buildah. Features Minimal footprint Fast

Containers 379 Sep 15, 2021
A library that creates a terminal-like window with feature-packed drawing of text and easy input handling. MIRROR.

BearLibTerminal provides a pseudoterminal window with a grid of character cells and a simple yet powerful API for flexible textual output and uncompli

Tommy Ettinger 31 Sep 5, 2021
Alacritty - A fast, cross-platform, OpenGL terminal emulator

Alacritty is a modern terminal emulator that comes with sensible defaults, but allows for extensive configuration. By integrating with other applications, rather than reimplementing their functionality, it manages to provide a flexible set of features with high performance. The supported platforms currently consist of BSD, Linux, macOS and Windows.

Alacritty 34.7k Sep 14, 2021
A cat(1) clone with syntax highlighting and Git integration.

A cat(1) clone with syntax highlighting and Git integration. Key Features • How To Use • Installation • Customization • Project goals, alternatives [中

David Peter 29.4k Sep 15, 2021
Cross platform terminal library rust

Cross-platform Terminal Manipulation Library Crossterm is a pure-rust, terminal manipulation library that makes it possible to write cross-platform te

crossterm-rs 1.3k Sep 17, 2021
The module graph logic for Deno CLI

deno_graph The module graph/dependency logic for the Deno CLI. This repository is a Rust crate which provides the foundational code to be able to buil

Deno Land 21 Aug 26, 2021
Build terminal user interfaces and dashboards using Rust

tui-rs tui-rs is a Rust library to build rich terminal user interfaces and dashboards. It is heavily inspired by the Javascript library blessed-contri

Florian Dehau 5.6k Sep 11, 2021
A Rust library for building interactive prompts

inquire is a library for building interactive prompts on terminals. Demo Source Usage Put this line in your Cargo.toml, under [dependencies]. inquire

Mikael Mello 62 Sep 17, 2021
Build terminal dashboards using ascii/ansi art and javascript

blessed-contrib Build dashboards (or any other application) using ascii/ansi art and javascript. Friendly to terminals, ssh and developers.

Yaron Naveh 14.5k Sep 10, 2021
A curated list of replacements for existing software written in Rust

Awesome Alternatives in Rust A curated list of replacements for existing software written in Rust. If you want to contribute, please read CONTRIBUTING

Takayuki Maeda 1.9k Sep 9, 2021
Cross-platform Rust library for coloring and formatting terminal output

Coloring terminal output Documentation term-painter is a cross-platform (i.e. also non-ANSI terminals) Rust library for coloring and formatting termin

Lukas Kalbertodt 68 Jul 3, 2021
yayagram - Play nonograms/picross in your terminal

yayagram is a puzzle game in which you fill out a grid with cells based on logic and number clues.

r00ster 29 Aug 21, 2021
A more intuitive version of du in rust

A more intuitive version of du in rust

andy.boot 3k Sep 20, 2021
miniserve - a CLI tool to serve files and dirs over HTTP

?? For when you really just want to serve some files over HTTP right now!

Sven-Hendrik Haase 2.6k Sep 15, 2021
A template for bootstrapping a Rust TUI application with tui-rs & crossterm

rust-tui-template A template for bootstrapping a Rust TUI application with tui-rs & crossterm. tui-rs The library is based on the principle of immedia

Orhun Parmaksız 25 Sep 10, 2021
A Text User Interface library for the Rust programming language

Cursive Cursive is a TUI (Text User Interface) library for rust. It uses ncurses by default, but other backends are available. It allows you to build

Alexandre Bury 2.5k Sep 13, 2021
Low-level Rust library for implementing terminal command line interface, like in embedded systems.

Terminal CLI Need to build an interactive command prompt, with commands, properties and with full autocomplete? This is for you. Example, output only

HashMismatch 40 Jul 4, 2021