hornet

hornet is a Performance Co-Pilot (PCP) Memory Mapped Values (MMV) instrumentation library written in Rust.

Contents

• What is PCP MMV instrumentation?
• Usage
• API
  • Singleton Metric
  • Instance Metric
  • Special Metrics
  • Client
  • Monitoring metrics
• License

What is PCP MMV instrumentation?

Performance Co-Pilot is a systems performance analysis framework with a distributed and scalable architecture. It supports a low overhead method for instrumenting applications called Memory Mapped Values (MMV), in which instrumented processes share part of their virtual memory address space with another monitoring process through a common memory-mapped file. The shared address space contains various performance analysis metrics stored in a structured binary data format called MMV; it's formal spec can be found here. When processes wish to update their metrics, they simply write certain bytes to the memory mapped file, and the monitoring process reads it at appropriate times. No explicit inter-process communication, synchronization or systems calls are involved.

Usage

• Add the hornet dependency to your Cargo.toml

  [dependencies]
  hornet = "0.1.0"

• Include the hornet crate in your code and import the following modules

  extern crate hornet;
  
  use hornet::client::Client;
  use hornet::client::metric::*;

API

There are essentially two kinds of metrics in hornet.

Singleton Metric

A singleton metric is a metric associated with a primitive value type, a Unit, a Semantics type, and some metadata. A primitive value can be any one of i64, u64, i32, u32, f64, f32, or String,

The primitive value type of a metric is determined implicitly at compile-time by the inital primitive value passed to the metric while creating it. The programmer also needn't worry about reading or writing data of the wrong primitive type from a metric, as the Rust compiler enforces type safety for a metric's primitive value during complilation.

Let's look at creating a simple i64 metric

let mut metric = Metric::new(
    "simple", // metric name
    1, // inital value of type i64
    Semantics::Counter,
    Unit::new().count(Count::One, 1).unwrap(), // unit with a 'count' dimension of power 1
    "Short text", // short description
    "Long text", // long description
).unwrap();

If we want to create an f64 metric, we simply pass an f64 inital value instead

let mut metric = Metric::new(
    "simple_f64", // metric name
    1.5, // inital value of type f64
    Semantics::Instant,
    Unit::new().count(Time::Sec, 1).unwrap(), // unit with a 'time' dimension of power 1
    "Short text", // short description
    "Long text", // long description
).unwrap();

And similarly for a String metric

let mut metric = Metric::new(
    "simple_string", // metric name
    "Hello, world!".to_string(), // inital value of type String
    Semantics::Discrete,
    Unit::new().unwrap(), // unit with no dimension
    "Short text", // short description
    "Long text", // long description
).unwrap();

The detailed API on singleton metrics can be found here.

Instance Metric

An instance metric is similar to a singleton metric in that it is also associated with a primitive valye type, Unit, and Semantics, but additionally also holds multiple independent primitive values of the same type. The same type inference rules also hold for instance metrics - the type of the inital value determines the type of the instance metric.

Before we can create an instance metric, we need to create what's called an instance domain. An instance domain is a set of String values that act as unique identifiers for the multiple independent values of an instance metric. Why have a separate object for this purpose? So that we can reuse the same identifiers as a "domain" for several different but related instance metrics. An example will clear this up.

Suppose we are modeling the fictional Acme Corporation factory. Let's assume we have three items that can be manufactured - Anvils, Rockets, and Giant Rubber Bands. Each item is associated with a "count" metric of how many copies have been manufactured so far, and a "time" metric of how much time has been spent manufacturing each item. We can create instance metrics like so

/* instance domain */
let indom = Indom::new(
    &["Anvils", "Rockets", "Giant_Rubber_Bands"],
    "Acme products", // short description
    "Most popular products produced by the Acme Corporation" // long description
).unwrap();
  
/* two instance metrics */

let mut counts = InstanceMetric::new(
    &indom,
    "products.count", // instance metric name
    0, // inital value of type i64
    Semantics::Counter,
    Unit::new().count(Count::One, 1).unwrap(),
    "Acme factory product throughput",
    "Monotonic increasing counter of products produced in the Acme Corporation factory since starting the Acme production application."
).unwrap();

let mut times = InstanceMetric::new(
    &indom,
    "products.time",  // instance metric name
    0.0, // inital value of type f64
    Semantics::Instance,
    Unit::new().time(Time::Sec, 1).unwrap(),
    "Time spent producing products",
    "Machine time spent producing Acme Corporation products."
).unwrap();

Here, our indom contains three identifiers - Anvils, Rockets and Giant_Rubber_Bands. We've created two instance metrics - counts of type i64 and times of type f64 with relevant units and semantics.

The detailed API on instance metrics can be found here.

Updating metrics

So far we've seen how to create metrics with various attributes. Updating their primitive values is pretty simple.

For singleton metrics, the val(&self) -> &T method returns a reference to the underlying value, and the set_val(&mut self, new_val: T) -> io::Result<()> method updates the underlying value and writes to the memory mapped file. The arguments and return values for these methods are generic over the different primitive types for a metric, and hence are completely type safe.

For instance metrics, the val(&self, instance: &str) -> Option<&T> method returns a reference to the primitive value for the given instance identifier, if it exists. The set_val(&mut self, instance: &str, new_val: T) -> Option<io::Result<()>> method updates the primitive value for the given instance identifier, if it exists. These methods are similarly generic over primitive value types.

Special metrics

Singleton metrics and instance metrics are powerful and general enough to be used for a wide variety of performance analysis needs. However, for many common applications, simpler metric interfaces would be more appropriate and easy to use. Hence hornet includes 6 high-level metrics that are built on top of singleton and instance metrics, and they offer a more specialized and simpler API.

Counter

A Counter is a singleton metric of type u64, Counter semantics, and unit of 1 count dimension. It implements the following methods: up to increment by one, inc to increment by a delta, reset to set count to the inital count, and val to return the current count.

let mut c = Counter::new(
    "counter", // name
    1, // inital value
    "", "" // short and long description strings
).unwrap();

c.up(); // 2
c.inc(3); // 5
c.reset(); // 1

let count = c.val(); // 1

The CountVector is the instance metric version of the Counter. It holds multiple counts each associated with a String identifier.

Gauge

A Gauge is a singleton metric of type f64, Instant semantics, and unit of 1 count dimension. It implements the following methods: inc to increment the gauge by a delta, dec to decrement the gauge by a delta, set to set the gauge to an arbritrary value, and val which returns the current value of the gauge.

let mut gauge = Gauge::new("gauge", 1.5, "", "").unwrap();

gauge.set(3.0).unwrap(); // 3.0
gauge.inc(3.0).unwrap(); // 6.0
gauge.dec(1.5).unwrap(); // 4.5
gauge.reset().unwrap();  // 1.5

The GaugeVector is the instance metric version of the Gauge. It holds multiple gauge values each associated with an identifier.

Timer

A Timer is a singleton metric of type i64, Instant semantics, and a user specified time unit. It implements the following methods: start starts the timer by recording the current time, stop stops the timer by recording the current time and returns the elapsed time since the last start, and elapsed returns the total time elapsed so far between all start and stop pairs.

let mut timer = Timer::new("timer", Time::MSec, "", "").unwrap();

timer.start().unwrap();
let e1 = timer.stop().unwrap();

timer.start().unwrap();
let e2 = timer.stop().unwrap();

let elapsed = timer.elapsed(); // = e1 + e2

Histogram

A Histogram is a high dynamic range (HDR) histogram metric which records u64 data points and exports various statistics about the data. It is implemented using an instance metric of f64 type and Instance semantics. The Histogram metric is infact essentially a wrapper around the Histogram object from the hdrsample crate, and it exports the maximum, minimum, mean and standard deviation statistics to the MMV file.

let low = 1;
let high = 100;
let sigfig = 5;

let mut hist = Histogram::new(
    "histogram",
    low,
    high,
    sigfig,
    Unit::new().count(Count::One, 1).unwrap(),
    "Simple histogram example", ""
).unwrap();

let range = Range::new(low, high);
let mut thread_rng = thread_rng();

for _ in 0..100 {
    hist.record(range.ind_sample(&mut thread_rng)).unwrap();
}

Much of the Histogram API is largely similar to the hdrsample API.

Client

In order to export our metrics to a memory mapped file, we must first create a Client

let client = Client::new("client").unwrap(); // MMV file will be named 'client'

Now to export metrics, we simply call export

client.export(&mut [&mut metric1, &mut metric2, &mut metric3]);

If you have a valid PCP installation, the Client writes the MMV file to $PCP_TMP_DIR/mmv/, and otherwise it writes it to /tmp/mmv/.

After metrics are exported through a Client, all updates to their primitive values will show up in the MMV file.

Monitoring metrics

With a valid PCP installation on a machine, metrics can be monitored externally by using the follwing command

$ pminfo -f mmv._name_

where _name_ is the name passed to Client while creating it.

Another way to inspect metrics externally is to dump the contents of the MMV file itself. This can be done using a command line tool called mmvdump included in hornet. After issuing cargo build from within the project directory, mmvdump can be found built under target/debug/.

Usage of mmvdump is pretty straightforward

$ ./mmvdump simple.mmv

Version    = 1
Generated  = 1468770536
TOC count  = 3
Cluster    = 127
Process    = 29956
Flags      = process (0x2)

TOC[0]: toc offset 40, metrics offset 88 (1 entries)
  [725/88] simple.counter
      type=Int32 (0x0), sem=counter (0x1), pad=0x0
      unit=count (0x100000)
      (no indom)
      shorttext=A Simple Metric
      longtext=This is a simple counter metric to demonstrate the hornet API

TOC[1]: toc offset 56, values offset 192 (1 entries)
  [725/192] simple.counter = 42

TOC[2]: toc offset 72, strings offset 224 (2 entries)
  [1/224] A Simple Metric
  [2/480] This is a simple counter metric to demonstrate the hornet API

License

Licensed under either of

• Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.