Prometheus exporter that scrapes data in different formats

Overview

data-exporter

A prometheus exporter that scrapes remote data or local files and converts them to prometheus metrics. It is similar to json_exporter, but adds the possibility to transform the data before metrics are extracted, and is not limited to only support JSON data.

Docker image

A docker image is built as part of this repository. Find the published versions here

Local development

Run tests

cargo test

Run the exporter

cargo run -- --config examples/config.yaml

Configuration

data-exporter is configured via a configuration file in YAML format (see this example), and via command-line flags.

Command-line flags

Run the help command to get all available flags

data-exporter --help

Configuration file

In the configuration file all scraped metrics are configured.o

metrics: [<metric_config>]

<metric_config>

# name of the metric when scraped
name: <string>

# metric help string when scraped
help: <string>

# targets to scrape data from
targets: [<target_config>]

# pipeline stages to transform the data before parsing it
pipeline_stages: [<pipeline_stage_config>]

# parser for parsing metrics from data
parser: <parser_config>

# set a constant value for the metric, it is required to set either this or `value` in `parser_config`
value: <float64>

<target_config>

file

type: file

# path to local file
path: <string>

http

type: http

# url to scrape
url: <string>

<pipeline_stage_config>

jq

type: jq

# jq query to execute on data
query: <string>

regex

type: regex

# regex pattern
pattern: <regex>

# replacement string
replace: <string>

<parser_config>

json

type: json

# fields to extract as labels
labels: [<string>]

# field to extract as value, it is required to set either this or `value` in `metric_config`
value: <string>
You might also like...
Parse and encoding of data using the SCTE-35 standard.

SCTE-35 lib and parser for Rust Work in progress! This library provide access to parse and encoding of data using the SCTE-35 standard. This standard

A HashMap/Vector hybrid: efficient, ordered key-value data storage in Rust.

hashvec A HashVec is a hash map / dictionary whose key-value pairs are stored (and can be iterated over) in a fixed order, by default the order in whi

Library and proc macro to analyze memory usage of data structures in rust.
Library and proc macro to analyze memory usage of data structures in rust.

Allocative: memory profiler for Rust This crate implements a lightweight memory profiler which allows object traversal and memory size introspection.

A compact generational arena data structure for Rust.

Compact Generational Arena This crate provides ArenaT, a contiguous growable container which assigns and returns IDs to values when they are added t

A real-time data backend for browser-based applications.

DriftDB DriftDB is a real-time data backend for browser-based applications. For more information, see driftdb.com. Structure of this repo docs/: main

A fast rendezvous in rust where data can optionally be swapped between the two threads.

rendezvous_swap A rendezvous is an execution barrier between a pair of threads, but this crate also provides the option of swapping data at the synchr

A special rope, designed to work with any data type that is not String

AnyRope AnyRope is an arbitrary data type rope for Rust, designed for similar operations that a rope would do, but targeted at data types that are not

bare-bones "reactive programming" (change propogation) using a central data dependency graph

mini-rx: Tiny reactive programming change propagation a la scala.rx Cargo documentation Example use mini_rx::*; fn example() { // setup let side_ef

Platform independent data channels for WebRTC/Rust.

preach Platform independent data channels Preach provides an abstraction for WebRTC data channels that runs on both native and web platforms. Preach m

Releases(v0.4.2)
  • v0.4.2(Dec 16, 2022)

    What's Changed

    • Cargo update by @fredr in https://github.com/fredr/data-exporter/pull/16 and https://github.com/fredr/data-exporter/pull/19
    • Add cargo deny by @fredr in https://github.com/fredr/data-exporter/pull/17
    • Update to axum 0.6 by @fredr in https://github.com/fredr/data-exporter/pull/18
    • Set valid address as default, and let clap handle parsing by @fredr in https://github.com/fredr/data-exporter/pull/20
    • Bump rust by @fredr in https://github.com/fredr/data-exporter/pull/21

    Full Changelog: https://github.com/fredr/data-exporter/compare/v0.4.1...v0.4.2

    Source code(tar.gz)
    Source code(zip)
  • v0.4.1(Nov 29, 2022)

    What's Changed

    • bump logfmt by @fredr in https://github.com/fredr/data-exporter/pull/15

    Full Changelog: https://github.com/fredr/data-exporter/compare/v0.4.0...v0.4.1

    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(Aug 14, 2022)

    What's Changed

    • Use axum and metrics-rs instead of actix by @fredr in https://github.com/fredr/data-exporter/pull/8
    • Update dependencies by @fredr in https://github.com/fredr/data-exporter/pull/9
    • Tracing subscriber and format logs with logfmt by @fredr in https://github.com/fredr/data-exporter/pull/10
    • Add code lints by @fredr in https://github.com/fredr/data-exporter/pull/11
    • Update to Rust 1.63 by @fredr in https://github.com/fredr/data-exporter/pull/12

    Full Changelog: https://github.com/fredr/data-exporter/compare/v0.3.1...v0.4.0

    Source code(tar.gz)
    Source code(zip)
  • v0.3.1(Dec 21, 2021)

    What's Changed

    • Set user agent on http requests by @fredr in https://github.com/fredr/data-exporter/pull/5
    • prep. for release, and fix helm chart by @fredr in https://github.com/fredr/data-exporter/pull/6

    Full Changelog: https://github.com/fredr/data-exporter/compare/v0.3.0...v0.3.1

    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Dec 20, 2021)

    What's Changed

    • Feature: add regex parser by @fredr in https://github.com/fredr/data-exporter/pull/4
    • Make pipeline_stages optional by @fredr in https://github.com/fredr/data-exporter/pull/4

    Full Changelog: https://github.com/fredr/data-exporter/compare/v0.2.1...v0.3.0

    Source code(tar.gz)
    Source code(zip)
  • v0.2.1(Dec 20, 2021)

    What's Changed

    • Regex stage should replace all occurrences by @fredr in https://github.com/fredr/data-exporter/pull/1
    • add github action for ci by @fredr in https://github.com/fredr/data-exporter/pull/2

    Full Changelog: https://github.com/fredr/data-exporter/compare/v0.2.0...v0.2.1

    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Nov 29, 2021)

Owner
Fredrik Enestad
Fredrik Enestad
Serde support for n-dimensional arrays from self-describing formats

serde-ndim Overview This crate provides a way to serialize and deserialize arrays of arbitrary dimensionality from self-described formats such as JSON

Ingvar Stepanyan 5 Apr 3, 2023
Game development practices with Rust programming language. I want to use different crates for this.

Hazır Oyun Motorlarını Kullanarak Rust Dili Yardımıyla Oyunlar Geliştirmek Rust programlama dilinde oyun geliştirmek için popüler birkaç hazır çatıyı

Burak Selim Senyurt 16 Dec 27, 2022
An expression based data notation, aimed at transpiling itself to any cascaded data notation.

Lala An expression oriented data notation, aimed at transpiling itself to any cascaded data notation. Lala is separated into three components: Nana, L

null 37 Mar 9, 2022
Proof-of-concept for a memory-efficient data structure for zooming billion-event traces

Proof-of-concept for a gigabyte-scale trace viewer This repo includes: A memory-efficient representation for event traces An unusually simple and memo

Tristan Hume 59 Sep 5, 2022
Data structures and algorithms for 3D geometric modeling.

geom3d Data structures and algorithms for 3D geometric modeling. Features: Bezier curve and surface B-Spline curve and surface Spin surface Sweep surf

Junfeng Liu 31 Sep 20, 2022
A tool to deserialize data from an input encoding, transform it and serialize it back into an output encoding.

dts A simple tool to deserialize data from an input encoding, transform it and serialize it back into an output encoding. Requires rust >= 1.56.0. Ins

null 11 Dec 14, 2022
A simple and convenient way to bundle owned data with a borrowing type.

A simple and convenient way to bundle owned data with a borrowing type. The Problem One of the main selling points of Rust is its borrow checker, whic

Dmitry Zamkov 20 Dec 21, 2022
A library for transcoding between bytes in Astro Notation Format and Native Rust data types.

Rust Astro Notation A library for transcoding between hexadecimal strings in Astro Notation Format and Native Rust data types. Usage In your Cargo.tom

Stelar Software 1 Feb 4, 2022
An implementation of a predicative polymorphic language with bidirectional type inference and algebraic data types

Vinilla Lang Vanilla is a pure functional programming language based on System F, a classic but powerful type system. Merits Simple as it is, Vanilla

Zehao Chen 73 Aug 4, 2022
A parser for the perf.data format

linux-perf-data This repo contains a parser for the perf.data format which is output by the Linux perf tool. It also contains a main.rs which acts sim

Markus Stange 8 Dec 29, 2022