A library to access BGPKIT Broker API and enable searching for BGP data archive files over time from public available data sources.

Related tags

Miscellaneous bgpkit-broker
Overview

BGPKIT Broker

Crates.io MIT licensed Twitter

BGPKIT Broker is a online data API service that allows users to search for publicly available BGP archive files by time, collector, project, or data type. The service indexes the archives in close to real-time (delay is less than 5 minutes). Currently, we are indexing BGP table dump and updates files from RIPE RIS and RouteViews.

This Rust library provides access to the BGPKIT Broker API with the capability to search and paginate results.

Example

use bgpkit_broker::{BgpkitBroker, BrokerItem, QueryParams};

let mut params = QueryParams::new();
params = params.start_ts(1634693400);
params = params.end_ts(1634693400);
params = params.page_size(10);
params = params.page(2);

let mut broker = BgpkitBroker::new("https://api.broker.bgpkit.com/v1");
broker.set_params(&params);

// method 1: create iterator from reference (so that you can reuse the broker object)
// same as `&broker.into_iter()`
for item in &broker {
println!("{:?}", item);
}

// method 2: create iterator from the broker object (taking ownership)
let items = broker.into_iter().collect::<Vec<BrokerItem>>();

assert_eq!(items.len(), 48);

Contribution

Issues and Pull Requests

If you found any issues of this Rust library or would like to contribute to the code base, please feel free to open an issue or pull request. Code or documentation issues/PRs are both welcome.

Data Provider

If you have publicly available data and want to be indexed BGPKIT Broker service, please send us an email at [email protected]. Our back-end service is designed to be flexible and should be able to adapt to most data archiving approaches.

On-premise Deployment

We provide service to allow companies to host their own BGP Broker backend on-premise to allow maximum performance and customization. If you are interested in deploying one, please contact us at [email protected].

Built with ❤️ by BGPKIT Team

BGPKIT is a small-team start-up that focus on building the best tooling for BGP data in Rust. We have 10 years of experience working with BGP data and believe that our work can enable more companies to start keeping tracks of BGP data on their own turf. Learn more about what services we provide at https://bgpkit.com.

https://bgpkit.com/favicon.ico

Comments
  • bgpkit-broker-backend integration

    bgpkit-broker-backend integration

    I am trying to understand how to use bgpkit-broker locally. I believe what bgpkit-broker-backend achieves is to substitute the remote MRT databases with a local copy. Following this, should I replace reference URLs (as seen in the below) with the "0.0.0.0:18888" to make bgpkit-broker talk to the local API instead of the remote one? Or is it more intricate than this? https://github.com/bgpkit/bgpkit-broker/blob/25b3c3c4ed5d373245efb105bc47bda4a1ac478e/src/lib.rs#L301

    Thank you for your support.

    opened by egecemkirci 2
  • Add `.latest()` function call

    Add `.latest()` function call

    The .latest() function call returns the latest MRT file for each collector we keep tracking.

    The query goes to RESTful API endpoint at https://api.broker.bgpkit.com/v2/latest.

    #[derive(Debug, Serialize, Deserialize)]
pub struct CollectorLatestItem {
    /// timestamp of the file
    pub timestamp: chrono::NaiveDateTime,
    /// Delay in seconds between the time file available and the timestamp of the file
    pub delay: f64,
    /// the collector id of the item: e.g. `rrc00`
    pub collector_id: String,
    /// type of the data item: `rib` or `update`
    pub data_type: String,
    /// the URL to the data item file
    pub item_url: String,
    /// the URL to the route collector
    pub collector_url: String,
    /// rough file size extracted from the hosting site page
    pub rough_size: i64,
    /// exact file size extracted by crawling the file
    pub exact_size: i64,
}

    This pull request should resolve issue #5.

    opened by digizeph 0
  • Major overhaul

    Major overhaul

    This is a major overhaul of the broker Rust SDK.

    The main noticeable changes include:

    1. new easier-to-use constructor with default values for query and broker URL;
    2. factory-pattern-style constructor allowing assigning query params directly;
    3. simplified internal iterator implementation;
    4. main query function renamed to .query().
    5. added .latest() function to gather information about the latest MRT file for each collector we keep track of.

    This pull request contains multiple breaking changes to v0.4.

    This resolves issue #5 and #6.

    opened by digizeph 0
  • [feature] add default base URL for better usability

    [feature] add default base URL for better usability

    Currently, we need to explicitly specify the URL for the broker instance. We should add a default broker instance for better developer experience, especially for those who don't want to bother with setting their own separate instances.

    enhancement 
    opened by digizeph 0
  • [feature] add function to get latest data

    [feature] add function to get latest data

    https://api.broker.bgpkit.com/v2/latest

    data format:

    • array
      • object:
        • timestamp
        • delay
        • collector_id
        • data_type
        • item_url
        • rough_size
        • exact_size
        • collector_url
    [
{
"timestamp": "2022-07-19T18:00:00",
"delay": 1520.979425,
"collector_id": "route-views.amsix",
"data_type": "rib",
"item_url": "http://archive.routeviews.org/route-views.amsix/bgpdata/2022.07/RIBS/rib.20220719.1800.bz2",
"rough_size": 135266304,
"exact_size": 0,
"collector_url": "http://archive.routeviews.org/route-views.amsix/bgpdata"
},
...
]
    enhancement 
    opened by digizeph 0
Releases(v0.5.0)

  • v0.5.0(Dec 19, 2022)

    Breaking Change

    This version changes the core interface of how BgpkitBroker objects are constructed. It will not be compatible with V0.4 and before.

    Added features

    • Implement Display for BrokerItem and QueryResult by @digizeph in https://github.com/bgpkit/bgpkit-broker/pull/10
    • replace reqwest with ureq by @digizeph in https://github.com/bgpkit/bgpkit-broker/pull/12
    • Add .latest() function call by @digizeph in https://github.com/bgpkit/bgpkit-broker/pull/14

    Major Overhaul (https://github.com/bgpkit/bgpkit-broker/pull/13)

    The main noticeable changes include:

    1. new easier-to-use constructor with default values for query and broker URL;
    2. factory-pattern-style constructor allowing assigning query params directly;
    3. simplified internal iterator implementation;
    4. main query function renamed to .query().
    5. added .latest() function to gather information about the latest MRT file for each collector we keep track of.
    6. improved documentation

    Full Changelog: https://github.com/bgpkit/bgpkit-broker/compare/v0.4.1...v0.5.0

    Source code(tar.gz)
    Source code(zip)

  • v0.4.1(Jul 10, 2022)

    What's Changed

    • Fix collector filtering by @digizeph in https://github.com/bgpkit/bgpkit-broker/pull/4

    Full Changelog: https://github.com/bgpkit/bgpkit-broker/compare/v0.4.0...v0.4.1

    Source code(tar.gz)
    Source code(zip)

  • v0.4.0(May 4, 2022)

    Breaking Change

    This version no longer support the V1 backend API. If you have a V1 API running, please consider switching to V2 API, or use hosted https://api.broker.bgpkit.com/v2 as the broker_url.

    What's Changed

    • Support V2 broker API by @digizeph in https://github.com/bgpkit/bgpkit-broker/pull/3

    Full Changelog: https://github.com/bgpkit/bgpkit-broker/compare/v0.3.2...v0.4.0

    Source code(tar.gz)
    Source code(zip)

  • v0.3.2(Oct 29, 2021)

  • v0.3.1(Oct 29, 2021)

    Feature

    Added new new_with_params constructor function for BgpkitBroker. This allows users to construct a broker instance directly with query parameters, without needing to make a mut broker and then set the query parameters.

    Example:

        let broker = BgpkitBroker::new_with_params(
        "https://api.broker.bgpkit.com/v1",
        QueryParams{
            start_ts: Some(1634693400),
            end_ts: Some(1634693400),
            page: 2,
            ..Default::default()
        });


    // method 1: create iterator from reference (so that you can reuse the broker object)
    // same as `&broker.into_iter()`
    for item in &broker {
        println!("{:?}", item);
    }
    Source code(tar.gz)
    Source code(zip)

  • v0.3.0(Oct 23, 2021)

    New Feature: Iterator

    Calling BGPKIT Broker API is never easier with iterator!

    Simply call for loop on a broker object will give you a stream of items until it reaches the end:

    for item in broker {
   println!("{:?}", item);
}

    Now with iterator implemented, you can also do some iter magic like map, for_each, count etc.

    broker.into_iter().count()

    Improved Documentation

    In this version release, we also took our time fleshing out the documentation of this crate, and can be viewed on crates.io at https://docs.rs/bgpkit-broker/0.3.0/bgpkit_broker/

    Source code(tar.gz)
    Source code(zip)
Owner
BGPKIT
BGPKIT
GitHub
A project for managing all Pop!_OS sources

Pop!_OS Pop!_OS is designed for people who use their computer to create; whether it’s complicated, professional grade software and products, sophistic

Pop!_OS 2k Jan 6, 2023
📜 A pci.ids-compliant library for getting information about available PCI devices.

aparato A pci.ids-compliant library for getting information about available PCI devices. Usage Add the following to your project's Cargo.toml file: ap

Aziz Ben Ali 22 Nov 14, 2022
The nightly_crimes!{} macro commits horrible crimes to allow you to enable nightly features on the stable compiler.

The nightly_crimes!{} macro commits horrible crimes to allow you to enable nightly features on the stable compiler.

Mara Bos 151 Dec 16, 2022
Enable floating point exceptions with backtraces.

Batman Have you ever written a function like this that was hard to debug? fn main() { let signal = [""; 16].join(&format!("{}", f64::sqrt(50.3 - 5

Jay Oster 2 Nov 15, 2022
An investigation to enable ethernet with smoltcp on the SAM E70 XPLAINED ULTRA EVALUATION KIT

An investigation to enable ethernet with smoltcp on the SAM E70 XPLAINED ULTRA EVALUATION KIT

James Munns 2 Mar 10, 2022
Telegram bot for searching in Arch User Repository ( AUR ); Implemented using rust.

AurSearchBot A Telegram Inline Search Bot Written in Rust Introduction Telegram Bot that can search AUR ( Arch User Repository ) in inline mode. This

AlenPaulVarghese 3 Feb 15, 2022
A static mail HTML archive for the 21st century, written in Rust

?? Crabmail ?? self-hosted / github mirror A static mail HTML archive for the 21st century, written in Rust. Includes helpful "modern" features that e

Alex Wennerberg 18 Oct 11, 2022
Hi I'm Sophy, a discord bot in devlopment, soon I'll be available to help everyone (❁´◡`❁)

Sophy Bot Hi I'm Sophy, a discord bot in devlopment, soon I'll be available to help everyone (❁´◡`❁) Contribution Do you like me and want to help me?

Far Dragi 0 May 30, 2022
A Garry's Mod module that lets you check which fonts are available on the system

gm_fontsx Since apparently we're never getting a proper way to check for installed fonts on Garry's Mod, this has to exist ?? Usage require("fontsx")

Earu 4 Mar 14, 2022
A publicly available implementation of knewjade's NNUE based search heuristic for HATETRIS.

A publicly available implementation of knewjade's NNUE based search heuristic for HATETRIS.

Felipe 4 Mar 13, 2023
`fugit` provides a comprehensive library of `Duration` and `Instant` for the handling of time in embedded systems, doing all it can at compile time.

fugit fugit provides a comprehensive library of Duration and Instant for the handling of time in embedded systems, doing all it can at compile time. T

Emil Fresk 40 Oct 2, 2022
A repository full of manually generated hand curated JSON files, which contain the API Types that the Discord API returns.

Discord API Types A repository full of manually generated hand curated JSON files, which contain the API Types that the Discord API returns. Also did

Unofficial Discord Documentation 1 Sep 16, 2022
This is a public snapshot of Fly's init code. It powers every Firecracker microvm we run for our users.

Fly Init This is a public snapshot of Fly's init code. It powers every Firecracker microvm we run for our users. It is Rust-based and we thought makin

fly.io 186 Dec 30, 2022
Single-operator public liveness notification

Single-operator public liveness notification

Charlotte Som 1 Feb 16, 2022
NSE is a rust cli binary and library for extracting real-time data from National Stock Exchange (India)

NSE Check out the sister projects NsePython and SaveKiteEnctoken which are Python & Javascript libraries to use the NSE and Zerodha APIs respectively

Techfane Technologies 4 Nov 28, 2022
Community SVD file, peripheral access crate in embedde Rust for WinnerMicro W800, W801 & W806 chip

Community SVD file, peripheral access crate in embedde Rust for WinnerMicro W800, W801 & W806 chip

Luo Jia 37 Jul 31, 2022
Allow raw pointer access without those pesky unsafe blocks everywhere!

Allow raw pointer access without those pesky unsafe blocks everywhere!

null 1 Jan 23, 2022
Modrinth API is a simple library for using, you guessed it, the Modrinth API in Rust projects

Modrinth API is a simple library for using, you guessed it, the Modrinth API in Rust projects. It uses reqwest as its HTTP(S) client and deserialises responses to typed structs using serde.

null 21 Jan 1, 2023
Rust bindings for the KING OF TIME API

Rust bindings for the KING OF TIME API Example Prints if you are at work or not at work. $ cargo run --example tc -- status Record the time you start

Idein Inc. 2 Oct 11, 2021