DataFrame & its adaptors

Jacob Xie

Last update: Dec 12, 2022

Related tags

Data processing fabrix

Overview

Fabrix

Fabrix is a lib crate, who uses Polars Series and DataFrame as fundamental data structures, and is capable to communicate among different data sources, such as Database (MySql/Postgres/Sqlite), File, BSON/JSON and etc. Furthermore, ETL process among different sources are provided as well, and additionally, manipulation or operation on data itself is enhanced.

Structure

├── core
│   ├── value.rs                        // the smallest data unit
│   ├── series.rs                       // series of value
│   ├── dataframe.rs                    // collection of series, with index series
│   ├── row.rs                          // row-wise data structure
│   ├── util.rs                         // utility functions
│   └── macros.rs
│
├── sources
│   ├── db
│   │   ├── sql_builder                 // SQL builder
│   │   │   ├── adt.rs                  // algebraic data type
│   │   │   ├── query_ddl.rs            // ddl query: check table or schema
│   │   │   ├── query_dml.rs            // dml query: select and etc
│   │   │   ├── mutation_ddl.rs         // ddl mutation: create/alter/drop table
│   │   │   ├── mutation_dml.rs         // dml mutation: insert/update/delete data
│   │   │   ├── interface.rs            // SQL builder & ddl/dml logic interface
│   │   │   ├── builder.rs              // SQL builder & ddl/dml logic implement
│   │   │   └── macros.rs
│   │   │
│   │   └── sql_executor
│   │       ├── types.rs                // Conversion between Sql data type and Fabrix `Value`
│   │       ├── processor.rs            // Sql row process, turn raw sql row into `Vec<Value>` or `Row`
│   │       ├── loader.rs               // Database loader, CRUD logic implementation
│   │       ├── executor.rs             // Sql executor, business logic implementation
│   │       └── macros.rs
│   │
│   ├── file
│   │   ├── xl                          // Excel
│   │   │   ├── util.rs
│   │   │   ├── worksheet.rs
│   │   │   ├── workbook.rs
│   │   │   └── executor.rs
│   │   │
│   │   ├── csv                         // CSV
│   │   │
│   │   └── parquet                     // Parquet
│   │
│   └── bson
│
├── errors.rs                           // error handling
│
├── macros.rs                           // helpful macros
│
├── prelude.rs                          // prelude of this crate
│
└── lib.rs

Examples

under construction...

Note

Progression of dataframe/core: value -> series -> dataframe -> row
Progression of dataframe/sources: db -> file -> json
- db: sql_builder -> sql_executor
- file: xl -> csv -> parquet

DataFrame / Series data processing in Rust

black-jack While PRs are welcome, the approach taken only allows for concrete types (String, f64, i64, ...) I'm not sure this is the way to go. I want

30 Dec 10, 2022

Provides multiple-dtype columner storage, known as DataFrame in pandas/R

brassfibre Provides multiple-dtype columner storage, known as DataFrame in pandas/R. Series Single-dtype 1-dimentional vector with label (index). Crea

21 Nov 28, 2022

A dataframe manipulation tool inspired by dplyr and powered by polars.

dply is a command line tool for viewing, querying, and writing csv and parquet files, inspired by dplyr and powered by polars. Usage overview A dply p

14 May 29, 2023

Peakrs Dataframe is a library and framework facilitates the extraction, transformation, and loading (ETL) of data.

Peakrs Dataframe Peakrs Dataframe is a library and framework facilitates the extraction, transformation, and loading (ETL) of data. Its first applicat

5 Sep 6, 2023

Detects usage of unsafe Rust in a Rust crate and its dependencies.

cargo-geiger ☢️ A program that lists statistics related to the usage of unsafe Rust code in a Rust crate and all its dependencies. This cargo plugin w

1.1k Dec 26, 2022

mee6 but its foss

mee6 leveling clone but its free and open source stuff u need: rust + cargo postgres redis discord bot token use the thing: setup stuff in .env build

14 Aug 15, 2021

CLI utility to move (or rename) your files to a new location and redirect all of its symbolic links, to the new path

Move Links CLI utility to move (or rename) your files to a new location and redirect all of its symbolic links, to the new path (or name). Usage execu

18 May 22, 2022

Akasio is a simple HTTP server that redirects traffic based on a JSON redirect table. This is its Rust implementation.

This page is inaccurate and is pending updates. Akasio (Rust) Description Akasio is a simple HTTP server that redirects traffic based on a JSON redire

5 May 2, 2022

Detects usage of unsafe Rust in a Rust crate and its dependencies.

cargo-geiger ☢️ A program that lists statistics related to the usage of unsafe Rust code in a Rust crate and all its dependencies. This cargo plugin w

1.1k Jan 8, 2023

miette is a diagnostic library for Rust. It includes a series of traits/protocols that allow you to hook into its error reporting facilities, and even write your own error reports!

1.2k Jan 1, 2023

Midnote is a terminal application that reads a MIDI file and displays you its notes bar-by-bar, while playing it.

MIDNOTE Midnote is a terminal application that reads a MIDI file and displays you its notes bar-by-bar, while playing it. Goals As a blind musician my

4 Oct 30, 2022

Hatch new projects like a chick coming out of its egg

🐣 Cargo Hatch Hatch new projects like a chick coming out of its egg. Cargo hatch is a cargo init/cargo new on steroids, allowing complex templates th

6 Nov 2, 2022

☁ file.AsyncWrite - because THEY wont do it!!!!!! and its in RUST

☁ gm_async_write Simple module that adds file.AsyncWrite and file.AsyncAppend to Garry's Mod. These functions are mostly based off file.AsyncRead and

18 Dec 24, 2022

A plugin for Jupyter Notebooks that shows you its energy use.

Jupyter Energy Jupyter Notebooks are a data science tool mostly used for statistics and machine learning, some of the most energy-intensive computing

3 Jun 29, 2022

Black-box integration tests for your REST API using the Rust and its test framework

restest Black-box integration test for REST APIs in Rust. This crate provides the [assert_api] macro that allows to declaratively test, given a certai

10 Nov 23, 2022

Detects usage of unsafe Rust in a Rust crate and its dependencies.

cargo-geiger ☢️ Looking for maintainer: https://github.com/rust-secure-code/cargo-geiger/issues/210 A program that lists statistics related to the usa

1.1k Jan 4, 2023

IDP2P is a peer-to-peer identity protocol which enables a controller to create, manage and share its own proofs as well as did documents

IDP2P Experimental, inspired by ipfs, did:peer and keri Background See also (related topics): Decentralized Identifiers (DIDs) Verifiable Credentials

5 Oct 31, 2022

An efficient implementation of Partitioned Label Trees & its variations for extreme multi-label classification

Omikuji An efficient implementation of Partitioned Label Trees (Prabhu et al., 2018) and its variations for extreme multi-label classification, writte

73 Nov 7, 2022

A highly modular Bitcoin Lightning library written in Rust. Its Rust-Lightning, not Rusty's Lightning!

Rust-Lightning is a Bitcoin Lightning library written in Rust. The main crate, lightning, does not handle networking, persistence, or any other I/O. Thus, it is runtime-agnostic, but users must implement basic networking logic, chain interactions, and disk storage. More information is available in the About section.

850 Jan 3, 2023

Comments

Refactor
Remove unsafe code and add new dependency ref-cast;

Remove SeriesViewer and FabrixViewer traits;

Remove SeriesRef struct;

Remove FabrixRef struct and add FabrixDataFrame.
opened by Jacobbishopxy 0

DataFrame & its adaptors

Related tags

Overview

Fabrix

Structure

Examples

Note

You might also like...

DataFrame / Series data processing in Rust

Provides multiple-dtype columner storage, known as DataFrame in pandas/R

A dataframe manipulation tool inspired by dplyr and powered by polars.

Peakrs Dataframe is a library and framework facilitates the extraction, transformation, and loading (ETL) of data.

Detects usage of unsafe Rust in a Rust crate and its dependencies.

mee6 but its foss

CLI utility to move (or rename) your files to a new location and redirect all of its symbolic links, to the new path

Akasio is a simple HTTP server that redirects traffic based on a JSON redirect table. This is its Rust implementation.

Detects usage of unsafe Rust in a Rust crate and its dependencies.

miette is a diagnostic library for Rust. It includes a series of traits/protocols that allow you to hook into its error reporting facilities, and even write your own error reports!

Midnote is a terminal application that reads a MIDI file and displays you its notes bar-by-bar, while playing it.

Hatch new projects like a chick coming out of its egg

☁ file.AsyncWrite - because THEY wont do it!!!!!! and its in RUST

A plugin for Jupyter Notebooks that shows you its energy use.

Black-box integration tests for your REST API using the Rust and its test framework

Detects usage of unsafe Rust in a Rust crate and its dependencies.

IDP2P is a peer-to-peer identity protocol which enables a controller to create, manage and share its own proofs as well as did documents

An efficient implementation of Partitioned Label Trees & its variations for extreme multi-label classification

A highly modular Bitcoin Lightning library written in Rust. Its Rust-Lightning, not Rusty's Lightning!

Comments

Refactor

Owner

Jacob Xie

Rust DataFrame library

A Rust DataFrame implementation, built on Apache Arrow

DataFrame / Series data processing in Rust

Provides multiple-dtype columner storage, known as DataFrame in pandas/R

A dataframe manipulation tool inspired by dplyr and powered by polars.

Extra iterator adaptors, iterator methods, free functions, and macros.

Adaptors from AsyncRead/AsyncWrite to Stream/Sink using futures.

Dataframe structure and operations in Rust

Rust DataFrame library

A Rust DataFrame implementation, built on Apache Arrow