Simple crate that wraps a tokio::process into a tokio::stream

Related tags

Database rust stream process tokio async-stream
Overview

CI coveralls crates.io doc.rs

tokio-process-stream

tokio-process-stream is a simple crate that wraps a tokio::process into a tokio::stream

Having a stream interface to processes is useful when we have multiple sources of data that we want to merge and start processing from a single entry point.

This crate provides a core::stream::Stream wrapper for tokio::process::Child. The main struct is ProcessStream, which implements the trait, yielding one Item enum at a time, each containing one line from either stdout (Item::Stdout) or stderr (Item::Stderr) of the underlying process until it exits. At this point, the stream yields a single Item::Done and finishes.

Example usage:

use tokio_process_stream::ProcessStream;
use tokio::process::Command;
use tokio_stream::StreamExt;
use std::error::Error;

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    let mut sleep_cmd = Command::new("sleep");
    sleep_cmd.args(&["1"]);
    let ls_cmd = Command::new("ls");

    let sleep_procstream = ProcessStream::try_from(sleep_cmd)?;
    let ls_procstream = ProcessStream::try_from(ls_cmd)?;
    let mut procstream = sleep_procstream.merge(ls_procstream);

    while let Some(item) = procstream.next().await {
        println!("{:?}", item);
    }

    Ok(())
}
You might also like...
A Rust application that inserts Discogs data dumps into Postgres

Discogs-load A Rust application that inserts Discogs data dumps into Postgres. Discogs-load uses a simple state machine with the quick-xml Rust librar

Dylan 7 Dec 9, 2022
Persist EWW histories into SQLite
Persist EWW histories into SQLite

EWW History Extension Persist EWW histories into SQLite. Besides EWW, packages below are also supported: elfeed Welcome to open an issue if you have a

null 5 Sep 23, 2022
SubZero - a standalone web server that turns your database directly into a REST/GraphQL api

What is this? This is a demo repository for the new subzero codebase implemented in Rust. subZero is a standalone web server that turns your database

subZero 82 Jan 1, 2023
Unpack a CircleCI orb.yml into separate files compatible with orb packing

CircleCI Orb Unpacking Tool This CLI tool is designed to facilitate the process of migrating a CircleCI Orb to the Orb Development Kit format, where c

Xavientois 2 Jul 29, 2022
Unpack a CircleCI orb.yml into separate files compatible with orb packing

CircleCI Orb Unpacking Tool This CLI tool is designed to facilitate the process of migrating a CircleCI Orb to the Orb Development Kit format, where c

Voiceflow 2 Oct 18, 2022
store virtually unlimited, large files on discord by splitting them into small chunks

Discord Drive This repo hold an MVP implementation of an concept to store data on discord by dividing them into chunks of 7MB each, which are then upl

Piyush Bhangale 5 Feb 1, 2023
A simple key value database for storing simple structures.

Perdia-DB A simple key value database for storing simple structures. No nesting of structures is supported, but may be implemented in the future. Toke

Perdia 4 May 24, 2022
🧰 The Rust SQL Toolkit. An async, pure Rust SQL crate featuring compile-time checked queries without a DSL. Supports PostgreSQL, MySQL, SQLite, and MSSQL.

SQLx 🧰 The Rust SQL Toolkit Install | Usage | Docs Built with ❤️ by The LaunchBadge team SQLx is an async, pure Rust† SQL crate featuring compile-tim

launchbadge 7.6k Dec 31, 2022
A Rust crate for writing servers that speak PostgreSQL's wire protocol

Convergence A Rust crate for writing servers that speak PostgreSQL's wire protocol. Additionally, the experimental convergence-arrow crate enables con

ReservoirDB 63 Jan 2, 2023
Comments
  • Add `ProcessChunkStream`

    Add `ProcessChunkStream`

    • Add ProcessChunkStream for streaming chunks instead of lines.
    • Rework ProcessStream to generic ChildStream<Sout, Serr>.
    • Rename concrete ProcessStream to ProcessLineStream (ProcessStream still works as an alias for backward compatibility).
    • Add TryFrom<&mut Command> for ChildStream.
    • Update dependency required minimums & avoid semver breakage using 0 versions.

    I think this is fully backward compatible, so should be able to publish as 0.1.1.

    Example

    use tokio_process_stream::{Item, ProcessChunkStream};
use tokio::process::Command;
use tokio_stream::StreamExt;

let mut procstream: ProcessChunkStream = Command::new("/bin/sh")
    .arg("-c")
    .arg(r#"printf "1/2"; sleep 0.1; printf "\r2/2 done\n""#)
    .try_into()?;

while let Some(item) = procstream.next().await {
    println!("{:?}", item);
}

    Resolves #1

    opened by alexheretic 2
  • Add support for streaming chunks

    Add support for streaming chunks

    I'd like to listen to ffmpeg output, however it prints over the same line with \r so ProcessStream will not yield this non-line at runtime.

    Having another stream type that returns chunks would allow eager streaming of such output.

    opened by alexheretic 2
Releases(v0.2.0)
Owner
Leandro Lisboa Penz
Programming language enthusiast with Rust and Haskell adventures - and 15+ years of professional embedded C programming with a lot of python.
Leandro Lisboa Penz
GitHub
This is a small demo of how to transform a simple single-server RocksDB service written in Rust into a distributed version using OmniPaxos.

OmniPaxos Demo This is a small demo of how to transform a simple single-server RocksDB service into a distributed version using OmniPaxos. Related res

Harald Ng 6 Jun 28, 2023
a tokio-enabled data store for triple data

terminusdb-store, a tokio-enabled data store for triple data Overview This library implements a way to store triple data - data that consists of a sub

TerminusDB 307 Dec 18, 2022
Asyncronous Rust Mysql driver based on Tokio.

mysql-async Tokio based asynchronous MySql client library for rust programming language. Installation Library hosted on crates.io. [dependencies] mysq

Anatoly I 292 Dec 30, 2022
A tokio-uring backed runtime for Rust

tokio-uring A proof-of-concept runtime backed by io-uring while maintaining compatibility with the Tokio ecosystem. This is a proof of concept and not

Tokio 726 Jan 4, 2023
Incomplete Redis client and server implementation using Tokio - for learning purposes only

mini-redis mini-redis is an incomplete, idiomatic implementation of a Redis client and server built with Tokio. The intent of this project is to provi

Tokio 2.3k Jan 4, 2023
📺 Netflix in Rust/ React-TS/ NextJS, Actix-Web, Async Apollo-GraphQl, Cassandra/ ScyllaDB, Async SQLx, Kafka, Redis, Tokio, Actix, Elasticsearch, Influxdb Iox, Tensorflow, AWS

Fullstack Movie Streaming Platform ?? Netflix in RUST/ NextJS, Actix-Web, Async Apollo-GraphQl, Cassandra/ ScyllaDB, Async SQLx, Spark, Kafka, Redis,

null 34 Apr 17, 2023
AsyncRead/AsyncWrite interface for rustls-on-Tokio

rustls-tokio-stream rustls-tokio-stream is a Rust crate that provides an AsyncRead/AsyncWrite interface for rustls. Examples Create a server and clien

Deno 7 May 17, 2023
Awesome books, tutorials, courses, and resources for the Tokio asynchronous runtime ecosystem. ⚡

Awesome Tokio Tokio is an asynchronous runtime for the Rust programming language. It provides the building blocks needed for writing network applicati

Marcus Cvjeticanin 59 Oct 27, 2023
Fault-tolerant Async Actors Built on Tokio

Kameo ???? Fault-tolerant Async Actors Built on Tokio Async: Built on tokio, actors run asyncronously in their own isolated spawned tasks. Supervision

Ari Seyhun 135 Jul 25, 2024
🐸Slippi DB ingests Slippi replays and puts the data into a SQLite database for easier parsing.

The primary goal of this project is to make it easier to analyze large amounts of Slippi data. Its end goal is to create something similar to Ballchasing.com but for Melee.

Max Timkovich 20 Jan 2, 2023