Simple crate that wraps a tokio::process into a tokio::stream

Last update: Sep 13, 2022

Overview

tokio-process-stream

tokio-process-stream is a simple crate that wraps a tokio::process into a tokio::stream

Having a stream interface to processes is useful when we have multiple sources of data that we want to merge and start processing from a single entry point.

This crate provides a core::stream::Stream wrapper for tokio::process::Child. The main struct is ProcessStream, which implements the trait, yielding one Item enum at a time, each containing one line from either stdout (Item::Stdout) or stderr (Item::Stderr) of the underlying process until it exits. At this point, the stream yields a single Item::Done and finishes.

Example usage:

use tokio_process_stream::ProcessStream;
use tokio::process::Command;
use tokio_stream::StreamExt;
use std::error::Error;

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    let mut sleep_cmd = Command::new("sleep");
    sleep_cmd.args(&["1"]);
    let ls_cmd = Command::new("ls");

    let sleep_procstream = ProcessStream::try_from(sleep_cmd)?;
    let ls_procstream = ProcessStream::try_from(ls_cmd)?;
    let mut procstream = sleep_procstream.merge(ls_procstream);

    while let Some(item) = procstream.next().await {
        println!("{:?}", item);
    }

    Ok(())
}

You might also like...

Persist EWW histories into SQLite

EWW History Extension Persist EWW histories into SQLite. Besides EWW, packages below are also supported: elfeed Welcome to open an issue if you have a

5 Sep 23, 2022

SubZero - a standalone web server that turns your database directly into a REST/GraphQL api

What is this? This is a demo repository for the new subzero codebase implemented in Rust. subZero is a standalone web server that turns your database

82 Jan 1, 2023

Unpack a CircleCI orb.yml into separate files compatible with orb packing

CircleCI Orb Unpacking Tool This CLI tool is designed to facilitate the process of migrating a CircleCI Orb to the Orb Development Kit format, where c

2 Jul 29, 2022

Unpack a CircleCI orb.yml into separate files compatible with orb packing

CircleCI Orb Unpacking Tool This CLI tool is designed to facilitate the process of migrating a CircleCI Orb to the Orb Development Kit format, where c

2 Oct 18, 2022

store virtually unlimited, large files on discord by splitting them into small chunks

Discord Drive This repo hold an MVP implementation of an concept to store data on discord by dividing them into chunks of 7MB each, which are then upl

5 Feb 1, 2023

A simple key value database for storing simple structures.

Perdia-DB A simple key value database for storing simple structures. No nesting of structures is supported, but may be implemented in the future. Toke

4 May 24, 2022

🧰 The Rust SQL Toolkit. An async, pure Rust SQL crate featuring compile-time checked queries without a DSL. Supports PostgreSQL, MySQL, SQLite, and MSSQL.

SQLx 🧰 The Rust SQL Toolkit Install | Usage | Docs Built with ❤️ by The LaunchBadge team SQLx is an async, pure Rust† SQL crate featuring compile-tim

7.6k Dec 31, 2022

A Rust crate for writing servers that speak PostgreSQL's wire protocol

Convergence A Rust crate for writing servers that speak PostgreSQL's wire protocol. Additionally, the experimental convergence-arrow crate enables con

63 Jan 2, 2023

This is a maintained rust project that exposes the cpp driver at cpp-driver in a somewhat-sane crate.

cassandra-rs This is a maintained rust project that exposes the cpp driver at https://github.com/datastax/cpp-driver/ in a somewhat-sane crate. For th

51 Aug 20, 2020

Comments

Add `ProcessChunkStream`
Add ProcessChunkStream for streaming chunks instead of lines.

Rework ProcessStream to generic ChildStream<Sout, Serr>.

Rename concrete ProcessStream to ProcessLineStream (ProcessStream still works as an alias for backward compatibility).

Add TryFrom<&mut Command> for ChildStream.

Update dependency required minimums & avoid semver breakage using 0 versions.

I think this is fully backward compatible, so should be able to publish as 0.1.1.

Example

use tokio_process_stream::{Item, ProcessChunkStream}; use tokio::process::Command; use tokio_stream::StreamExt; let mut procstream: ProcessChunkStream = Command::new("/bin/sh") .arg("-c") .arg(r#"printf "1/2"; sleep 0.1; printf "\r2/2 done\n""#) .try_into()?; while let Some(item) = procstream.next().await { println!("{:?}", item); }

Resolves #1
opened by alexheretic 2
Add support for streaming chunks

I'd like to listen to ffmpeg output, however it prints over the same line with \r so ProcessStream will not yield this non-line at runtime.

Having another stream type that returns chunks would allow eager streaming of such output.

opened by alexheretic 2

Releases(v0.2.0)

v0.2.0(Jan 29, 2022)

Also: no more busy wait
Source code(tar.gz)
Source code(zip)

Simple crate that wraps a tokio::process into a tokio::stream

Related tags

Overview

tokio-process-stream

You might also like...

Persist EWW histories into SQLite

SubZero - a standalone web server that turns your database directly into a REST/GraphQL api

Unpack a CircleCI orb.yml into separate files compatible with orb packing

Unpack a CircleCI orb.yml into separate files compatible with orb packing

store virtually unlimited, large files on discord by splitting them into small chunks

A simple key value database for storing simple structures.

🧰 The Rust SQL Toolkit. An async, pure Rust SQL crate featuring compile-time checked queries without a DSL. Supports PostgreSQL, MySQL, SQLite, and MSSQL.

A Rust crate for writing servers that speak PostgreSQL's wire protocol

This is a maintained rust project that exposes the cpp driver at cpp-driver in a somewhat-sane crate.

Comments

Add `ProcessChunkStream`

Example

Add support for streaming chunks

Releases(v0.2.0)

v0.2.0(Jan 29, 2022)

Owner

Leandro Lisboa Penz

This is a small demo of how to transform a simple single-server RocksDB service written in Rust into a distributed version using OmniPaxos.

a tokio-enabled data store for triple data

Asyncronous Rust Mysql driver based on Tokio.

A tokio-uring backed runtime for Rust

Incomplete Redis client and server implementation using Tokio - for learning purposes only

📺 Netflix in Rust/ React-TS/ NextJS, Actix-Web, Async Apollo-GraphQl, Cassandra/ ScyllaDB, Async SQLx, Kafka, Redis, Tokio, Actix, Elasticsearch, Influxdb Iox, Tensorflow, AWS

AsyncRead/AsyncWrite interface for rustls-on-Tokio

Awesome books, tutorials, courses, and resources for the Tokio asynchronous runtime ecosystem. ⚡

🐸Slippi DB ingests Slippi replays and puts the data into a SQLite database for easier parsing.

A Rust application that inserts Discogs data dumps into Postgres