Follow up to https://github.com/oconnor663/shared_child.rs/pull/13

I have not added tests or verified that this works as intended yet. But I wanted to get the PR in place so the solution and design could be discussed.

Using the newest duct.rs release and the newest error-chain I get a compile error when trying to link the duct's error type.

Cargo.toml:

[package]
name = "error-chain-failure"
version = "0.1.0"

[dependencies]
error-chain = "0.8.1"
duct = "0.6.0"

src/lib.rs:

#[macro_use]
extern crate duct;
#[macro_use]
extern crate error_chain;

error_chain!{
    links {
        ShellError(duct::Error, duct::ErrorKind);
    }
}

Compiler output:

   Compiling error-chain-failure v0.1.0 (file:///.../error-chain-failure)
error[E0308]: mismatched types
  --> src/lib.rs:6:1
   |
6  |   error_chain!{
   |  _^ starting here...
7  | | 	links {
8  | | 		ShellError(duct::Error, duct::ErrorKind);
9  | | 	}
10 | | }
   | |_^ ...ending here: expected struct `error_chain::State`, found a different struct `error_chain::State`
   |
   = note: expected type `error_chain::State` (struct `error_chain::State`)
   = note:    found type `error_chain::State` (struct `error_chain::State`)
note: Perhaps two different versions of crate `error_chain` are being used?
  --> src/lib.rs:6:1
   |
6  |   error_chain!{
   |  _^ starting here...
7  | | 	links {
8  | | 		ShellError(duct::Error, duct::ErrorKind);
9  | | 	}
10 | | }
   | |_^ ...ending here
   = note: this error originates in a macro outside of the current crate

error: aborting due to previous error

error: Could not compile `error-chain-failure`.

Updating the dependencies fixes this error.

Would it be possible to implement something like tail -f on the output of running a command? Currently, I believe the entire stdout and stderr output are buffered until the command completes. For long-running commands that incrementally produce output, it would be useful to see the output as it is produced.

I understand this may be too complex and/or beyond the scope of duct, but I think it is a useful feature to consider.

See #55 for context.

I went ahead and implemented a simple simple support for changing uid/gid.

This should not be considered done as it is -- at the very least it's missing documentation and possibly also some tests (I haven't looked at how duct is tested yet). I mostly made it to see how easy it would be to add the functionality and to give us something concrete to discuss.

The main open question is whether to do this as unix::ExpressionExt or just add conditional functions on Expression itself. I'm leaning towards keeping it as it is, but let me know what you think.

If you like it, I will go ahead an make this a more finished PR.

Probably start will need to keep more data around. We might need another type of explicit tree to mirror the expression tree, which knows how to wait and kill its children, similar to Rust's stdlib.

Hi!

A common usability issue with constructing a command line is that each argument must be a separate string literal:

cmd!("git", "symbolic-ref", "--short", "HEAD")

I think a nice qol improvement would be to automatically split the first argument of cmd! on whitespace, so the same could be written more compactly as

cmd!("git symbolic-ref --short HEAD")

Note that this allows one to supply additional arguments just fine:

cmd!("git symbolic-ref --short", commit_hash)

Does this look like a useful feature to have?

I use your amazing libraries on the tauri repo and I'm having an issue because I set both stderr and stdout pipes, and I lose some stdout information in this case.

The command I'm running for this test is curl --progress-bar http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_30fps_normal.mp4 -o file.mp4. Right now I'm using shared_child+os_pipe, but I also tried using duct directly but I didn't fix this problem for me.

Snippet using shared_child+os_pipe:

use std::process::Command;
use os_pipe::pipe;
use shared_child::SharedChild;

let mut command = std::process::Command::new("curl");
command.args(["--progress-bar", "http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_30fps_normal.mp4", "-o", "test.mp4"]);
let (stdout_reader, stdout_writer) = pipe()?;
let (stderr_reader, stderr_writer) = pipe()?;
let (stdin_reader, stdin_writer) = pipe()?;
command.stdout(stdout_writer);
command.stderr(stderr_writer);
command.stdin(stdin_reader);

let shared_child = SharedChild::spawn(&mut command)?;

std::thread::spawn(move || {
  let reader = BufReader::new(stdout_reader);
  for line in reader.lines() {
    println!("stdout: {:?}", line);
  }
});

std::thread::spawn(move || {
  let reader = BufReader::new(stderr_reader);
  for line in reader.lines() {
    println!("stderr: {:?}", line);
  }
});

std::thread::spawn(move || {
  let _ = match shared_child.wait() {
    Ok(status) => {
      println!("finished with status {:?}", status);
    }
    Err(e) => {
      println!("error: {}", e);
    }
  };
});

Using duct:

use duct::cmd;
use os_pipe::pipe;

let (stdout_reader, stdout_writer) = pipe()?;
let (stderr_reader, stderr_writer) = pipe()?;
let (stdin_reader, stdin_writer) = pipe()?;

let mut expression = cmd(self.program, self.args);
expression = expression.stdout_file(stdout_writer);
expression = expression.stderr_file(stderr_writer);
expression = expression.stdin_file(stdin_reader);

let handle = expression.start()?;

std::thread::spawn(move || {
  let reader = BufReader::new(stdout_reader);
  for line in reader.lines() {
    println!("stdout: {:?}", line);
  }
});

std::thread::spawn(move || {
  let reader = BufReader::new(stderr_reader);
  for line in reader.lines() {
    println!("stderr: {:?}", line);
  }
});

std::thread::spawn(move || {
  let _ = match handle.wait() {
    Ok(status) => {
      println!("finished with status {:?}", status);
    }
    Err(e) => {
      println!("error: {}", e);
    }
  };
});

Removing command.stderr() and expression.stderr_file() calls fixes the problem and I get the stdout messages again.

Duct provides methods like Expression::stdin_file and Expression::stdout_file to use some existing file for stdin and stdout (i.e. file descriptors 0 and 1), but there doesn't seem to be any way to give a child process an arbitrary set of file descriptors. It would be useful to be able to run a child process with an arbitrary set of file descriptors, like you can from the shell e.g.

./command 5<somefile.txt 6>someotherfile.txt 7<>athirdfile.txt 8<&4

This is useful because it allows the parent process to give the child access to a file that the child doesn't have permission to open itself.

I'm trying to invoke a command of the following format:

$ foo --long-arg='A','B'

But when I try to do something like this:

cmd("foo", vec!["--long-arg='A','B','C'"])

It seems what actually gets executed is:

$ foo --long-arg=\'A\',\'B\',\'C\'

Where are these escape symbols coming from?

Would be nice to be able to directly write to a buffer of bytes or something similar (to prevent lots of allocation if you keep repeating a command).

(great crate by the way :), found it when I was trying to get process::Command to redirect stderr to stdout)

In order to fully leverage the power of error chain it would be nice to make the error module public. This way, chaining errors is possible from other crates. I need this functionality for better error handling inside a new tool that depends on duct.rs.

For more information, see https://github.com/brson/error-chain/issues/120

This adds an equivalent stderr_reader method analogous to reader. I have a case where I want to read stderr continuously since it usually shows progress output from commands like curl and git, while I only want the stdout output when the command has already finished.

I want to do the shell equivalent of mycommand 2>&1 > /tmp/output.log with duct, and I accomplish this with:

  duct::cmd(program, args)
    .stderr_to_stdout()
    .unchecked()
    .stdout_file(logf)
    .run()

This works great, except that when I run it in a Windows service, I get an "invalid handle" error. I figure this stems from Windows services not having standard input or outputs.

However, I wonder if it's a fundamental limitation or if it can work? If I run std::process::Command and use the .output() method I can capture all the stdout and stderr and write them to a file, but I really would like to "preserve order" of the output, as is done when redirecting stderr to stdout.

This can be done through use of unchecked today, but it would be great to have an ergonomic way to do so given how much duct is built around making the correct thing easier to do.

The main challenge is that wait() etc return a &Output, while Rust errors are generally 'static. stdout and stderr are unbounded so if we clone data, it can be quite expensive.

Some thoughts:

• allow opting into printing out stdout/stderr, have it turned off by default
• as part of opting in, specify a beginning and end length, truncating the output in the middle
• the data is UTF-8 in many cases, but it might not be, have some handling around that:
  • truncation will need special handling around multibyte UTF-8 sequences given that the main purpose is to print out data
  • assume that if someone's asking for output to be returned, they generally expect it to be UTF-8 and just use String::from_utf8_lossy while printing the data out
• use a custom error type for clients that care about it, and add a conversion to io::Error through a From impl for compatibility

1. ffmpeg -list_devices true -f dshow -i dummy always returns 1, even though it prints the desired output: https://trac.ffmpeg.org/wiki/DirectShow

I'm calling duct like this:

let output = cmd!("ffmpeg", "-list_devices", "true", "-f", "dshow", "-i", "dummy")
	.stdout_stderr_swap() // temp workaround (see below)
	.read()?;

duct fails with:

Error:
    command ["ffmpeg", "-list_devices", "true", "-f", "dshow", "-i", "dummy"] exited with code 1

Thus not returning the output that I want to parse.

How can I get the output, regardless of exit status?

2. Btw, ffmpeg prints the output in stderr always, to keep stdout for piping video out. (It prints errors to ffmpeg.log). What's the closest to a .read() equivalent for stderr? :)

Tokio has a Command struct which provides async command handling, using signals on Unix and native APIs on Windows. Wondering how hard it would be to make duct support tokio commands in addition to std::process::Command instances.

Currently, duct's Handle method appears to provide two methods:

1. wait(), which waits for process exit and for output handles (stdout/stderr) to be closed, and
2. try_wait(), which checks if the process has exited and:

• if it hasn't, returns Ok(None)
• if it has exited, calls wait() and blocks on handles being closed.

I have a use case for a wait() which does not want to wait on handles being closed: specifically, I'm writing a test runner which wants to be able to detect leaked processes and fail a test if the process has exited without handles being closed.

Would it make sense to support that in duct? I'm not exactly sure what a reasonable API for this is. Grandchild handling is so messy on Unix anyway.