Bash, Pipes, and Standard Input

CLI aficionados working on any of the many *nix platforms are likely familiar with at least a few of the types of pipes available for creating powerful one-liners to get things done. One idiom I find particularly interesting is that used by the RVM installation process.

$ bash < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer)

In one line, the script is both downloaded and executed without ever needing to be written to disk.

Recently, Justin Kulesza and I have been working on setting up some new virtual machines using Puppet. Needing to quickly bootstrap the process, installing the Puppet dependencies and setting the system time, we decided to adopt the above idiom for our bootstrapping script.

This worked great until we reached a point in our script where we need to pause and wait for a command to be run manually on the Puppet master (a separate machine). It was then that we realized we needed to make our script interactive.

Let’s take a quick look at the man page for bash:

NAME
bash - GNU Bourne-Again SHell

SYNOPSIS
bash [options] [file]

COPYRIGHT
Bash is Copyright (C) 1989-2005 by the Free Software Foundation, Inc.

DESCRIPTION
Bash is an sh-compatible command language interpreter that executes commands read from the standard input or
from a file. ...

Note: “executes commands read from the standard input or
from a file“.

There are two ways that bash can be invoked to run a script.

You can provide the script to bash via standard input, or you can provide the script’s filename as an argument.

Let’s take another look at that invocation of the RVM installer.

If you account for the redirection, you find that this is the functional equivalent of:

curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer | bash

Now it’s much easier to see the problem.

In this case, we’ve provided the script to bash via standard input.

This means that bash’s standard input is being provided by our script, and cannot be inherited from the parent shell.

What if we were to provide our script’s filename as an argument instead?
Then we could preserve the inherited standard input for later use.

But wait, our script doesn’t have a file name, it’s hosted on GitHub, and the entire point of this one-liner is to skip writing the script to disk.

That’s where process substitution comes in.

All we have to do is drop the extra ‘<' in our invocation and the output of curl will be transformed into a file descriptor. To illustrate: [gist id=1389536] What is /dev/fd/63?

$ cat /dev/fd/63
/bin/cat: /dev/fd/63: Bad file descriptor

If we try to cat it out after the fact, it’s already vanished.

Let’s try something else.

$ cat  <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer)
#!/usr/bin/env bash

shopt -s extglob
PS4="+ \${BASH_SOURCE##\${rvm_path:-}} : \${FUNCNAME[0]:+\${FUNCNAME[0]}()}  \${LINENO} > "
...

Aha! Process substitution is creating for us a temporary pipe that we can use like a file.

This is something we can provide as an argument to bash.

So, if we simply drop the less than character, we will be providing our script to bash as an argument, leaving standard input untouched.

Life is good.