Wednesday, July 23, 2008

Simple Multiple-Stream Output Redirection With Zsh On Linux and Unix

Hey there,

Following up on yesterday's post regarding using zsh's extended globbing functionality, today we're going to look at another feature supported by zsh, that can't be done nearly as easily in most other shells. Today we're going to look at some unique ways you can manipulate input and output redirection, even with multiple streams, while avoided a lot of extra code, and maybe a temp file or three :) This stuff is actually quite cool and I still count myself among the surprised that these things don't work in bash or ksh. The end objectives can be accomplished by all of the shells, of course, but we're looking for the most direct approach.

Again, these test were done on bash, ksh and zsh on Solaris Unix and RedHat Linux, so your results may vary. They probably won't though, because I couldn't find any version of bash or ksh that can do most of this stuff (see bullet point 4, later... :) So, without any further fanfare, let's get on with the zsh input/output redirection enhancements!

1. Redirecting output to two or more files on the command line: Generally, in bash and ksh (heretofore assumed to be the "other" shells), you can redirect output to multiple files using a variety of different methods. However, if you limit yourself to the standard output redirect operator, the following would not work (the output being limited to the last redirection to STDIN, since they just overwrite each other in succession):

bashksh # echo hi >file1 >file2
bashksh # cat file1
bashksh # cat file2
hi


although it will work in zsh, and for more than just two files (pick a number... I'm not sure how high you can go with this):

zsh # echo hi >file1 >file2 >file3
zsh # cat file1
hi
zsh # cat file2
hi
zsh # cat file3
hi


2. Redirecting output from two or more files on the command line: The same rules that applied to example 1, work the same way in reverse (with the each instance of redirection from STDOUT overwriting the previous). For instance, you'll get this output from bash and ksh:

bashksh # cat <file1 <file2
this is the content in file2


while in zsh, you get this:

zsh # cat <file1 <file2 <file3
this is the content in file1
this is the content in file2
this is the content in file3


3. Using a pipe to "cat," instead of the "tee" command, if you want to pump output to the screen while running an operation on a file. In other shells, you would normally use "tee" to capture output and watch as a process ran through it's paces. That would work great, but trying to do it with a pipe and "cat" would fail:

bashksh # pwd |tee filename
/home/techserv/mg131
bashksh # cat filename
/home/techserv/mg131
bashksh # pwd >filename|cat
bashksh # cat filename
/home/techserv/mg131


Note that, above, the "cat" command doesn't spit any output to the screen during the process' run; only to the file. In zsh, you can use the pipe to "cat" (shown above) as a substitute for "tee":

zsh # pwd >filename|cat
/home/techserv/mg131
zsh # cat filename
/home/techserv/mg131


4. And, lastly, we'll take a look at the redirection of command output (within parentheses) and go straight to the complicated stuff. This is actually listed as a feature of zsh (and, technically, it is), but my tests with ksh and bash have produced equal results, so it doesn't fit my definition of a "feature" of zsh (that being something unique to zsh, and not directly reproducible in other shells). For instance, this line returns the exact same output in all three shells:

bashkshzsh # sort <(grep -v "^#" <(paste -d: <(cut -d: -f1 /etc/passwd) <(cut -d: -f3 /etc/passwd) ) ) |egrep ':0|:1|:2|:3'
bin:2
daemon:1
listen:37
root:0
sys:3


While this is definitely cool stuff (And the redirection of lists generated by commands within parentheses can be used as multiple streams of input and output for all three shells, as well), my conclusion, to date, is that most advanced shells have this functionality built in. If you "must" see it fail, you can always rely on straight-up vanilla Bourne shell:

sh # sort <(grep -v "^#" <(paste -d: <(cut -d: -f1 /etc/passwd) <(cut -d: -f3 /etc/passwd) ) ) |egrep ':0|:1|:2|:3'
syntax error: `(' unexpected
sh # bin:2
bin:2: not found
sh # daemon:1
daemon:1: not found
sh # listen:37
listen:37: not found
sh # root:0
root:0: not found
sh # sys:3
sys:3: not found


Now, if bash and ksh had come back with that, I would have no reservations about labeling this type of multiple-stream output redirection and massaging a feature of zsh.

Yes, the ending to this post "was" anti-climactic, but at least you can be reasonably sure I'm remaining objective ;)

Cheers,

, Mike