Using Bash To Access The Network Via File Descriptors

Greetings,

It's been a while since we looked at down-and-dirty shell tricks, and this one comes as a tangential follow-up to our previous post on finding and reading files in the shell when your system is on the verge of a crash.

The first thing I'd like to clear up before we proceed is that, in re-reading that post, I notice that I was being a bit of a ksh-snob in the section regarding reading the contents of a regular file through a file descriptor to save on resource usage. My example line in that post, after exec'ing file descriptor 7, was:

host # while read -u7 line; do echo $line; done

And, as most bash fans probably noted, that syntax doesn't work in bash. That's what I get for only verifying the output in ksh ;) What that line should have read (since this will work in sh, ksh and bash) was:

host # while read line;do echo $line;done <&7

Apologies all around. In any event, that "mistake" serves as a fairly decent segue into what we're going to be looking at today. That same basic process of exec'ing file descriptors to read files directly is how we're going to use bash to read from the network directly using /dev/tcp.

And, just to be clear, this "does not" work in Solaris ksh or sh. It will work on Solaris and every Linux I've tested, but only, to my knowledge, in bash. Every version of ksh I've tested (even Solaris 10) can't create the virtual file system required by this operation. I've attached a small bash script, at the end of the post, to check a mail server and http server, so you have a working example of the process.

This method of accessing the network is very useful to know how to do and, going through it step by step, is actually very simple to accomplish. We'll use a mail server as our example since it involves both reading and writing to the network file descriptor.

The first thing you'll want to do is "exec" the file descriptor you'll be using to communicate on the network via tcp. You can do this like so (Just be sure not to exec file descriptors 0, 1 or 2, as these should already be assigned by your operating system as "Standard Input (STDIN)," "Standard Output (STDOUT)," and "Standard Error (STDERR)" :

host # exec 9<>/dev/tcp/mail_server.xyz.com/25 <-- Here we create file descriptor 9 via exec, open the /dev/tcp/mail_server.xyz.com/25 "virtual file" and assign file descriptor 9 to it.

Note that this line is where the script, or process, will either make or break. ksh (Definitely for Solaris 9 and 10) does not allow the creation of the virtual filesystem below /dev/tcp that this operation requires. If you attempt to do this in ksh you will most likely get the following error:

host # /dev/tcp/mail_server.xyz.com/25: cannot create

Assuming we're using bash, and that worked okay, we can now write to, and read from, our mailserver over port 25. This is analogous to doing something like Telnetting to port 25, but we're using a built in feature of the bash shell to access the network via a pseudo file system created under /dev/tcp (It acts much like a /proc filesystem, although you can't see what you've created by looking at the file that /dev/tcp links to, since it's a character device file and the directory path /dev/tcp/mail_server.xyz.com/25 doesn't "really" exist :)

Now we can send input, and receive output, using standard file descriptor redirection commands. A few "for instances" below:

host # echo "HELO xyz.com" >&9 <--- This sends the HELO string to our mailserver via file descriptor 9, which we exec'ed above.
host # read -r RESPONSE <&9 <--- This reads the data coming in on file descriptor 9 with as little interpretation as possible (for instance, it treats the backslash (\) as a backslash and doesn't imbue it with it's usual "special" shell powers).
host # echo -n "The mail server responded to our greeting with this load of sass: "
host # echo $RESPONSE

And, now, we can write to, and read from, file descriptor 9 until we have nothing left to say ;) When we're all finished, I find it's good practice to close the input and output channels for our file descriptor (although closing the input should be sufficient in most cases), and, if you want to or need to, you can also dump the remainder of the file descriptor's contents before you close it down.

host # cat <&9 2>&1 <--- Dump all of the remaining input/output left on file descriptor 9 to our screen.
host # 9>&- <--- Close the output file descriptor
host # 9<&- <--- Close the input file descriptor

Now, you're back to the way things were and you shouldn't be able to read or write from file descriptor 9 anymore (or, if you're a glass-half-full person, it's now available for use again :) I've tacked on a small script to demonstrate some basic functionality, but, hopefully, you'll have some fun playing around with this and figure out more, and better, ways to manipulate your server's interaction with the network through the file system.

host # ./check_net.sh <--- As simple as that to run it :)

Cheers,





This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/bash
#
# check_net.sh
#
# 2008 - Mike Golvach - eggi@comcast.net
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

mailserver="mailserver.xyz.com"
httpserver="www.xyz.com"
domain="xyz.com"

echo "Testing mail server functionality"
exec 9<>/dev/tcp/$mailserver/25
read -r server_version <&9
echo "Server reports it is: $server_version"
echo "HELO $domain" >&9
read -r greeting <&9
echo "Server responded to our hello with: $greeting"
echo "VRFY username" >&9
read -r vrfy_ok <&9
echo "Server indicates that this is how it feels about the VRFY command: $vrfy_ok"
echo "quit" >&9
read -r salutation <&9
echo "Server signed off with: $salutation"
echo "Dumping any remaining data in the file descriptor"
cat <&9 2>&1
echo "Closing input and output channels for the file descriptor"
9>&-
9<&-
echo "--------------------------------------------------"
echo "Testing web server functionality - Here it comes..."
exec 9<>/dev/tcp/$httpserver/80
echo "GET / HTTP/1.0" >&9
echo "" >&9
while read line
do
        echo "$line"
done <&9
echo "Dumping any remaining data in the file descriptor"
cat <&9 2>&1
echo "Closing input and output channels for the file descriptor"
9>&-
9<&-
echo "done"

, Mike

linux unix internet technology

A Simple Trick To Keep Your SSH Session From Timing Out

Hey there,

I thought I'd write a simple something this Saturday; just a little trick that's served me well over the years.

This has to do with any login session in a terminal window (Telnet, SSH, etc). Although this may not always be the case, most places I've ever worked at have had, at least, a few machines were they enforced time-outs on your login. This can be frustrating if you're in the middle of doing something, get called away to do something else, and come back to a "Disconnected!" dialogue box.

Usually, you should be able to get away with just putting a line like this in your .profile or .bash_profile:

TMOUT=0;export TMOUT

But, I've found that a lot of setups don't honor this shell setting. Instead, they take a measure of your activity and log you out if you don't produce enough in your session. So, when you type:

sqlplus @database_query

Even if that takes all day to run, you'll still get disconnected in 5 minutes.

This simple shell loop has almost always worked for me:

while :;do print -n "* ";sleep 15;done

Substitute either
echo -n "* "
or
echo "*\c" <--- \c is the escape character representation of a space.

for the print statement in that little one-line loop, depending on your shell (sh doesn't support the "print" statement) and its implementation of echo (you might be using the built-in or the system binary; in any event - one of these three varieties should do the trick for you.

I leave the carriage-return out on purpose so I can come back later and look at:

*******_

instead of pages of single *'s along the side of my screen, forcing me to scroll back forever to remember what I was doing ;)

This trick basically works because, even though you're not directly interacting with your terminal, you're sending packets back and forth every time that print (or echo) statement executes.

Enjoy a little less stress. Your terminal windows should now be waiting for you instead of threatening to leave :)

, Mike

linux unix internet technology

Only Printing Out Matches With Perl Regular Expressions

Hey there,

This will deal with something every shell/perl scripter has to deal with from time to time. How do I extract "whatever" from this file and just output my match?

We'll assume you have a logfile that consists of lines like this:

<td class="prompt" height="15" width="30%"><span><td class="prompt" height="15" width="30%">Text-Text_Text</td>
<td class="tablecontent" width="70%">09/23/08 </td></span>

and you just want to get the date.

It's relatively easy to do (although this case is far from challenging since the date format is so different than the rest of the lines).

First you need to figure out what you're not going to print out. So, after you read the file into an array and are iterating through it (or however you prefer to process your files with Perl), you'll first want to put this line in your loop.

This will make sure that you don't bother printing out any lines that don't contain your match:

next if $match !~ /\d\d\/\d\d\/\d\d/;

This says don't do anything and continue to the next line if the pattern "\d(Any Decimal Character)\d\/(A forward slash - backslashed, since it's a special character)\d\d\/\d\d" isn't on the line.

Then you'll want to print out only the date for all the lines that contain it, with this following line:

$match =~ s/^.* (\d\d\/\d\d\/\d\d) .*$/$1/; print $match

Note that the $1 on the right hand side of the expression represents everything we've captured in between parentheses on the left hand side of the expression. We've basically substituted the entire line with the date :) And just so we see some output, we print the contents of $match (the date).

Simple enough :)

, Mike

linux unix internet technology