More Fun With Bash Networking

Greetings,

You may recall a post we did about a week or so ago on accessing the network directly with bash. Today's post is a bit of a follow up to that introduction, with some tips and clarification added for clarity and/or enjoyment ;)

1. The basic setup (this will be the only part revisited from our original post on bash networking. In order to get any of this started, you'll need to exec a file descriptor in bash. The file descriptor can, theoretically, be any number, but you should avoid 0, 1 and 2, as these are generally reserved by most Unix or Linux shells for standard input, standard output and standard error I/O. To exploit bash's "virtual" /dev/tcp filesystem (note that you can use /dev/udp as well; it just depends what you're planning on doing - if you wanted to interact with an old NIS server, you might "need" to use the udp protocol), the setup is simple enough. Assuming we wanted to use file descriptor 9 to hit a google web server on port 80, this is what we'd type on the command line:

host # exec 9<>/dev/tcp/www.google.com/80 <--- We've now connected to www.google.com.

2. The rest of it depends on what you want to do. The most basic operation would be to send information to the new file descriptor, read information from it and close it. Those three things can be done like this:

host # echo -e "GET /search?q=linux+unix+menagerie HTTP/1.0\n\n" >&9 <--- And now, we've requested a search page
host # while read line <&9
do
echo -n $line >&1
done <--- Now, we read the HTML page that gets returned.
host # exec 9<&- <--- This closes the standard input for our new file descriptor
host # exec 9>&- <--- and this closes the standard output

3. Using HTTP 1.1 instead of 1.0. This is actually quite easily done (and you can do it through Telnet to port 80 on any given host, as well. Telnet will allow you to connect to any specified port on a host and interact with it. You just need to know what to type ;). It just requires a bit more typing on our part. Assuming we have the initial file descriptor set up, per step 1, we would do this to initiate an HTTP 1.1 connection and read from it.

host # echo -e "GET /search?q=linux+unix+menagerie HTTP/1.1" >&9 <--- Note the lack of the double carriage return. This isn't absolutely necessary, but if those returns are typed, they obviate the next line.
host # echo -e "Host: www.google.com\nConnection: close\n" >&9 <--- This line tells HTTP/1.1 that we want to close the connection once our query, or page request, completes. HTTP 1.1 will keep the connection open until it times out unless you do this, while HTTP 1.0 will close it immediately after you send one request.

And then you can go ahead and read the response and close the file descriptors in the same manner as above.

4. Implementing host headers. You may find that more and more web servers require "host headers" when you send them an HTTP 1.1 request. These, technically are only required for virtual hosting, but it's becoming more and more common for out-of-the-box webservers to run the main server as a virtual host. It's easy to send a request to our file descriptor in this manner, as well. We just need to modify it slightly.

host # echo -e "GET /search?q=linux+unix+menagerie HTTP/1.1\nhost: http://www.google.com\n\n" >&9

5. Sending a POST request rather than the standard GET. This again, is just a modification on the main method. All you need to tell the server is that you're performing a POST action and send it the POST data (This is basically how the folks in security check to see if they can mess with your cgi forms):

host # echo -e "POST /search?q=linux+unix+menagerie HTTP/1.1\n\nHere Are All My POST Variable=Value Pairs" >&9

6. Ignoring the header information when requesting an HTML page using bash networking through file descriptors. This is kind of "voodoo," but is almost 100% guaranteed to work. The first blank line you encounter when you read your response to the query you send to a web server is (almost) always the end of the header section. The header section includes varying information on web server type, version, age, etc, but isn't worth reading if you just want the web page back. You can avoid looking at it by running this after you make the initial GET request:

host # while read <&9
do
 line=${response//$'\r'/}
 if [ -z "$line" ]
 then
  break
 fi
done

And then you can move on and continue reading from the file descriptor, and closing it, just like in step 2:

host # while read response <&9
do
 echo -n $response >&1
done
exec 9<&-
exec 9>&-

7. What you get if you settle. Interestingly enough, bash's "virtual" tcp (or udp) filesystem defaults to connecting to a webserver. If you simply exec a new file descriptor without any arguments, the "virtual" /dev/tcp, and /dev/udp, file system is still created. It defaults to port 80 and the tcp protocol. In fact, the virtual file system "always" exists. Why? Because your shell automatically opens file descriptors 0, 1 and 2 when it initializes. So, basically, when you login, you've already got a path to the net :) How to use it meaningfully is another thing entirely, but you can do this right out of the gate, just after logging in:

host # echo -n "GET / HTTP/1.0\n\n" >/dev/tcp/www.google.com/80

although, thankfully, it doesn't work all that simply and involves serious risk.

Exec'ing an additional file descriptor is a better practice when doing this sort of thing, because you don't want to muck with the 3 basic ones. If you make a mistake on file descriptor 9, for example, it's no big deal if it gets disconnected or something else weird happens to it. If file descriptor 0 get whacked, you won't be able to type to the terminal, and if file descriptors 1 or 2 get destroyed, you won't be able to see some, or all, of the output the shell is sending to your terminal. In either case, you risk losing your connection to your terminal session entirely.

Again, you can use bash's "virtual" networking to connect to any port that already has a service listening on it. Unfortunately, as far as I know, at this time, there is no way to "initiate" (or bind to) a network port using this method and the bash shell (or any Unix or Linux shell, for that matter). If it ever becomes possible, I'll be sure to post about it, because it will open up a whole new can of worms for all of us ;)

Again, if you would like further basic explanation (and a sample script) regarding setting up bash's "virtual" network connections, see our previous post on bash file descriptor networking.

Cheers, and enjoy!

, Mike

linux unix internet technology