Thursday, April 16, 2009

Beginner's Shell Scripting On Linux And Unix: Why And How

Hey There,

Recently I was asked a very simple question that I'm pretty sure I've never addressed in this blog over the mass of posts already churned out. Shell scripting has been addressed, to a great degree, on this blog, but (even the beginner stuff) has included a presumption of certain knowledge. It's easy to forget what it was like back when I started working on Linux and Unix, so I often make the cardinal sin of assuming everyone has already asked this question and had it answered.

The question was simple (although, a three-parter) : I know how to use Unix/Linux enough to get around on the command line, but why would I want to write shell scripts? And how do I write a shell script, anyway? Isn't it just going to be more hassle than I already have to deal with?

The above was only-slightly paraphrased ;)

So, even though I could dump these questions, and their answers, off by conveniently linking to any number of other sites, I thought it might be a good idea to just answer them here. I've never posted on the subject before and it can't hurt to have this basic information available in our site search.

IMPORTANT NOTE FOR LINUX/UNIX USER'S AND SHELL SCRIPTERS WHO CAN ANSWER ALL THREE QUESTIONS: This post is probably going to be very boring for you ;) I'll do my best to see that it doesn't come off too dry, but I won't be writing about anything you don't know already, and I will be purposefully leaving out certain things that may drive you crazy ;)

So, to attack the question, I've decided to respond in two parts (since the final two naturally flow together) and keep this post as simple as possible. My feeling is that if you're reading this and aren't sure what shell scripting is and/or why you should even bother to make use of it, you're not looking to be weighed down by a ton of technical jargon and over-explanation which would probably just cloud the issue and be more difficult to understand than necessary. Here we go :)

Q: Why would I want to write shell scripts? Isn't it just going to be more hassle than I already have to deal with?

A: To answer the question "Why?," the most succinct answer would be "Why Not?" There are several good reasons to write shell scripts (for your own use or for the use of others), one of which addresses the "hassle" issue ;)

1. Shell scripting, at its most basic, is actually very easy to master (assuming that you can get around well enough on the Unix or Linux command line to accomplish whatever it is you need to do). In fact, your knowledge of the command line will be almost-directly commensurate with your ability to write shell scripts of varying complexity.

2. Writing a shell script is only slightly (and I mean "very" slightly) more of a hassle than doing the exact same thing on the command line. It can, very literally, be thought of as a way of saving your command line work so that you never have to worry about remembering it ever again.

3. If you've written a shell script once, you never have to write it again (unless you want to improve, or modify, it - well, you can have your own reasons. It's none of my business, anyway ;). If you hammer out a command line to get your chores done (assuming you don't have one of many other methods of saving command line history enabled), you're going to need to keep hammering out that command line every time you do your work.

4. For a very basic example of how writing simple shell scripts can "save" you from hassle. Consider the following situation:

Every day at work you come in, log in to your Unix/Linux terminal and type:

host # cd /my/main/work/directory
host # ls |grep myuserid.txt
host # rm myuserid.txt
host # find . -name core -print >>myuserid.txt
host # cat myuserid.txt|xargs rm -f
host # cd

While that example above illustrates more than a few opportunities for improvement, that's not today's issue ;) Now imagine if, instead of logging in to your Unix/Linux terminal and grinding out those six lines, all you had to do was log in and type the following (we're keeping this so basic and simple, we're not even going to deal with using the shebang line in your script or setting the script up to be executable on its own):

host # /bin/bash myscript

Much less hassle, yeah? :)

Q: How do I write a shell script, anyway?

A: It's actually extremely simple (and we're keeping it extremely simple for now). There are a lot more steps you can take to make the whole perception of "hassle" wither even further, as well as make your script writing even more convenient for you, but that's for another day. I'm fighting every instinct in my head to try and keep this as "basic" as possible and not introduce any "logical constructs" or "error checking" into your first script.

The first thing to consider, when answering this question, is another question. "What" is a shell script, anyway? It goes a long way to making the "how" obvious. A shell script, in its most pristine state, is simply a collection of command lines. No joking. That's all it has to be, and it can be just that simple.

Consider the example above (point number 4 of the answer to the first question). If you simply take all those lines that you type everyday and put them in a file, you will have written your first shell script! So, open up your favorite editor and either straight-type or cut-and-paste all those lines into it. It's very important, since we're not going to address issues of efficiency beyond the basic, that you type each command line on a separate line in your new file. It should look exactly like this (your commands may differ, of course ;)

cd /my/main/work/directory
ls |grep myuserid.txt
rm myuserid.txt
find . -name core -print >>myuserid.txt
cat myuserid.txt|xargs rm -f

Save that file as "myscript" (or whatever else you'd rather call it) and you're all done. You've written your first shell script!

Now, instead of typing 6 lines of commands, you only have to type one. Since we're not getting into setting up your shell script to be executable on its own, you just need to invoke it with a shell (preferably the one you use to type those commands with every day). If you're not sure what shell you use, you can find out that information by simply typing the following at the command line:

host # echo $SHELL

and, assuming that your environment variables are all set up correctly (arggh, losing focus... ;), that command will return the value of your shell. Now you can execute your shell script easily in either one of two ways.

a. If you're in the same directory as your script, you can execute it by running:

host # /bin/bash myscript

b. If you're not in the same directory as your script, or prefer to use full paths, you can invoke it like this:

host # /bin/bash /directory/where/the/script/is/myscript

and all of your commands will be executed, in order, as if you had manually typed them at the command line yourself.

That, in an nutshell, is the how and the why of writing a basic shell script. It should also address the "hassle" issue ;) Taking this simple example (and learning no more about how to create shell scripts than you've learned so far), you'll probably realize that there are several other things you do every day (all of which require prodigious typing) that you can put into shell scripts and stave off carpal-tunnel syndrome for at least a little while longer ;)

Over time, you should begin to experience the convenience shell scripting has to offer you and become curious about how much more it can help you by reducing the hassle of your workaday tasks and introducing automation of the mundane. Eventually, you may even become interested in more advanced concepts and begin to enjoy the full benefit of shell scripting.

The main thing to keep in mind (no matter how complex your shell script ever becomes) is that a shell script (shebang line excluded) is merely a collection of commands that you could run, yourself, from the command line. Keep in mind, also, that your shell includes many features that were not showcased here today. Even when you avail yourself of those, any shell script can be written on the command line and anything you can write on the command line can easily become a shell script.

I hope that long-winded answer to a fairly short (but definitely loaded ;) question was of some help to you. If you're interested in learning more, search around this blog or elsewhere on the Internet. If you can't find the answer to a question you have, feel free to email me via the link at the top right of the page. My response time isn't what it used to be (before I began inviting people to write me ;), but I will get back to you as soon as I can.

As a final exercise to demonstrate the simplicity of very-basic shell scripting (assuming, as above, that /bin/bash is your shell), try the following at your command line:

host # echo hi
host # echo "echo hi" >>newscript
host # /bin/bash newscript

Nice work :)


, Mike

Discover the Free Ebook that shows you how to make 100% commissions on ClickBank!

Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.