Questions Regarding Perl References In Linux And Unix

Hey There,

Today's post is a follow up to our post, from earlier this week, regarding understanding Perl variable references. The feedback was generally positive, but one particular question was asked by quite a few people. And that question was:

If there's nothing new under the sun, then how can there be more things in heaven and earth than are dreamt of in anyone's philosophy?

To be quite honest, I have no idea. I quit pondering the meaning of "it all" when I was introduced to nihilism at the age of 18, which, consequently, fit in perfectly with my intensive practical application of Onanism. My philosophy was both fun "and" meaningless ;)

But, that wasn't the real question. I just went off on a tangent there for a moment, as it's my wont to do. Apologies for any trauma caused by the high-brow self-abuse joke. It has nothing to do with Linux, Unix or Perl unless you, like Sammy Hagar, consider it all mental masturbation ;)

The real question on people's minds was this:

What purpose, at all, do references in Perl serve, and why would I ever want to complicate things by using them?

This is a valid question; especially if you aren't hard-core into Perl or only use it as a work aide. Quite frankly, I, myself, rarely have any use for references. But here's a quick rundown of a few reasons why it's good to know them and understand how they work (Note that, in Perl, references and pointers - as they're more commonly known in programming languages like C - are terms that often get used interchangeably. For our purposes, they both mean the same thing).

1. For folks who've been using Perl since version 4 and earlier, references were an absolute "must" to understand if you wanted to do any semi-complex work with subroutines. At that point in time, you could only pass scalar variables as arguments to a subroutine. This made it impossible for you to, for instance, pass an array, hash or more complex variable as an argument to your subroutine. Thus, the reference made it possible for you to pass an array or hash as an argument to your subroutine, since the Perl reference to either was (and is) always a scalar variable. I think this is the one issue that is really being addressed when folks question the usefulness of Perl references these days. As the below scriptlet shows, the days of only being able to pass a scalar variable to your subroutines are long since gone. The language has been improved upon and, now, you can pass arrays, hashes, etc, directly to your subroutines as arguments, without having to make use of scalar references to them (more on that in point 2). Check out the following for a demonstration of how Perl can now handle these types of arguments (a few of you actually already have this in your mail, as I put it together to demonstrate this principle earlier this week in response to several emails - I aim to please :):

host # cat shell.pl
#!/usr/bin/perl

$bfile = "what the heck";
@bfile = qw(what the heck);

scalarsub($bfile);
arraysub(@bfile);

sub scalarsub   {

       my $file=shift;
       print "SCALAR: F $file\n";
}

sub arraysub    {

       my @file=@_;
       foreach $item (@file)   {
               print "ARRAY: F $item\n";
       }
}
--- test run of script ---------
host # ./shell.pl
SCALAR: F what the heck
ARRAY: F what
ARRAY: F the
ARRAY: F heck

As you can see, above, today's Perl doesn't require you to reference non-scalar variables if you want to pass them as arguments to a subroutine.

2. References still do have a place in Perl, even when it comes to subroutines. If it sounds like I'm contradicting what I just wrote, please allow me to dig myself further into a hole... I mean, explain ;)

Consider, if you will, the following situation in which making use of Perl references would actually save you considerable time and, probably, a gray hair or two. If you have a simple subroutine (like in our scriptlet above), it is very easy to pass it several scalar arguments (assuming no restrictions on the amount of arguments you can pass) and, logically, any amount, and/or combination, of different types of variables. But, what happens when you call your subroutine with arguments like this?:

@array1 = qw(1 2 3 4 5);
@array2 = qw(1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19);
@array3 = qw(1 2 3);
mynewsubroutine(@array1, @array2, @array3);
@_ = 1 2 3 4 5 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 1 2 3; <-- within your subroutine.
my (@array1,@array2,@array3) = @_;

Now, while there are more than several clever ways you can get around parsing this input (which is one of the great things about Perl), it can be difficult to process this sort of input within the subroutine; especially if the arrays, themselves, are of variable and/or undetermined size (perhaps @array1 contains 5 scalar variables, @array2 contains 19 scalar variables and @array3 contains an additional 3 scalar variables - we'll, again, stay away from passing arrays of hashes of arrays for now ;). This can constitute a lot of work on the programmer's part to craft the subroutine in such a way that it handles all of the input and keeps it separated, correctly, into the three groups, since the totality of the arguments passed to a subroutine can be referenced by @_ and, if three separate arrays are passed, @_ will contain all 3 of them (27 scalar variables) and you can't separate them by index @_[0], etc, since the input to your subroutine is, in effect, one large array, rather than a collection of 3 smaller arrays. In effect, and in reality, your subroutine would put all of the members of all of the 3 arrays into @array1.

Hopefully, I'm not drifting too far off of the reservation, here, as I fear I'm wandering into the land of overly convoluted explanation again ;) The advantage to you, the programmer, in this instance, would be that you could use Perl references to pass scalar references to each array to your subroutine. As difficult as it would be to manage the above three arrays using straight passing to your subroutine (you would have to determine their size before passing them in and then keep track of that by passing even more information, not to mention re-allocating those members back into the subroutine's local arrays), it would be a bit easier to pass your subroutine three references to your three arrays, like so:

mynewsubroutine(\@array1, \@array2, \@array3);

Now, the reason this makes life easier for you is that you're only passing your subroutine 3 scalar variables, rather than an indeterminate amount of any other kind. Inside the subroutine, you can easily work with the variables using the dereferencing techniques we went over in our understanding Perl references post. For instance, you could do "foreach" on all 3 arrays separately, and process the internal array variables that way, effectively making the size of any of those arrays inconsequential (less work for you), like so:

my $array1 = shift;
foreach $thingy (@{$array1}) {
 print "Thingy $thingy\n";
}

I'm oversimplifying to a certain degree, but this is getting out of hand just swimmingly in spite of my best efforts ;)

3. Although I promised I wouldn't go here, I must (but I'll keep it theoretical so we can dispense with it post-haste). If you ever do any super-complex programming, like building matrices, doing crazy manipulation of filehandles (remember that "any" Perl data type can be referenced) or things along the nature of manipulating arrays of hashes of multiple nested arrays of hashes, using references can make it a lot simpler for you to keep your head from exploding (which is a lousy way to end the day ;)

Hopefully, these answers have been somewhat helpful, and/or illustrative regarding the question at hand (why use references at all, and are they even really necessary?) If you're interested in reading more on the subject, check out Chapter 18 of bjnet.edu.cn's introduction to Perl. It's written very well and is probably simpler to follow than my meandering over-explanations ;) Plus, all the other chapters are available online, so you could download them all and have yourself a very nice free Perl reference in however long it takes you to download the HTML.

Cheers to all, and thank you for your response to the post that spawned this one. There's nothing quite so rewarding as knowing that I'm actually writing something at least a few people are really interested in reading :)

, Mike

---

Discover the ClickBank affiliate program that pays 100% commission!

---

Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

---

Understanding Perl Variable References On Linux And Unix.

Hey there,

Today we're going to take a look at a part of Perl that a lot of folks shy away from; mostly because (from my experience) they feel it's too abstract a notion or too complicated to understand. For today, I'm referring to Perl references ;) And here's the thing; nothing could be farther from the truth. It's just about as simple as the sentence preceding the last. When I referred to Perl references I was, for the most part, laying the foundation for easily understanding the entire concept. If you attack the problem semantically, and try not to think of it as a bunch of backslashes and arrows and symbols, it makes perfect sense :) If I'm wrong, and this post leaves you reeling in confusion and pain, please let me know so I'll stop being so cavalier with my prose :)

So, let's take a look at Perl references and how they can be used, most basically, in a step-by-step fashion; from the simplest of beginnings to the not-so-complex middle (We'll leave references to hashes of arrays of references to other hashes for some other day ;)

For all of these examples, we'll use command line examples, so you can cut and paste them to try them out, rather than pretend that we're inside a Perl script.

1. A simple way to look at Perl References: The basis of any Perl reference is the variable, or value that you're referring to. At the most basic level, any variable assignment is a reference. For instance, look at these basic statements:

host # perl -e '$a = "bob";'
host # perl -e '@a = qw(bob joe);'
host # perl -e '%a = (bob => joe);

These are all just simple variable assignments (with $ indicating a scalar variable, @ representing an array and % representing a hash). However, you can think of them as references (which will make the transition to understanding textbook references much more smooth. The variable $a, for example, has an assigned value of the string "bob." So, if you look at that in a different way, the $a variable refers to the string "bob," or (another way) the variable $a is NOT the string "bob," but a reference to that string (or scalar) value.

BTW, if this part of the post is beyond where you're at with Perl, take a look back at some of our older posts on simple arithmetic and simple variables in Perl that deal with these more basic principles. There should be enough links on those two pages to connect you to all the other ones on this site. If not, the blog search feature (although it's very generous in its interpretations -- search for the letter "a" to see what I mean ;) should help you find what you need.

2. Looking at actual Perl References: A textbook Perl reference is the same thing as we discussed in point 1, except taken up (or out) one meta-level. So instead of having the relationship of reference ($a) and referent ("bob") that we had before, we're going to assign one scalar variable a reference to any of the three variables from before, rather than from the variables directly to the values. So, to reference any of these three we could do the following (note that for this basic lesson, the Perl reference will always be a scalar since, at its core, it always is; even if that scalar value is a part of a larger array or hash). The symbol that denotes that you're setting your variable's value to a reference is the backslash (\) character:

host # perl -e '$a_ref1 = \$a;'
host # perl -e '$a_ref2 = \@a;'
host # perl -e '$a_ref3 = \%a;'

So now we have three very simple Perl references. $a_ref1 has the value of a reference to the $a scalar variable, $a_ref2 has the value of a reference to the @a array and $a_ref3 has the value of a reference to the %a hash. (Note that you can have a Perl variable refer to itself, although the uses for this are somewhat limited and generally not necessary for basic Perl scripting. Ex: $a_ref4 = \$a_ref4 <-- $a_ref4 has the value of a reference to itself.

3. Extracting values from Perl References: This is just as easy as extracting values from regular variables, except, as before, you have think one more hop. Whereas, with a regular variable, you would extract the value of that variable directly, with a Perl reference, you need to extract the value of the variable that is being referenced by your reference. It sounds worse than it is ;) For instance, if we accept that the scalar variable $a is equal to "bob," we know that we can extract the value of $a by doing the following (as before):

host # perl -e '$a = "bob";print "$a\n";'

Whereas, if we create a reference (another scalar variable) to the variable $a, and call that $a_ref1, we need to extract the value from the variable that we are referencing. A simple and comfortable approach to extracting this value would be the following:

host # perl -e '$a = "bob"; $a_ref1 = \$a; print "${$a_ref1}\n";'

In this instance we've simply peeled the onion, so to speak (insert your favorite peelable vegetable or fruit here ;). In order to extract the variable of $a from the Perl reference $a_ref1 variable, we just stripped it layer by layer. To deconstruct the print statement above, we'll go backward from the statement we used to print the value of the $a_ref1 Perl reference:

a. ${$a_ref1} is what we call to print the value of the variable $a.

b. ${$a_ref1} is actually equal to ${\$a} since $a_ref1's value is a reference to $a (as denoted by "$a_ref1 = \$a;")

c. ${\$a} is equal to ${a} since the we're dealing directly with the referent. $a_ref1 (the variable with the value of the reference actually points to a hex address in memory (usually associated with a Perl file type). You can see the difference in the output of the two commands below:

host # # perl -e '$a = "bob"; $a_ref1 = \$a; print "$a_ref1\n";'
SCALAR(0x2e250) <-- This is the hexadecimal memory space that the $a_ref1 reference refers to. Your results may vary :)

perl -e '$a = "bob"; $a_ref1 = \$a; print "${\$a}\n";'
bob <-- This is the value of the referenced variable $a, which we know (from before) is equal to "bob" ($a = "bob" from what seems like so far up the page ;)

d. And, even though we don't need to tell you this, just for completeness' sake: ${a} (or $a - same thing) equals "bob".

4. Extracting values from Perl References that aren't scalar: Finally, some good news :) The principles above apply to all sorts of variable dereferencing. So, for instance, if you wanted to extract the value of the array reference $a_ref2, you could get it by doing:

host # perl -e '@a = qw(bob joe); $a_ref2 = \@a; print "@{$a_ref2}\n";'
bob joe <-- The whole thing
host # perl -e '@a = qw(bob joe); $a_ref2 = \@a; print "@{$a_ref2}[0]\n";'
bob <-- array index 0
host # perl -e '@a = qw(bob joe); $a_ref2 = \@a; print "@{$a_ref2}[1]\n";'
joe <-- array index1

and the same basic principle applies to hashes (%{$a_ref3} would get you all those values). Basically, all you need to do to extract the value of a one-level-deep Perl Reference is to wrap the reference-variable in a curly brackets and preface that with the appropriate symbol ($ for scalar, @ for array, % for hash, etc).

5. What to do if you have no idea what kind of Perl Reference you're dealing with: Fortunately, there exists - in the very heart of Perl - a function to deal with just this sort of predicament. It's called, for some strange reason, "ref" ;) On many systems, doing something like this:

host # perl -e '@a = qw(bob joe); $ref_type = ref(\@a); print "$ref_type\n";'
ARRAY

is all you need to do to get back the type of reference you're dealing with (Obviously, we knew it was an array since we're doing these self-contained command line scripts, but you could use the ref function against any Perl Reference and get the value from it. One thing to note about the ref function is that it doesn't always work as expected. For instance, if you call the function ref on a straight-up scalar, array or hash variable, it should return "undefined." This is normal, since those straight-up variables are "not" references. However, sometimes, even when you are dealing with a reference, you won't get any feedback on your command line. This isn't to say that ref doesn't know what kind of reference you're working with; just that it's not in the mood to tell you ;)

You can get around this little hassle pretty simply by just writing a simple type-check. So if you run the following:

host # perl -e '%a = (bob => joe); $ref_type = ref(\%a); print "$ref_type\n";'

and you don't get the return of

HASH

as you would expect to, you can figure out what the return from ref was anyway. The two most basic ways to do this range from cowboy to academic ;)

a. Cowboy: Just print the variable that points to the reference, like we did above, to get the hexadecimal address (instead of the value of the referenced variable), since this is accompanied by the reference type:

host # perl -e '%a = (bob => joe); $ref_type = \%a; print "$ref_type\n";'
HASH(0x2e26c)

b. Academic: Use a simple if-condition to test and see what kind of output ref returns

host # perl -e '%a = (bob => joe); $ref_type = ref(\%a); if ($ref_type eq "HASH") {print "HaSH FOUND!\n"};'
HaSH FOUND!

Of course, you could check just to see if the value is even "defined," since, if it isn't, you're not dealing with a reference. 99% of the time, Perl will do the right thing and tell you what the ref function returns. For all I know, the 1% of the time it doesn't work for me is because I completely screwed up ;)

Perl also deals with references to a lot of different file and object types to which these same basic principles apply. So, if you're dealing with a pipe or another type of file or variable, you can still use the principles above to help you out. And, for simplicity's sake (until you get used to being utterly confused while in a state of mostly-understanding ;), for every level of referencing that gets added on, you just need to derefence that many times backward (as shown above) to make your way back to the original value of the original variable(s)!

And that wraps up that :)

Hope that helps shed some light on basic Perl References and, again, I'd love to hear what you think about this post; especially with regards to how you felt about it (Was it too simplistic? Too Complicated? Hard to understand? Easy Peasy? ;)

Cheers,

, Mike

---

Discover the ClickBank affiliate program that pays 100% commission!

---

Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

---

Tainted Perl On Linux or Unix - Helping You Protect You From Yourself

Hey there,

Generally, when you're writing a Perl script to help you automate any Unix or Linux tasks, you don't really need to worry about security. Aside from the fact that you could delete everything on your system or write an infinitely recursing loop that will chew up all the CPU... on second thought, thinking about security is probably a good idea most of the time ;) Super-heavy security checking isn't really necessary for small things, but is always a good idea to work toward when writing scripts to execute important system function and/or for the use of others.

This is where Perl's "Taint" comes into play. It's kind of like "-w"'s less-tolerant cousin. While running Perl with "-w" will print out all sorts of warnings if your code is suspect, Taint will shut you down. Some builds of Perl may have a "-t" option that acts more like the "-w" flag (stronger checking, but only prints out the warnings).

Perl's Taint mode can be added to any script by simply changing the shebang line from:

#!/usr/bin/perl

to

#!/usr/bin/perl -T

Not too much extra work to get it set up, although now you'll have more things to consider when you write your script ;) Taint mode will cause your script to fail, now, if it feels the script is not secure enough! Interestingly enough, Perl will generally turn Taint mode on automatically if you change any Perl script's permissions to setuid or setgid (which is when it's probably needed the most :)

One of the most important things Perl's Taint does is "not" allow external data (input) to be used in any routine, or action, that will affect other data (or whatever else) external to your script, unless you sanitize that input first.

For instance, the following would not be allowed by Perl Taint (Note that most actions that cause Taint errors are system calls, backtick operations or exec calls):

$variable = $ARGV[0]; <--- We've assigned the first argument on our script's command line to the $variable variable.
system("$variable"); <--- and now we're executing that argument as a command from within the Perl script!

Obviously, in this example (assuming the name of our script is PerlScript and it runs as a user of sufficient privilege), we could do something like the following and cause a big problem:

host # ./PerlScript "rm -rf *"

Ouch! This sort of thing is actually seen a lot in CGI programming, with the assumption being that the "nobody" user that most folks run their Web Server as, can't do all that much damage. Consider, however, that (in the "nobody" example) the "nobody" user probably does have permission to delete all of your html and cgi-bin files. That would be headache enough, even though you'd still get to keep your operating system ;)

Taint also acts to protect you comprehensively. In our limited example above, adding the -T flag would have protected us against that contaminated $variable variable. However, Taint will also work on arrays and hashes and, even better, will treat them as collections of scalar variables (which they, essentially, are). So you can, reasonably, end up in a situation where only some values in an array or hash are Tainted, while the rest of them are considered perfectly safe (or unTainted).

If you have the -T flag enabled, whenever Perl runs into a situation where the data (input or output) is considered Tainted, it will terminate the execution of your script with a brief explanation ( like a description of what variables are insecure and/or what insecure dependencies they have on other variables) describing why you might be in trouble if you run your script as-is.

A great thing for your script (and your security) is that it takes relatively little effort to sanitize a Tainted variable.

For instance, in our example above, $variable was considered Tainted as soon as it became assigned the value of $ARGV[0] (You wouldn't get the actual error until you tried to use that Tainted variable data). If you wanted to clean that variable before running the system call on the next line, you'd just need to process it.

So, while this chunk of code would be considered unsecure (or Tainted):

$variable = $ARGV[0];
system("$variable");

This chunk of sanitized code (actually, just with a sanitized $variable) would be considered secure (or unTainted):

$variable = $ARGV[0];
$variable =~ s/;//g;
system("$variable");

Obviously, there's a lot more to go into when it comes to Perl's Taint, but, hopefully, this has served as an easy-to-understand introduction. Now you can feel free to read the 50 pages of detailed specifications ;)

At the very least, you can use the Taint flag to check your Perl scripts while you write them, and then remove it when you want to put your work out there for everyone to use.

Best wishes,

, Mike

Porting Associative Arrays Between Perl, Shell and Awk

Hey There,

And, for the last time (this week) we're back to porting :) One thing you might have noticed, in the title of today's post, is that the C programming language has been dropped. We received some feedback, and also generally agree, that throwing C in with Perl, Shell and Awk creates a layer of confusion that distracts from the point of these posts. Which is, of course, to demonstrate how simply one can use concepts, and apply them easily, with a little translation, between three very useful Unix and Linux programming (or scripting) languages.

We'll leave the original posts alone, so if you go back to revisit our original post regarding simple string variables and its follow-up that dealt with simple arrays, please just ignore all that stuff about C. If there's sufficient interest at a later time, we'll write up a series of posts on porting between (for instance) Perl and C and try to contain the amount of collateral damage ;)

Today, we're going to look at associative arrays (which are generally referred to as "Hashes" in Perl and as "lookup tables" from time to time) and how we can define, add value to, and extract value from, them in Perl, bash and awk.

The main differences between a "regular" array and an associative array is, at its most basic, found in the way that the array is indexed and the way in which you can add and extract value from the array. This is why I noted that, in bash, the only type of arrays available (at this point) are "regular" or "one dimensional" arrays, which we looked at in our last post on array porting ( Please note that I'm stuck using version "2.05b.0(1)" and am unaware of any newer release (up to bash 3.2) which directly supports this functionality. I "am" looking forward to it, though :)

The good news is that there is a hack to get around this, and emulate associative arrays, if you want to drive yourself crazy. We will definitely follow up this post with a post regarding how to do that (it'll be a post in and of itself, so we'll not belabour that point any longer, here). Also, in awk, there's, technically, no such thing as a "regular array." If you dump an Awk array, there's no guarantee that the contents will come out in the order you expect (which is also true of the Perl hash) and you can also use keys with them, instead of being limited to indexes. This will begin to make sense, if it doesn't already, as we go along. Terminology being what it is, I may have upset a lot of people already ;)

And, now that I've dug myself a deep enough hole, let's get started with associative arrays ;)

The associative arrray is sometimes defined as a "lookup table" for good reason. Compared with "associative array" and "hash," it's a name that's more descriptive of how the associative array works. Basically, on the left hand side of any assignment (the variable before the value) you have the name of the associative array along with a "key," like: $bob{"key"} - On the right hand side of the equation you have a simple value. You can see how the term "lookup table" makes the most sense as, in this case, if you wanted to look up the value of "key" you could find it in the associative array named "bob" simply by naming it. The associative array can also be thought of as having "keys" and "values," rather than "indexes" and "values," like regular, one dimensional, arrays have.

Since Bash only supports one-dimensional arrays up to (at least) version 2.05b.0(1), none of today's lesson applies. We will, however, definitely follow up with a post dedicated to "faking" associative arrays with bash.

1. Defining, Initializing or Declaring an associative array. As was the case with simple variables and arrays, no explicit declaration of an associative array is absolutely necessary in either of our (now) two languages:

Ex: We need to define an associative array called MySimpleHash. Even though it's unnecessary, we could do so like this:

In Perl: Just type "%MySimpleHash;" - The % symbol indicates an associative array (or hash) in Perl, as opposed to the $ sign, which indicates a scalar (or simple, or string) variable and the @ symbol which indicates an array. Perl hashes can also be created by simply defining their elements on-the-fly.

In Awk: Just type "declare MySimpleHash" - Again, associative arrays in Awk can be created by referencing their components and work exactly like one-dimensional arrays.

2. Assigning values to the associative array. This is very straightforward in both of our two remaining languages:

Ex: We want to assign the keys "MySimpleKey0," "MySimpleKey1" and "MySimpleKey2" ( with values, in order, of "MySimpleValue0", "MySimpleValue1," and "MySimpleValue2") to the associative array named MySimpleHash (Note that any values that contain spaces should be quoted - it's actually good practice to quote any string that is a being used as a key or a value in an associative array. This is generally not necessary for integer values).

In Perl: Just type "$MySimpleHash{"MySimpleKey0"} = MySimpleValue0; $MySimpleHash{"MySimpleKey1"} = MySimpleValue1; $MySimpleHash{"MySimpleKey2"} = MySimpleValue2;" - Spaces between the variable, "=" sign and value are optional. Note that we have to use the $ symbol when referring to an element of a Perl hash, while we use the % symbol to refer to the entire hash (or associative array. Same thing. But you know that by now ;) Also, last thing, notice that the square brackets, used to refer to regular array indexes, are replaced by curly brackets.

In Awk: Just type "MySimpleHash["MySimpleKey0"] = "MySimpleValue0"; MySimpleHash["MySimpleKey1"] = "MySimpleValue1"; MySimpleHash["MySimpleKey2"] = "MySimpleValue2";" - Spaces between the variable, "=" sign and value are not, technically, necessary, but recommended. Also, note that all of the "MySimpleKey" keys, and "MySimpleValue" values, are placed within double quotes in the assignment. This is sometimes necessary for string values, but usually not always necessary for numeric values.

3. Extracting the value from your simple associative array. This is no longer "trivial," but still not too terribly difficult to do. Let's look some stuff up in these tables :)

Ex: We want to print the value of all of the keys of the MySimpleHash associative array. This is also fairly simple in our dwindling nation of two languages ;) Note that we will be iterating over the array key/value pairs in order to print them all out. This is where you'll notice one very specific characteristic of the associative array (or hash). The associative array does it's own indexing based on an internal algorithm and may not spit out the same values in the same order every time you dump the contents. This is done, by the programming language, for maximum efficiency with regards to information storage and retrieval (with the assumption that you'll be looking for a particular "key"'s value, I suppose).

In Perl: Just type "while (($Key, $Value) = each %MySimpleHash) { print "$Key equals $MySimpleHash{$Key}\n";}"
- Note that the $ character needs to precede the variable name when you want to retrieve any of an associative array's key or value elements. Also, when iterating over the hash, you need to refer to the hash directly with the % prefix (printing %MySimpleHash would print out all of that hash's elements - generally all squished together with no separating space) - The \n, indicating a carriage-return, line-feed or new-line isn't necessary, but is nice if you don't want your output on the same line as your next command prompt:

host # perl -e '$MySimpleHash{"MySimpleKey0"} = MySimpleValue0; $MySimpleHash{"MySimpleKey1"} = MySimpleValue1; $MySimpleHash{"MySimpleKey2"} = MySimpleValue2;while( ($key, $value) = each %MySimpleHash ) { print "SimpleKey: $key, SimpleValue: $value.\n";}'
SimpleKey: MySimpleKey0, SimpleValue: MySimpleValue0.
SimpleKey: MySimpleKey2, SimpleValue: MySimpleValue2.
SimpleKey: MySimpleKey1, SimpleValue: MySimpleValue1.

For a goof, let's print out the entire hash at once, so you can see how the indexing is done automatically by Perl (i.e. it might not come out in the same order that the data was logically entered by us) :

host # perl -e '$MySimpleHash{"MySimpleKey0"} = MySimpleValue0; $MySimpleHash{"MySimpleKey1"} = MySimpleValue1; $MySimpleHash{"MySimpleKey2"} = MySimpleValue2;print %MySimpleHash;print "\n";'
MySimpleKey0MySimpleValue0MySimpleKey2MySimpleValue2MySimpleKey1MySimpleValue1

And, if you can see in the crunch there, it has indeed ordered our key/value pairs as 0, 2, 1, instead of the 0, 1, 2 that you would expect from a regular array!

In Awk: Just Type "for (x in MySimpleHash) { print x "=" MySimpleHash[x]; }" - Note that the $ or % symbol "must not" precede the variable name, or key name, when you want to get the value. Note that, just like regular arrays, awk associative arrays need to be iterated over to be entirely printed out. Again, here, we should see how awk has decided to index our key/value pairs, which might be different than the order in which we entered them (you'll generally see this more if you mix numeric and alpha keys in your awk associative array (here, we seem to get lucky :) :

host # echo |awk '{MySimpleHash["MySimpleKey0"] = "MySimpleValue0"; MySimpleHash["MySimpleKey1"] = "MySimpleValue1"; MySimpleHash["MySimpleKey2"] = "MySimpleValue2";for (x in MySimpleHash) { print x "=" MySimpleHash[x]; }}'
MySimpleKey0=MySimpleValue0
MySimpleKey1=MySimpleValue1
MySimpleKey2=MySimpleValue2

An interesting side fact about awk associative arrays is that, even if they come out in the order you expect, they may not be indexed exactly as you indexed them. For instance, when you enter the first, second and third value, you would expect them to occupy indexes 0, 1 and 2 from an regular awk array. An awk associative array may have those values indexed as 0,44 and 117. Let's see what happens if we pull keys 0, 1 and 2 out from MySimpleHash:

host # echo |awk '{MySimpleHash["MySimpleKey0"] = "MySimpleValue0"; MySimpleHash["MySimpleKey1"] = "MySimpleValue1"; MySimpleHash["MySimpleKey2"] = "MySimpleValue2";print MySimpleHash[0] MySimpleHash[1] MySimpleHash[2];}'

host #

Nothing. Goose eggs; as expected. Since awk creates the association for you, the simple numeric indexing won't work to retrieve the value for, say, "MySimpleKey2." If you want to force numeric indexing, you just need to make sure that all of your indexes are numbers (which is how we emulated a regular array in our previous post on working with regular arrays). Note, also, that if you want to make sure that a number you enter as an index is treated like a key in a key/value pair, the simplest way to do this is to enclose it in "double quotes."

Sadly, even though this article ran a bit long, there are still a lot of "little things" about associative arrays (or hashes, lookup tables, or whatever you prefer to call them) that can be explored. Hopefully, you're encouraged to get down to the nitty-gritty and rise above the attempted basic-ness of this post (Now I'm starting to make up words ;)

In our next addition to this threaded series of posts, we'll start looking at programming constructs (loops, conditionals, etc) and explore how we can use the basic variable knowledge we've acquired so far to get some real work done now(after all, that's less work for us to do later ;)

Cheers,

, Mike

Working with Linux RPM's

This post is a continuation, of sorts, of my last post. This is more of a general-audience post. Most experienced admins know most of this stuff already. Like I mentioned previously, I try to write this blog with an appreciation for what it was like when I first started out in the business. I owe my success to a great many patient and helpful people.

In this post, I wanted to hit on the basics of working with RPM's in Linux (RPM stands for the Redhat Package Management system - basically, they're the software packages that make up your system). In later posts we'll go into some neat tricks... But for now, we'll stick with the basics. Knowing the basics in any field of interest is invaluable in growing and mastering that skillset, just like knowing your ABC's can really help if you ever intend to read or write :)

Check the bottom for a recap of all the RPM options we're going to use and their literal meanings:

1. To display the basic information for any RPM, just type:

host # rpm -qi RPM_NAME - like:

host # rpm -qi bash
Name : bash Relocations: /usr
Version : 2.05 Vendor: Red Hat, Inc.
Release : 8.2 Build Date: Mon 28 Jun 2004 10:33:55 AM CDT
Install date: Thu 12 Jan 2006 01:25:27 PM CST Build Host: host.redhat.com
Group : System Environment/Shells Source RPM: bash-2.05-8.2.src.rpm
... EDITED OUT FOR BREVITY'S SAKE!

2. If you're not sure where to start with the above command, just have RPM spit out all the packages it knows about and pipe that to more, like so:

host # rpm -qa|more
redhat-logos-1.1.3-1
glibc-2.2.4-32.18
cracklib-2.7-12
dosfstools-2.7-1
gdbm-1.8.0-11
...

3. Now that you've figured out what package you want to inspect (Note that you don't have to include the full name to get the information from RPM. The redhat-logos-1.1.3-1 program can be referred to simply as redhat-logos) and have gotten some basic information about it, you can list out all the files associated with the package like this:

host # rpm -ql bash
/bin/bash
/bin/bash2
/bin/sh
/etc/skel/.bash_logout
...

4. Here's one that doesn't require a lot of output, since it's somewhat of a re-explanation. You can add the -p flag to the examples in points 1 and 3 if you're querying an RPM package, and not the RPM database!

host # rpm -qip bash-2.05-8.2.i386.rpm <--- Listing out information for the RPM package itself.
host # rpm -qlp bash-2.05-8.2.i386.rpm <--- Listing out files associated with the RPM package itself.

5. Of course, you may find a file and want to know what RPM package it belongs to. You can get that by typing:

host # rpm -qif /etc
Name : filesystem Relocations: (not relocateable)
Version : 2.1.6 Vendor: Red Hat, Inc.
Release : 2 Build Date: Mon 20 Aug 2001 03:34:02 PM CDT
Install date: Thu 12 Jan 2006 01:24:41 PM CST Build Host: host.redhat.com
Group : System Environment/Base Source RPM: filesystem-2.1.6-2.src.rpm Vendor: Red Hat, Inc.
... (Just as long as the description in point 1)

6. If you want to install a new RPM, you'll need the package file, and would run RPM like this:

host # rpm -i bash-2.05-8.2.i386.rpm

This isn't very interesting (which may be what you want -- I don't care to look at verbose output "all" the time). You can spice it up by adding the -v and/or -h flag, like so:

host # rpm -ivh bash-2.05-8.2.i386.rpm

7. If you want to uninstall an RPM, you'll just need to know the abbreviated name, like I mentioned in point 4). You can also make this as verbose and visually entertaining as the system will allow with -v and/or -h:

host # rpm -e bash

Note that this command would return an error if you had multiple instances of the bash RPM installed. In that case, you could still abbreviate, but would have to include the version number. So you'd type

host # rpm -e bash-2.05.8.2

instead of just bash.

So, to recap, and possibly explain anything I may have glossed over, these basic commands should get you started working with the RPM package management facility on Linux. The translations of the flags we've covered are as follows:

Major flags (usually the ones preceded with a dash, but you can arrange the flags in whatever order you choose - just be careful - see note in the minor flags):

q = query
i = install
e = remove/uninstall

Minor flags

i = information (not the same as the major flag i. Of course, you'll probably never use -ii or -ei, as the combinations would be redundant and opposite, respectively.
a = all
l = list
p = RPM package file (e.g. whatever.rpm)
f = file
v = verbose
h = hash (prints lots of # symbols while it completes your request :)

Enjoy getting started working with RPM packages. They're one of the foundations of the Linux operating system. In fact, a combination of certain packages actually "is" the operating system. Knowing how to manipulate them and have them work for you can make it easier to explore many other things (like new software you've always wanted to install and try out :)

Best wishes,

, Mike

linux unix internet technology