Happy Birthday To My Wife: Firefox Spotted In Outer Space

Hey there,

Today's post is dedicated to my wife, who shall remain ageless ;)

These photos are actually from about a year ago, when we were just starting this blog and the rest of my family ceased to exist for a little while :) You can check out the original post on blog.wired.com, but I've included the entire post here (minus the surrounding link-love logos) Her birthday was actually yesterday, if you put any stock in the date stamp on these posts ;)

Happy **th birthday, sweety :) (Her actual age has been removed to protect the innocent... who am I kidding. I'm guilty as Hell ;)

---

Firefox Logo Spied In Deep Space

By Charlie Sorrel April 11, 2008 | 6:43:02 AMCategories: Elsewhere in the Tubes  

One of these is a photograph taken by the Hubble space telescope on December 17, 2002, featuring the variable star V838 Monocerotis. The other is the same photograph overlaid with a familiar logo. Can you tell which is which?

Below is another photo of the same star, from the Wikipedia entry on V838 Monocerotis. If the above photo has been photoshopped to look more like the Firefox logo, the editing seems minimal: it's been rotated relative to the photo below, and some of the gas cloud may have been edited out around the "tail" of the fox.

If you've got a link to the original Hubble photo used above, let us know in the comments so we can compare. UPDATE: An eagle-eyed commenter, Wevah, spotted the original version of the Firefox-like nova photo on Wikipedia. It looks like the only modification was to rotate the image. It must be a sign from the stars!

Firefox logo spotted in deep space by the Hubble Telescope

---

, Mike

---

Discover the Free Ebook that shows you how to make 100% commissions on ClickBank!

---

Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

---

Double Spacing In Awk, Perl and Shell on Linux and Unix

Hey there,

Surprisingly, yesterday's post on multiline spacing with sed on Linux and Unix got some reponse. Enough so, that I figured I'd follow up today and answer the call for equal representation for Perl, awk and the plain old shell (pick one, any one ;)

This post is going to be much the same as yesterday's, except with a few different tricks using a few different languages, rather than just focusing on one (sed). In fact, except for the fact that I'm mentioning sed every fifth word or so, and we're going to kick things off with a brief review of yesterday's simplest command, today should be a sed-free post ;)

First, we'll look at the file itself (appropriately named FILENAME):

host # cat FILENAME
this is the beginning
and now we're in the middle
this is the end

As you recall, from yesterday's post on using sed to handle multiline spacing, we could easily double space this file, using sed, with the following command <-- Note that I won't be repeating the output any more,
since it's the same for every command to follow and we'll assume, from checking out the above mentioned post, that triple-spacing, etc, is an easy concept to grasp once you've got the "double space" down:

host # sed G FILENAME
this is the beginning

and now we're in the middle

this is the end

host #

Awk was the next obvious candidate for "character management." The easiest (read: shortest) way I could figure, find and confirm, was this obvious statement:

host # awk '{print $0"\n"}' FILENAME

I also saw this solution floating around everywhere, but am jailed to a Solaris 8 box as I write. It doesn't work on the native awk, here, but perhaps this does work on Linux
or with Gnu awk. I found that it definitely does work with nawk on Solaris:

host # awk '1;{print ""}'

Perl came up next. It's a heavy hitter, compared to sed and awk, but actually just about as compact on the command line:

host # perl -pe '$_ .= "\n"' FILENAME

Then I decided to check out the shell (which means pretty much every command available) and quickly realized that anything you could really do involved some other basic editing command (or editor) like ed or similar programs. Not really novel. Just an endless parade of variations on the same theme. I decided to just go with this command line, since it's just using the shell and not overtly pulling in any external programs, to get the job done:

host # while read;do echo "$REPLY\n";done<FILENAME

and then this line, too, since (even though it's more convoluted) it's just a little bit shorter:

host # while read x;do echo "$x\n";done<FILENAME

Hopefully that wraps up this whole business with line spacing in files on Linux and Unix. I'm certainly always receptive to additional comments, but, hopefully we've said all that really needs to be said on this topic and can move on away from it tomorrow ;)

Cheers,

, Mike

Shell Script To Monitor Disk Usage On Linux and Unix

Hey There,

Today, we're going to take a look at a simple shell script to monitor disk space usage. It's been quite a while since we've touch on that, going back to a post from last November regarding finding space hogs on overlay mounts. The script has been kept simple (basically checking every partition for one fixed percentage full) to highlight other features.

The main intent here was to set up a monitor that would be able to handle a variety of Linux and Unix Operating Systems (all dependant on the "uname -s" output from that system) and focus on that area distinctly. We've limited our initial list to HP-UX, Solaris, SCO and OpenBSD.

In this case we're using a simple case statement to enumerate through the four *nix's we have listed here. Obviously, we could easily add more operating systems, and their variations of the "df" command, to our list and, if it ever got too big, either roll them into an array or simplify the script so that more OS's would fall under the same umbrella.

Also, notice that we're stepping through parsing of the df output more tediously than is actually necessary. For instance, the creation of the df output could be parsed with sed all in one fell swoop. Again, although it is generally considered best practice to compact your script/code, our hope here was that this would be easy to follow for as many people as possible. Some folks learn better by tackling the tough-stuff and working back to basics and some of us learn better by starting with the basics and putting them all together to create the tough-stuff. It's a long and convoluted statement of philosophy, to be sure, but fairly descriptive of what we actually mean ;)

If you prefer, on the line where we parse the TABLE file and trim it with sed, you can take out this part (or add to it) as it was placed in there as an example of how to ignore a specific partition (/usr):

-e '/usr$/d'

So, the line:

sed -e '1d' -e '/usr$/d' ${BASEDIR}/TABLE >> ${BASEDIR}/TABLE2

could be changed to:

sed -e '1d' ${BASEDIR}/TABLE >> ${BASEDIR}/TABLE2

which could then be simplified even further (since you don't need to use -e, even though it's okay to, if you only have one instruction to pass to sed) to become this:

sed '1d' ${BASEDIR}/TABLE >> ${BASEDIR}/TABLE2

And the entire script could be thusly compacted and streamlined, etc.
In any event, I hope this can be of some help (or, at least, an inspiration to reach higher ;) to you!

Best wishes,





This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/ksh

#
# dfvk.sh - Check partition % full
# across multiple OS'
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

trap 'rm -f ${BASEDIR}/TABLE ${BASEDIR}/TABLE2 ${BASEDIR}/MAILER;exit 1' 1 2 3 15

#BASEDIR="/tmp"
BASEDIR="tmp"
LOGHOST=`hostname`
MAILEES="you@yourhost.com"
LIMIT=90
OSVERSION=`uname -s`

case $OSVERSION in

        SunOS | OpenBSD )
                df -k >> ${BASEDIR}/TABLE
                ;;
        HP-UX )
                bdf >> ${BASEDIR}/TABLE
                ;;
        * )
                df -v >> ${BASEDIR}/TABLE
                ;;

esac

echo "$LOGHOST is getting ready to bother you!" >> ${BASEDIR}/MAILER
echo >> ${BASEDIR}/MAILER

sed -e '1d' -e '/usr$/d' ${BASEDIR}/TABLE >> ${BASEDIR}/TABLE2
sed 's/%//g' ${BASEDIR}/TABLE2 > ${BASEDIR}/TABLE
cat ${BASEDIR}/TABLE |while read ONE TWO THREE FOUR FIVE SIX
do

        case $OSVERSION in

                HP-UX | SunOS | OpenBSD )
                                if [ $FIVE -gt $LIMIT ]
                                then
                                        print "$SIX partition at ${FIVE}% capacity" >> ${BASEDIR}/MAILER
                                else
                                        continue
                                fi
                        break;;
                * )
                                if [ $SIX -gt $LIMIT ]
                                then
                                        print "$ONE partition at ${SIX}% capacity" >> ${BASEDIR}/MAILER
                                else
                                        continue
                                fi
                        break;;

        esac

done

MAILCOUNT=`cat ${BASEDIR}/MAILER |wc -l`
if [ $MAILCOUNT -gt 2 ]
then
        cat ${BASEDIR}/MAILER |mailx -s "$LOGHOST : Potential Paging Threat!" $MAILEES
fi

rm -f ${BASEDIR}/TABLE ${BASEDIR}/TABLE2 ${BASEDIR}/MAILER

, Mike

More Fun With Security Through Obfuscation On Linux And Unix

Hey There,

A few months ago, we ran a post on security through obfuscation, regarding ways to make your code less comprehensible to folks who didn't understand it as well as you do. While, at the core, this does nothing to actually "secure" your code from anyone with the patience to unravel it, it can be useful for keeping folks who know "just enough to do a lot of damage" from messing with your scripts ;) And, on another level, it can be a useful stalling technique, since you'll be making anyone work to figure out how your script/code works before they can change it without risking destroying it.

While I'm sure there are many more points to cover (and, hopefully, we'll cover them all eventually ;), today's addition to the arsenal is pretty obvious: Messing with your variable names :)

Most decent and responsible scripters will try to have their variable names make sense (if for no one's benefit than for their own - writing a confusing script isn't going to be worth it if even you can't figure it out later ;). So, when they use a variable to use as a counter, they might name it $counter, or something like that. @array_of_files is another such descriptive name. Very helpful later on when you need to modify your own stuff, and good practice so that, if you ever leave a place of employment, you won't be screwing your friends and co-workers who will, inevitably, get stuck maintaining your code.

However, assuming you do something as simple as keeping a separate file with a simple legend, like:

VAR ORIG
a file
b count
...

and so forth, naming your variables as arcanely as possible is a great way to make long, convoluted, stream-of-consciousness scripts as close to incomprehensible as possible ;) Having this sort of legend available can make it easy for you to do variable replacement within the "confusing" version of the script to make it easily understandable once again.

For instance, this Perl script to reverse all lines in a file on Linux or Unix can be rewritten to be even more confusing that it may already be.

Following is a working, but awful looking, revision of the original code, modified to include meaningless variables (scalar, array and filehandle), no indentation, no spaces between lines and no comments (except at the top). I've opted not to include a legend, since we have the original code hosted on this blog (linked to above) and you won't need to go through the hassle of reversing it.

In a future post, we'll look at creating such a legend for any script and backward-applying changes to your code (even though it would be much easier to do with "patch" ;)

Enjoy :)





This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/usr/bin/perl

# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

if ( $#ARGV != 0 )      {
exit(1);
}
$a=$ARGV[0];
open(A, "<$a");
@a = <A>;
close(A);
foreach $b (@a)      {
chomp($b);
@b = split(//, $b);
$c = @b;
undef(@c);
while ( $c > 0 )   {
push(@c, $b[$c-1]);
$c--;
}
$e = join(/ /, @c);
unshift(@f, "$e");
}
open(B, ">$a.reverse");
foreach $d (@f)    {
print B "$d\n";
}
close(B);
exit(0);

, Mike

Trimming Space in /var - The Problem with Solaris' lastlog

Here's another interesting tidbit from the "terminally boring" archives of system administration ;)

A lot of times, when you get a complaint that /var, on a Solaris box, is exceeding whatever size limitation you've placed on monitoring it, your first inclination is to go and wipe out the largest (but not most necessary) files immediately and see if that takes care of the problem.

Every once in a while, if you're checking around, you may notice that /var/adm/lastlog is gigantic. Theoretically, zeroing that out (catting /dev/null into it), should take care of your disk usage problem as it seems fairly obvious. Some of us would just leave it at that. The rest of us would check "df -k /var" again and notice that the percentage of partition space used is relatively the same. That doesn't seem to make any sense.

This is where the interesting part comes in. Solaris' implementation of lastlog has an interesting bug/feature that makes it seem larger than it is; but only some of the time.

The reason for this is that, while its size remains fairly static (about 24kb maximum), lastlog always indicates its size (when using "ls -l") relative to the user account id that last logged in (after the 24 Kb maximum is reached). The equation is roughly "the user account id number" multiplied by "28 bytes." So, when root logs in with a userid of "zero" (after you've zeroed out the file), it seems to grow to a size of 28 bytes (Yes, this is the minumum - and, yes, 28 times zero should equal zero ;) However, if you do an "ls -s" (to figure out the number of blocks) and a "du -k" (to figure out the size in Kb) on /var/adm/lastlog, you'll see that it's not really taking up all that much space. Below:

$ ls -l /var/adm/lastlog
-r--r--r-- 1 root root 28 Nov 19 17:51 lastlog
$ ls -s /var/adm/lastlog
2 /var/adm/lastlog
$ du -k /var/adm/lastlog
1 /var/adm/lastlog

If a user with a userid of 6504 logs in (after zeroing out the file) the block and Kb size will show the maximum (48 and 24, respectively), but "ls -l" reports:
$ ls -l /var/adm/lastlog
-r--r--r-- 1 root root 182112 Nov 19 17:53 lastlog

Crazy, yeah? But, interesting to know, and helpful, since you can avoid this file when trying to pare down the size of the /var partition.

As a caveat, the "fake" size reported by "ls -l" is only fake when lastlog is being manipulated by the Solaris Operating System in the manner in which it was specifically designed to be manipulated. If you copy that 500Mb file (or move it, tar it, etc) it pads all the "blank" space with NULLs and you end up having a file on your hands that really "is" insanely large!

, Mike

linux unix internet technology

Using Up All of a Disk Partition And Still Having 98% Free Space!

Yes, the title is somewhat misleading. You can't really use up all of any given disk partition and actually have 98% of that space still available for use. Not technically, anyway. This is something fun to try on anyone you like (or like playing pranks on) if you've got the time and the spare machine to do it.

You'd be surprised at how many Solaris admin's don't consider what's actually filling up the disk when you throw them a humdinger like this one. Even the one's who do still have to go an extra step. Here are the ingredients for the recipe:

1. Write a simple script to just make directories. You can write an infinite loop that executes "mkdir a;cd a" or get more complicated (to speed up the process) and write a little script that will cd into multiple subdirectories, exec itself and then re-run, ad infinitum. In any event, do this until your computer complains that there is no space left on the partition.

2. Check "df -k" to be sure, you should see something like:

Filesystem kbytes used avail capacity Mounted on
/dev/md/dsk/d5 8388608 119296 8269312 2% /mydir

that "2%" is the capacity it's at, so you have 98% of your disk free :) Too bad there's none left.

3. Now, since most admin's will figure this out (after using lsof, which doesn't really help), just confirm with "/usr/ucb/df -i" that the disk is, indeed, full to the brim.

Filesystem iused ifree %iused Mounted on
/dev/md/dsk/d5 494592 0 100% /mydir

I'm sure the astute observer has noticed that I'm simulating this information. I can't afford to do this for real right now ;)

So, this is the gist of it. You've used up all the available inodes, by creating a bunch of empty directories, but you've barely used up any disk space in process!

4. Now, to take it one step further (and further your enjoyment in the process), run this little Perl command line (assuming the top level directory you created on the /mydir partition was /mydir/newdir):

perl -e "unlink '/mydir/newdir';"

Now the top level directory, and just the top level directory, has been removed. The link to the inode no longer exists, making it impossible to cd into the directories below it, which still exist. To the naked eye it appears that there is nothing on /mydir that could possibly be causing the problem. Even utilities like "du" will report the correct total size (which is misleading) but not list the disk space usage for your non-existent directory when run as, say, "du -sk /mydir/*"

Eventually your co-worker and/or friend (maybe not anymore ;) will figure out, after doing an fsck, that the directory inode needs to be relinked and, if he's seen this before, will look in the /mydir/lost+found directory to find the missing "inode number" pointer and relink it manually.

That is why I highly recommend you remove this directory as well!

Enjoy your weekend. Remember - Please only do this in fun and, for your own safety, only to someone with a good sense of humor :)

, Mike

linux unix internet technology

Determining The Real Space Hogs On a Multiple-Overlay Mounted File System

This happens a lot to me, and probably most system administrators, at work. All the machines are being monitored (the "important" ones, anyway) 24x7. Of course, that's good because it's our job to make sure the machines keep running optimally and it would be a real drag to have to log in and check them manually.

However, no matter how sophisticated your monitoring system is, it's prone to giving you the usual headaches. A computer's only as smart as a computer, after all. On the one end of the spectrum, you'll get notified if the cpu so much as spikes under 10% idle for 1 minute of the day and on the other end, you won't find out that your machine is melting until you smell the smoke. In any event, some sort of automation is better than none!

Disk watching is something that's becoming more and more critical, in so far as performance assurance goes, as time marches on. What we used to store in MegaBytes, we're now storing in PetaBytes and it's still not enough space to store all that important music and video... I mean, customer data :P

The issue I'm addressing with today's bit of work, is how to deal with getting a clear assessment of disk space usage when you've got a multiple-overlay mounted file system. This may not be the proper industry term, so I'll explain that when I say this I mean a partition which has many other partitions mounted over it. For instance (all separate mount points):

/stuff
/stuff/IBM/aa
/stuff/IBM/bb
/stuff/oracle/db_files

...and so on. A lot of times, the overlain mounts number in the hundreds. So, in our instance, what do we do (in our role as administrators and blame-takers) to determine who's really using up all that space? Obviously, running a disk usage command like "du -sk" on the /stuff directory would be very time consuming, and it wouldn't accurately reflect what files/directories are eating up all the space on /stuff since most of the directories beneath /stuff are on entirely separate partitions!

The only way to really do it accurately is to ensure that you're only measuring the disk usage of files and directories that are on /stuff and not any of the overlain partitions. The little script below does just that. For your consideration:





This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/ksh

#
# 2007 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

trap 'rm tmpfile.$pid;echo "Caught Signal. Cleaning Up And Quitting...";exit 3"' 1 2 3 9 15

pid=$$
nicedate=`date "+%m/%d/%y`
date=`date +%m%d%y`

hostname=`hostname`

print ""
print "$hostname StuffCheck Output for $nicedate"
echo "---------------------------------------------------------------------"
print ""
printf "%-15s %-10s %-20s %-10s %-10s\n" Hostname Size Partition User Group
echo "---------------------------------------------------------------------"

if [ -d /stuff ]
then
cd /stuff
counter=0
ls -1d * 2>&1|while read x
do
if [ `df -k |grep "$x" >/dev/null 2>&1;echo $?` -ne 0 ]
then
if [ $counter -eq 0 ]
then
dir_array="$x"
let counter=$counter+1
else
dir_array="${dir_array}\t$x"
fi
fi
done
for y in `echo $dir_array`
do
du -dsk $y 2>&1
done|sort -rn|head -5|while read size partition
do
printf "%-15s %-10s %-20s %-10s %-10s\n" $hostname: $size $partition `ls -ld /stuff/$partition|/bin/awk '{ print $3}'` `ls -ld /stuff/$partition|/bin/awk '{ print $4}'`
done
else
print "No /stuff Directory found!!!"
fi

Note that the real time saver here is that we only run du on files and/or directories that we've already determined are not on any overlay mount (using df -k).

All in all, it's a fairly simple script to decipher, since I wrote it quickly and hardcoded a lot of things that could very well be variable (the partition name for instance) and tipped my hand to what I wanted to accomplish before the timer ran out by adding a "hostname" field to the output. But, I'm sure I'll end up making this more flexible as soon as the /things, /odds and /ends directories start getting overloaded :)

Take care,

, Mike

linux unix internet technology