Today, I thought we'd go for some lighter fare, so I put together another one of my simple shell scripts, specifically for Linux, that you can run on any system (although you do need some elevated privilege to get the correct information for a few of the commands) and print up a nicely formatted listing of system information (I tried to pick the stuff superiors tend to want to know).
If you run this in an SSH shell loop, it might just be a handy way to slap together a little system inventory to pacify the big guys ;) You can run it simply by calling it on the system you're running it on, like so:
./sysinfo.sh
Anyway, enjoy and best wishes. The work week is almost over :)
This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License#!/bin/bash
#
# sysinfo.sh
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#
echo -n "hostname: "
/bin/hostname
echo
echo -n "model: "
/bin/uname -pm
echo
echo -n "cpu count: "
/bin/dmesg|grep CPU|awk -F: '{print $1}'|sort -u|wc -l
echo
echo -n "disks online: "
fdisk -l|awk '{print $1}'|grep \/dev\/|sed 's/[1234567890]//'|sort -u|wc -l
echo
echo "disk types:"
echo
fdisk -l|awk '{print $1}'|grep \/dev\/|sed 's/[1234567890]//'|sed 's/\/dev\///'|sort -u|xargs -iboink grep "^boink" /var/log/dmesg
echo
echo "dns name and aliases:"
echo
nslookup `hostname`|grep Name;nslookup `hostname`|sed -n '/Alias/,$p'
echo
echo "Interfaces:"
echo
netstat -in|grep -v Kernel|grep -v Iface|grep -v lo|awk '{print $1}'|xargs -iboink cat /etc/sysconfig/network-scripts/ifcfg-boink|sed -n 1,2p
echo
echo "Access Restrictions:"
echo
if [ -f /etc/hosts.allow ]
then
cat /etc/hosts.allow
else
echo "No host based access restrictions in place"
fi
echo
echo -n "OS Release: "
uname -r
, Mike
linux unix internet technology
Thursday, January 31, 2008
Linux Shell Script To Gather Basic System Info
Wednesday, January 30, 2008
Recovering failed RAID disks on Linux
Hey there once more,
As I mentioned in yesterday's post regarding creating RAID disk sets we'll be looking at how to deal with RAID disk failures on Linux in today's post. Once we're done here, I think we can call it a wrap on this subject for a while (Are those sighs of relief I'm hearing? ;) At least until we come up with some good shell scripts to expedite some of this stuff.
Disk failure, on the hardware side, is too broad to cover in any great detail here, but the following basic steps should be followed (of course, as noted, your setup may require otherwise). The scenario here is that one of your disks has just gone "bad." It's beyond recovery.
1. If the disk is hot-swappable, simply remove it. If it isn't, you'll need to schedule downtime and remove the disk then. If the failed disk is your boot disk, you'll already have your downtime ;)
2. Replace the failed disk and restart your machine (and skip to step 6) if your failed disk isn't the boot disk.
3. If your failed disk is the boot disk, you'll next want to boot off of CD (We're using RedHat Linux AS), mount the root filesystem under a temporary mountpoint, and do the following:
host # mkdir /tmp/recovery
host # mount /dev/hda0 /tmp/recovery
host # chroot /tmp/recovery
host # grub --batch (This may take a while as grub probes and tries to guess where all of your drives are)
4. Once grub is finished probing, do the following at the "grub>" prompt:
grub> root (hd0,0)
root (hd0,0)
Filesystem type is ext2fs, partition type 0xfd
grub> setup (hd0)
setup (hd0)
...
Running "install /grub/stage1 (hd0) (hd0)1+16 p (hd0,0)/grub/stage2
/grub/grub.conf"... succeeded
grub> exit
5. Now take a second and verify that all is well while still running off of the CD, like so:
host # cat /boot/grub/device.map:
(fd0) /dev/fd0
(hd0) /dev/hda
host # df -k (should show the /dev/md0 mount)
host # umount /tmp/recovery
host # reboot (Be sure to set the grub device map for hd0 to /dev/hdc if /dev/hda has gone bye-bye)
6. Now that you have the disk physically replaced and you've booted back up, check the content of /proc/mdstat, like so:host # cat /proc/mdstat
Personalities : [raid1]
read_ahead 1024 sectors
...
md0 : active raid1 hda1[0]
307328 blocks [2/1] [U_]
...
unused devices: <none>
Again, the [2/1] listing indicates that one part of the mirror is not active in this two-way mirror set!
7. If this has been driving you crazy from the first post, we're now going to recreate the mirror so it's not incomplete (apologies for any stress-related reading injuries. Sometimes I go a long way to make a point ;). The first thing we'll do is repartition the disk, again, with fdisk (instructions in our previous post regarding creating RAID disk sets ) and we should end up with our partition table looking exactly the same:
host # fdisk -l /dev/hda (The partition table should look almost identical for /dev/hdc)Disk /dev/hda: 16 heads, 63 sectors, 77520 cylinders
Units = cylinders of 1008 * 512 bytes
Device Boot Start End Blocks Id System
/dev/hda1 * 1 6095 3071848+ fd Linux raid autodetect
/dev/hda2 6096 67047 30719808 fd Linux raid autodetect
/dev/hda3 67048 73142 3071880 fd Linux raid autodetect
/dev/hda4 75175 77206 1024096+ 82 Linux swap
8. As a matter of course, just to be sure, check /etc/raidtab to see what devices need to be mirrored with what devices:
host # cat /etc/raidtab
raiddev /dev/md0
raid-level 1
nr-raid-disks 2
chunk-size 64k
persistent-superblock 1
nr-spare-disks 0
device /dev/hda1
raid-disk 0
device /dev/hdc1
raid-disk 1
...
9. Now, we just need to add back all the partitions we lost:
host # raidhotadd -a /dev/md0 /dev/hda1
...
And keep tabs on the process by checking on /proc/mdstat:host # cat /proc/mdstat
Personalities : [raid1]
read_ahead 1024 sectors
...
md0 : active raid1 hda1[2] hdc1[1]
30716160 blocks [2/1] [_U]
[===========>..........] recovery = 45.9% (34790000/61432320) finish=98.7min speed=8452K/sec
unused devices: <none>
Once the RAID sync is done, you should be good to go! I'd reboot one more time, just to be sure. Especially if you were forced to take downtime in the first place :P
Take care,
, Mike
linux unix internet technology
Tuesday, January 29, 2008
Creating RAID disk sets on Linux
Today's post is a somewhat-continuation of yesterday's post on RAID disk monitoring. We looked at monitoring then, and (the reason I call this a somewhat-continuation) now we're going to take a step back and look at setting up those RAID disk sets (a simple mirror set in this example)in the first place ;) Note that all the examples I'm using today are taken from RedHat Linux AS and can be run in bash or any native shell. Scripting some of this out is near impossible, but someday soon I may give it a go.
In any event, once we've covered all this ground, we'll definitely look at how a handy shell script or two can save a whole lot of time on these processes :) So, off we go:
The first thing you'll want to do is to create your partitions on each physical disk, using the fdisk utility. I personally prefer it from the command line, so if you use Disk Druid or any other GUI interface, it should be easy enough to translate this to that so I don't have to over-explain.
The first thing we'll do is to create the disk partitions and assign them to an appropriate RAID device. The steps should roughly be the same as these:
1. Invoke fdisk at that command line.
2. Select "n" to add a new partition (anything from a slice of disk for mirroring two disks, to an entire disk for higher level RAID and mega-storage).
3. Once added, select "t" to change the partition flag and select "Linux Raid Autodetect" (selection "fd" on my console, possibly different on yours).
4. Select "a" to add a bootable flag to the partition, if necessary.
5. Save the new disk layout, and quit, by selecting "w"
6. Repeat as necessary
It's important, since we're doing this from the command line down-and-dirty, that we make sure that all partitions on both disks are "exactly" the same size. Unless your secondary mirror disk is larger, in which case you just need to make sure that you only use partitions larger than the originals to mirror to (It only makes sense. You can't copy a larger filesystem to a smaller one without losing data - The RAID utilities will discourage you from doing so, anyway).
Now, you'll need to put filesystems on all of those partitions. Generally, you should be able to take care of this using mkfs:
host # mkfs -t ext3 /dev/hda1
In the bizarre event that this fails, or you're using an older or different OS than I am, this should also work:
host # mkfs -t ext2 -j /dev/hda1
Now we'll use set up our /etc/raidtab file, with the following (assuming the same for all slices):host # raiddev /dev/md0
raid-level linear
nr-raid-disks 2
nr-spare-disks 0
chunk-size 64
persistent-superblock 1
device /dev/hda1
raid-disk 0
device /dev/hdc1
raid-disk 1
...
Then we'll initialize the RAID device, like so:
host # mkraid /dev/md0
Once again, per yesterday's post on RAID disk monitoring, we can check on the RAID device's progress by simply doing the following:
host # cat /proc/mdstat
and we'll know instantly if the RAID device is in a good or bad state, and how far along it is if it's still synching. You can use the above command to check status whenever you make a change to your RAID configuration for any valid RAID md devices.
If you ever want, or need, to remove any RAID partitions, you can do that like this (In this example, the hda1 slice is having issues on the md0 device):
host # raidsetfaulty /dev/md0 /dev/hda1
host # raidhotremove /dev/md0 /dev/hda1
Sometimes, as a failsafe (this depends on your situation and what device you're removing) you may want to re-install grub on your disk device, like so:
host # grub-install /dev/hda
And, finally for today, if you do end up having to remove a RAID mirror partition and it can be replaced inline (A whole disk in itself, for instance - possibly just a partition on a disk if bizarre filesystem corruption was the only issue and you've recreated the filesystem), do the following:
1. Add the new RAID devices as before (create, set filesystem type, assign same partition names if possible, etc) and check the status of the RAID groups. [2/2] means that the RAID group is ok, [2/1] means that the RAID group needs to be resynced (you'll note from yesterday that the RAID group we're working on is in a [2/1] state, with only one active mirror. If you noted that this could be, and should be, fixed - as well as the fact that this failure would drop that RAID partition because of this condition - you're sprinting ahead of my tutorial skills ;)host # cat /proc/mdstat
Personalities : [raid1]
read_ahead 1024 sectors
md0 : active raid1 hda1[0]
30716160 blocks [2/1] [U_]
...
unused devices: <none>
2. Determine your RAID group and mirrors from /etc/raidtab: host # cat /etc/raidtab
...
raiddev /dev/md0
raid-level 1
nr-raid-disks 2
chunk-size 64k
persistent-superblock 1
nr-spare-disks 0
device /dev/hda1
raid-disk 0
device /dev/hdc1
raid-disk 1
...
3. Now, you'll want to resync the RAID mirror sets that got goofed up when we had to replace the RAID parititions:
host # raidhotadd /dev/md0 /dev/hda1
4. Assuming you've lost more than 1 mirror, just repeat the procedure above as many times as necessary.
5. Lastly, assuming all went well, kick back and enjoy the view ;) - Note that Linux will normally queue resyncs and do them one at a time, so don't panic if you only see one out of 5 disk groups being worked on when you check this out!host # cat /proc/mdstat
...
md3 : active raid1 hda1[2] hdc1[1]
30716160 blocks [2/1] [_U]
[===========>..........] recovery = 45.9% (34790000/61432320) finish=98.7min speed=8452K/sec
...
unused devices: <none>
And that's it for today. In another near future post (probably tomorrow) we'll look at the fun involved with RAID disk failure ;)
Best Wishes,
Monday, January 28, 2008
A Few Linux RAID Disk Monitoring Tips
Hey there,
For today's post, I thought I'd put together a few Linux RAID disk monitoring tips. This list isn't meant to be too specific, but more of a catch-all of stuff you probably have to do over and over again. But sometimes not often enough that it burns into your brain ;)
At the very basic level, you can manage your disk partitions using fdisk, much like in Windows, although completely different (type "m" to get a menu listing of available commands once you bring this little utility up). You can use the "-l" option to just print out the partition menu and not use it in interactive mode:host # fdisk -l /dev/hda
Disk /dev/hda: 16 heads, 63 sectors, 77520 cylinders
Units = cylinders of 1008 * 512 bytes
Device Boot Start End Blocks Id System
/dev/hda1 * 1 6095 3071848+ fd Linux raid autodetect
/dev/hda2 6096 67047 30719808 fd Linux raid autodetect
/dev/hda3 67048 73142 3071880 fd Linux raid autodetect
/dev/hda4 75175 77206 1024096+ 82 Linux swap
If you want to display the status of your RAID devices, much the way metastat does on Solaris, it's much simpler. One of the great things about Linux is the way they've taken the /proc filesystem and turned into a really useful tool.
Entering the following will display the information you need. Note that this is a two way mirror with only one active mirror (I take the examples I can find on our network ;). Ideally, you'd want to have a [2/2] configuration for better HA.host # cat /proc/mdstat
Personalities : [raid1]
read_ahead 1024 sectors
md2 : active raid1 hda3[0]
62912 blocks [2/1] [U_]
md1 : active raid1 hda2[0]
153152 blocks [2/1] [U_]
md0 : active raid1 hda1[0]
307328 blocks [2/1] [U_]
If you're syncing up your RAID devices and want to check on the progress, just use the same command, like so:host # cat /proc/mdstat
Personalities : [raid1]
read_ahead 1024 sectors
md2 : active raid1 hdc3[1]
1020032 blocks [2/1] [_U]
md0 : active raid1 hdc1[1]
3068288 blocks [2/1] [_U]
md1 : active raid1 hda2[2] hdc2[1]
3068288 blocks [2/1] [_U]
[==>..................] recovery = 13.7% (894298/6136576) finish=10.2min speed=7833K/sec
unused devices:
Now, assuming all that is taken care of and you want to check out your RAID configuration, you'll just need to read your RAID configuration table (/etc/raidtab):host # cat /etc/raidtab
raiddev /dev/md0
raid-level 1
nr-raid-disks 2
chunk-size 64k
persistent-superblock 1
nr-spare-disks 0
device /dev/hda1
raid-disk 0
device /dev/hdc1
raid-disk 1
raiddev /dev/md1
raid-level 1
nr-raid-disks 2
chunk-size 64k
persistent-superblock 1
nr-spare-disks 0
device /dev/hda2
raid-disk 0
device /dev/hdc2
raid-disk 1
raiddev /dev/md2
raid-level 1
nr-raid-disks 2
chunk-size 64k
persistent-superblock 1
nr-spare-disks 0
device /dev/hda3
raid-disk 0
device /dev/hdc3
raid-disk 1
If you want to get more detailed with your analysis, there are (of course) built in commands to enable you to do so. lsraid is one of my favorites :)
To display a short listing of the md0 device, type the following:
# lsraid -A -a /dev/md0
[dev 9, 0] /dev/md0 8F88ACB0.7D06B4C4.FA677344.C3448700 online
[dev 3, 3] /dev/hda1 8F88ACB0.7D06B4C4.FA677344.C3448700 good
[dev 22, 3] /dev/hdc1 8F88ACB0.7D06B4C4.FA677344.C3448700 good
If you'd like to display a short listing of the RAID array that the disk hda1 belongs to (essentially the same output in this case), just type:
# lsraid -A -d /dev/hda1
[dev 9, 0] /dev/md0 8F88ACB0.7D06B4C4.FA677344.C3448700 online
[dev 3, 3] /dev/hda1 8F88ACB0.7D06B4C4.FA677344.C3448700 good
[dev 22, 3] /dev/hdc1 8F88ACB0.7D06B4C4.FA677344.C3448700 good
lsraid also comes in handy if you want to list out any faulty devices. This is something to consider putting in cron, or running regularly, so that you get notified of any failures in a timely fashion. No sense in having RAID's HA setup if you wait for all the components to fail before you replace any of them ;)
For a short listing, go with:
# lsraid -A -f -a /dev/md0
[dev 9, 0] /dev/md0 8F88ACB0.7D06B4C4.FA677344.C3448700 online
If you want, or need, all the gruesome details, lsraid will be happy to oblige. Just invoke it like this:lsraid -D -l -a /dev/md0
[dev 3, 3] /dev/hda1:
md version = 0.90.0
superblock uuid = 8F88ACB0.7D06B4C4.FA677344.C3448700
md minor number = 0
created = 1169715493 (Fri Jan 25 08:58:13 2007)
last updated = 1169933481 (Sun Jan 27 21:31:21 2007)
raid level = 1
chunk size = 64 KB
apparent disk size = 3068288 KB
disks in array = 2
required disks = 2
active disks = 2
working disks = 2
failed disks = 0
spare disks = 0
position in disk list = 0
position in md device = 0
state = good
[dev 22, 3] /dev/hdc1:
...
In a near future post (maybe tomorrow), we'll follow up with some tips on setting up your RAID arrays in Linux and walk through that process together.
Hope this helps you out some in the meantime :)
Cheers,
, Mike
linux unix internet technology
Sunday, January 27, 2008
Articles Up To Date On Linux Unix Local Hosting
Hey There,
I've finished re-tooling the last of my currently active articles on Linux, Unix, shell scripting, etc, and uploaded them to local storage.
As I mention, whenever I do this, I like to bring my articles in locally so they won't disappear when some editor somewhere finds a fault he/she neglected to harang me about earlier ;)
For today, I've posted both versions of an article on ways to beat certification tests :) There's not much more I can say about this. Even if you could write a shell script to automate the process, it might be quite a feat getting it to run on the PC's at the testing center. If you figure out how to do that discreetly, please email me. You'll get full credit for the idea, of course :)
Here they are - the two separate versions that may or may not be out there right now:
Beating Certification Tests - version 1
Beating Ceritification Tests - version 2
Hope you enjoy reading at least one of them and have a relaxing Sunday!
, Mike
linux unix internet technology
Saturday, January 26, 2008
Local Unix Linux Article Repository Updated!
Happy weekend,
As you may or may not know, I'm in the habit of submitting articles on Linux, Unix, shell scripting, etc, to article sites every now and again. Mostly to promote this blog, which shouldn't be a problem if you're reading this ;)
Now that we've got another socked away, I'm bringing it in-house. I do this for 2 reasons:
1. I have no idea when my online articles are going to be randomly deleted
2. Generally, I have to "edit for content" no matter where I submit. This is okay, although I must admit that it burns a little when I have to pour over an article to remove html tags and relevant links (against policy) only to have my words draped with 100's of advertisements for miracle cream ;)
For today, I've posted both versions of an article on ways to trick JumpStart into working across subnets without following the rules :) In a future post, we'll go into how to script out this kind of setup, so you can do it on the fly. Sometimes, when the seconds count, executing a shell script is a lot more comforting than methodically following a long set of rules ;)
Here they are - the two separate versions (Posting both because the two separate article sites mangled them in two separate ways):
Jumpstarting Across Subnets - version 1
Jumpstarting Across Subnets - version 2
Hope you enjoy reading one of them and have a great Saturday!
, Mike
linux unix internet technology
Friday, January 25, 2008
Linux Perl Script To Graph Out Paging, Memory and CPU
Hey there,
Yesterday's post has finally arrived. I've converted as much as I could from our previous posts into this one comprehensive script for RedHat Linux. I'm not going to overstuff this post with pictures, since the results you'll get here will look exactly like the ones in our previous posts for CPU, Paging and Memory graphing, except our Perl script today will create all 3 at once!
You may notice that the disk buffer reads and writes Graph has not been included here. This is on purpose and due to the fact that I couldn't find much more than opinion about what constituted the equivalent of Solaris cache reads/writes on Linux. The native sar didn't produce the exact same type of statistics, so I didn't want to throw out some garbage just for the sake of filling the page. As we look into this further, it seems that an entire post on how to determine and track Linux's buffer cache would be helpful, informative and quite lengthy all on it's own. I look forward to scripting that out in the near future :)
Today's script was put through its paces on a machine running RedHat AS2.1 (Pensacola) with sar from the sysstat RPM version 5.0.5-12.21as. I mention these specifics because its been my experience, in the past (when RedHat was free) that the column order would occasionally change when the sysstat RPM was updated. With that in mind, I've commented this script with the header notation for the column that I'm grabbing at each point in the script. This way, if it changes for you, all you'll need to do is change the array number to the one matching the data column we're graphing to keep this script pertinent.
These column notation comments (and all the other lines that varied enough from the Solaris script to necessitate changing) are noted in the script with:
# <-----
Again, this script can be run at any time, simply by calling it, since it interprets the sar data and creates the GIF Graphs when you run it, like so:
./linuxcpupagememgraph.pl <--- Worst script name ever? ;)
Over the weekend, we'll post on using these sorts of scripts in CGI and the security implications that go along with automating processes through the web browser (convenient, but sometimes dangerous!).
I hope you find this script helpful and flexible enough to use across however many different versions or flavors of Linux you run at your site.
Cheers,
This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License#!/usr/bin/perl
#
# linuxcpupagememgraph.pl
# Graph CPU, Paging and Memory
# statistics at will!
# Written specifically for Linux
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#
&graphsarB; # sar -g on Solaris # <-----
&graphsarr;
&graphsaru;
sub graphsarB { # <-----
use GIFgraph::bars;
open(SARG, "/usr/bin/sar -B|"); # <-----
@sarg = <SARG>;
close(SARG);
$my_graph = new GIFgraph::bars(800,600);
$my_graph->set(
x_label => 'Time',
y1_label => 'Number',
y2_label => 'Number',
title => 'Paging Activity by Server - host',
two_axes => 1,
y_min_value => 0,
y1_max_value => 20,
y2_max_value => 20,
y_tick_number => 5,
long_ticks => 1,
x_ticks => 0,
legend_marker_width => 24,
line_width => 10,
bar_spacing => 0,
gifx => 800,
gify => 600,
transparent => 1,
dclrs => [ qw( red green blue black) ],
);
$my_graph->set_legend( qw( Pages_Paged_Out Minor_Page_Faults-Buffer_Access Major_Page_Faults ) ); # <-----
$a = $b = $c = $d = 0;
foreach $line (@sarg) {
next if ( ($line !~ /:/) || ($line =~ /\//));
@line=split(" ", $line);
if ( ( $a % 12 ) != 0 ) {
$pandata0[$a] = undef;
} else {
$line[0] =~ s/:00$//;
$pandata0[$a] = "$line[0] $line[1]"; # <-----
}
if ( $line[3] == 0 ) { # <-----
$pandata1[$b] = 0;
} else {
$pandata1[$b] = $line[3]; # <----- pgpgout/s
}
if ( $line[3] == 0 ) { # <-----
$pandata2[$c] = 0;
} else {
$pandata2[$c] = $line[4]; # <----- fault/s
}
if ( $line[3] == 0 ) { # <-----
$pandata3[$d] = 0;
} else {
$pandata3[$d] = $line[5]; # <----- majflt/s
}
$a++;
$b++;
$c++;
$d++;
}
if ( ! $c ) {
@pandata0[0] = `date "+%H:%M"`;
@pandata1[0] = 0;
@pandata2[0] = 0;
@pandata3[0] = 0;
}
@data = (\@pandata0, \@pandata1, \@pandata2, \@pandata3);
$my_graph->plot_to_gif( "/your/data/dir/lsarB.gif", \@data ); # <-----
}
sub graphsarr {
use GIFgraph::lines;
open(SARR, "/usr/bin/sar -r|"); # <-----
@sarr = <SARR>;
close(SARR);
$my_graph = new GIFgraph::lines(800,600);
$my_graph->set(
x_label => 'Time',
y1_label => 'MegaBytes',
y2_label => 'MegaBytes',
title => 'Free Memory By Server - host',
y_min_value => 0,
y_tick_number => 5,
long_ticks => 1,
x_ticks => 0,
legend_marker_width => 24,
line_width => 10,
bar_spacing => 0,
gifx => 800,
gify => 600,
transparent => 1,
dclrs => [ qw( red green blue ) ],
);
$my_graph->set_legend( qw( Real_Memory_Available Swap_Memory_Available ) ); # <-----
$a = $b = $c = 0;
foreach $line (@sarr) {
next if ( ($line !~ /:/) || ($line =~ /\//));
@line=split(" ", $line);
if ( ( $a % 12 ) != 0 ) {
$pandata0[$a] = undef;
} else {
$line[0] =~ s/:00$//;
$pandata0[$a] = "$line[0] $line[1]"; # <-----
}
$pandata1[$b] = ($line[2]*8)/1000; # <----- kbmemfree
$pandata2[$c] = ($line[7]/2)/1000; # <----- kbswpfree
$a++;
$b++;
$c++;
}
if ( ! $c ) {
@pandata0[0] = `date "+%H:%M"`;
@pandata1[1] = 0;
@pandata2[2] = 0;
}
@data = (\@pandata0, \@pandata1, \@pandata2);
$my_graph->plot_to_gif( "/your/data/dir/lsarr.gif", \@data ); # <-----
}
sub graphsaru {
use GIFgraph::lines;
open(SARU, "/usr/bin/sar -u|"); # <-----
@saru = <SARU>;
close(SARU);
$my_graph = new GIFgraph::lines(800,600);
$my_graph->set(
x_label => 'Time',
y1_label => 'Percentage',
y2_label => 'Percentage',
title => 'CPU Usage By Server - host',
y1_max_value => 100,
y2_max_value => 100,
y_min_value => 0,
y_tick_number => 5,
long_ticks => 1,
x_ticks => 0,
legend_marker_width => 24,
line_width => 10,
bar_spacing => 0,
gifx => 800,
gify => 600,
transparent => 1,
dclrs => [ qw( red green ) ],
);
$my_graph->set_legend( qw( CPU_Utilization ) );
$a = $b = 0;
foreach $line (@saru) {
next if ( ($line !~ /:/) || ($line =~ /\//));
@line=split(" ", $line);
if ( ( $a % 12 ) != 0 ) {
$pandata0[$a] = undef;
} else {
$line[0] =~ s/:00$//;
$pandata0[$a] = "$line[0] $line[1]"; # <-----
}
$pandata1[$b] = 100-$line[7]; # <----- %idle
$a++;
$b++;
}
if ( ! $c ) {
@pandata0[0] = `date "+%H:%M"`;
@pandata1[1] = 0;
}
@data = (\@pandata0, \@pandata1);
$my_graph->plot_to_gif( "/your/data/dir/lsaru.gif", \@data ); # <-----
}
, Mike
linux unix internet technology
Thursday, January 24, 2008
A Unix And Linux Shell Script To Remind You Of Yesterday
For today's post, I thought we'd take a little break from all the graphing and harken back to a simpler time. Or, at least a time when my life was simpler ;)
Our script today is almost totally unnecessary in this day and age. If you use Perl's localtime function or know how to use the date command, this stuff is all taken care of for you. However, the script still stands as a good example of taking into account all the small things that go into something as simple as working with time and date variables that span the passing of a day.
Often, simple scripting constructs like these are overlooked, because they're built in to almost every Operating System and scripting language available at this point in time. However, we do use them almost every time we're tasked with automating menial administration tasks. For instance, if you want to roll the log file for any application, it's usually a good idea to do it just at midnight, at which point it's today, but you need to strip all of the values from your log file from yesterday and, for convenience, time stamp that chunked off log.
Our script today will absolutely positively work on RedHat Linux, Sun Solaris and every other flavor of Unix and/or Linux. It's about as basic as basic gets (although I did go with ksh/bash scripting to simplify the arithmetic and not confuse the issue with a whole bunch of cryptic expr calls).
This script is meant to be more of a function (or a file to be sourced into another script). I only included the bottom "echo" statements for illustrative purposes. You should remove those if you ever use this script, and I've included a legend at the bottom to show example values you can expect to get from the script's variables.
Have fun with this and check it out. If you can't avail yourself of any built-in date/time functions on your system, this might be of great use to you. Going through writing a script like this makes you appreciate the fact that you don't have to know all that much about the calendar or deal with all that messy stuff like "leap years," etc.
You can source in this file by saving it as whatever name you choose (let's say shelldate.h) and adding this line near the top of your script:
. /wherever/you/put/this/shelldate.h
Or you can use this as a function by just wrapping it in a function construct, like so:function shelldate {
CODE HERE
}
and then using it in your script like so:
shelldate
or
shelldate()
Best Wishes - We'll get back to porting those graph scripts to Linux on tomorrow's post :)
This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License#!/bin/ksh
###########################################################
# Dayback - For insertion into other #
# scripts. Takes all aspects of date #
# back one ( or more with slight #
# variation). #
###########################################################
# All variable output listed at bottom #
###########################################################
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#
###########################################################
###########################################################
# Main Variables #
###########################################################
SHORTALPHADAY=`date +%a`
LONGALPHADAY=`date +%A`
NUMBERDAY=`date +%d`
SHORTALPHAMONTH=`date +%b`
LONGALPHAMONTH=`date +%B`
NUMBERMONTH=`date +%m`
SHORTYEAR=`date +%y`
LONGYEAR=`date +%Y`
set -A SHORTALPHADAYS Mon Tue Wed Thu Fri Sat Sun
set -A LONGALPHADAYS Monday Tuesday Wednesday Thursday Friday Saturday Sunday
set -A SHORTALPHAMONTHS Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
set -A LONGALPHAMONTHS January February March April May June July August September October November December
###########################################################
# The Number Section #
###########################################################
###########################################################
# Numbered Days #
###########################################################
if [ $NUMBERDAY -eq 01 ]
then
if [ $SHORTALPHAMONTH == "Feb" -o "Apr" -o "Jun" -o "Aug" -o "Sep" -o "Nov" ]
then
YESTERNUMBERDAY=31
elif [ $SHORTALPHAMONTH == "Jan" -o "May" -o "Jul" -o "Oct" -o "Dec" ]
then
YESTERNUMBERDAY=30
elif [ $SHORTALPHAMONTH == "Mar" ]
then
# Check for leap year
let LEAPYEAR=${LONGYEAR}%4
if [ $LEAPYEAR -eq 0 ]
then
YESTERNUMBERDAY=29
else
YESTERNUMBERDAY=28
fi
fi
else
let YESTERNUMBERDAY=$NUMBERDAY-1
COUNT=`echo "$YESTERNUMBERDAY" |wc -m`
if [ $COUNT -eq 2 ]
then
YESTERNUMBERDAY=0${YESTERNUMBERDAY}
fi
fi
###########################################################
# Numbered Months #
###########################################################
if [ $NUMBERDAY -eq 01 ]
then
if [ $NUMBERMONTH -eq 01 ]
then
YESTERNUMBERMONTH=12
else
let YESTERNUMBERMONTH=$NUMBERMONTH-1
COUNT=`echo "$YESTERNUMBERMONTH" |wc -m`
if [ $COUNT -eq 2 ]
then
YESTERNUMBERMONTH=0${YESTERNUMBERMONTH}
fi
fi
else
YESTERNUMBERMONTH=$NUMBERMONTH
fi
###########################################################
# Numbered Years #
###########################################################
###########################################################
# Short Years #
###########################################################
if [ $NUMBERMONTH -eq 01 -a $NUMBERDAY -eq 01 ]
then
if [ $SHORTYEAR -eq 00 ]
then
let YESTERSHORTYEAR=99
else
let YESTERSHORTYEAR=$SHORTYEAR-1
COUNT=`echo "$YESTERSHORTYEAR" |wc -m`
if [ $COUNT -eq 2 ]
then
YESTERSHORTYEAR=0${YESTERSHORTYEAR}
fi
fi
else
YESTERSHORTYEAR=$SHORTYEAR
fi
###########################################################
# Long Years #
###########################################################
if [ $NUMBERMONTH -eq 01 -a $NUMBERDAY -eq 01 ]
then
let YESTERLONGYEAR=$LONGYEAR-1
else
YESTERLONGYEAR=$LONGYEAR
fi
###########################################################
# The Alpha Section #
###########################################################
###########################################################
# Alpha Days #
###########################################################
###########################################################
# Short Days #
###########################################################
i=0
while [ $i -ne 7 ]
do
if [ $SHORTALPHADAY == ${SHORTALPHADAYS[$i]} ]
then
if [ $SHORTALPHADAY == "Mon" ]
then
YESTERSHORTALPHADAY="Sun"
break
fi
let j=$i-1
YESTERSHORTALPHADAY=${SHORTALPHADAYS[$j]}
break
else
let i=$i+1
fi
done
###########################################################
# Long Days #
###########################################################
i=0
while [ $i -ne 7 ]
do
if [ $LONGALPHADAY == ${LONGALPHADAYS[$i]} ]
then
if [ $LONGALPHADAY == "Monday" ]
then
YESTERLONGALPHADAY="Sunday"
break
fi
let j=$i-1
YESTERLONGALPHADAY=${LONGALPHADAYS[$j]}
break
else
let i=$i+1
fi
done
###########################################################
# Alpha Months #
###########################################################
###########################################################
# Short Months #
###########################################################
if [ $NUMBERDAY -eq 01 ]
then
i=0
while [ $i -ne 12 ]
do
if [ $SHORTALPHAMONTH == ${SHORTALPHAMONTHS[$i]} ]
then
if [ $SHORTALPHAMONTH == "Jan" ]
then
YESTERSHORTALPHAMONTH="Dec"
break
fi
let j=$i-1
YESTERSHORTALPHAMONTH=${SHORTALPHAMONTHS[$j]}
break
else
let i=$i+1
fi
done
else
YESTERSHORTALPHAMONTH=$SHORTALPHAMONTH
fi
###########################################################
# Long Months #
###########################################################
if [ $NUMBERDAY -eq 01 ]
then
i=0
while [ $i -ne 12 ]
do
if [ $LONGALPHAMONTH == ${LONGALPHAMONTHS[$i]} ]
then
if [ $LONGALPHAMONTH == "January" ]
then
YESTERLONGALPHAMONTH="December"
break
fi
let j=$i-1
YESTERLONGALPHAMONTH=${LONGALPHAMONTHS[$j]}
break
else
let i=$i+1
fi
done
else
YESTERLONGALPHAMONTH=$LONGALPHAMONTH
fi
###########################################################
# Todays Variables: (Example) #
###########################################################
# SHORTALPHADAY = Thu #
# LONGALPHADAY = Thursday #
# NUMBERDAY = 01 #
# SHORTALPHAMONTH = Jan #
# LONGALPHAMONTH = January #
# NUMBERMONTH = 01 #
# SHORTYEAR = 98 #
# LONGYEAR = 1998 #
###########################################################
# Yesterdays Variables: (Example) #
###########################################################
# YESTERSHORTALPHADAY = Wed #
# YESTERLONGALPHADAY = Wednesday #
# YESTERNUMBERDAY = 31 #
# YESTERSHORTALPHAMONTH = Dec #
# YESTERLONGALPHAMONTH = December #
# YESTERNUMBERMONTH = 12 #
# YESTERSHORTYEAR = 97 #
# YESTERLONGYEAR = 1997 #
###########################################################
echo SHORTALPHADAY = $SHORTALPHADAY
echo LONGALPHADAY = $LONGALPHADAY
echo NUMBERDAY = $NUMBERDAY
echo SHORTALPHAMONTH = $SHORTALPHAMONTH
echo LONGALPHAMONTH = $LONGALPHAMONTH
echo NUMBERMONTH = $NUMBERMONTH
echo SHORTYEAR = $SHORTYEAR
echo LONGYEAR = $LONGYEAR
echo YESTERSHORTALPHADAY = $YESTERSHORTALPHADAY
echo YESTERLONGALPHADAY = $YESTERLONGALPHADAY
echo YESTERNUMBERDAY = $YESTERNUMBERDAY
echo YESTERSHORTALPHAMONTH = $YESTERSHORTALPHAMONTH
echo YESTERLONGALPHAMONTH = $YESTERLONGALPHAMONTH
echo YESTERNUMBERMONTH = $YESTERNUMBERMONTH
echo YESTERSHORTYEAR = $YESTERSHORTYEAR
echo YESTERLONGYEAR = $YESTERLONGYEAR
, Mike
linux unix internet technology
Wednesday, January 23, 2008
Perl Script To Graph Sar Cached Disk Reads and Writes
Note: Click the above picture to see it at 800x600.
Hey again,
For today's post, we're finally finishing up our four part series on graphing out sar in real time, that we last looked at yesterday. Today we'll be scripting out cached disk reads and writes on a server. This script goes in a little bit of a different direction than the previous three, as we're finally going to go hog-nuts and use bars and lines at the same time (not for the faint of heart ;)
This script was tested on Solaris, and the output from "sar -b" on RedHat is laid out in different order (so the array elements I use to plot the Graph values in this script may not give you the results you desire if you just run it straight-up in Linux). The script below is, again, written in Perl and will, run on Solaris Unix and RedHat Linux (Linux, again, with slight modification)
In our next post, we'll look at the modifications necessary to make all 4 of these scripts work on RedHat Linux. We'll also look at the basic concepts and what we're actually looking for, just in case the next version of the sysstat RPM changes the order of output again!
This script should be run on the host that you're doing the measuring on (since it runs sar at the time you invoke it), like so:
./sarcachegraph.pl
This graph is probably the most annoying out of the 4, which is why I saved it for last. Having cached reads at 100% is pretty much normal on Solaris, so I chose red to make it stand out even more than it would normally. My hope is that none of my superiors will want to look at this ever, and, if they do, they'll never want to again ;)
Remember, click on the picture above to see it at its actual resolution :)
Cheers,
This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License#!/usr/bin/perl
#
# Sarcachegraph.pl
#
# Graph Cached Disk Reads and Writes
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/3.0/us/">Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License</a>
#
use GIFgraph::mixed;
open(SARB, "/usr/sbin/sar -b|");
@sarb = <SARB>;
close(SARB);
$my_graph = new GIFgraph::mixed(800,600);
$my_graph->set(
x_label => 'Time',
y1_label => 'Percentage',
y2_label => 'Percentage',
title => 'Cache Reads and Writes by Server - host',
two_axes => 1,
y1_max_value => 100,
y2_max_value => 100,
y_min_value => 0,
y_tick_number => 5,
long_ticks => 1,
x_ticks => 0,
legend_marker_width => 24,
line_width => 10,
bar_spacing => 0,
gifx => 800,
gify => 600,
transparent => 1,
dclrs => [ qw( red green blue ) ],
types => [ qw( bars lines ) ],
);
$my_graph->set_legend( qw( Cache_Reads Cache_Writes ) );
$a = $b = $c = 0;
foreach $line (@sarb) {
next if ( ($line !~ /:/) || ($line =~ /\//));
@line=split(" ", $line);
if ( ( $a % 12 ) != 0 ) {
$pandata0[$a] = undef;
} else {
$line[0] =~ s/:00$//;
$pandata0[$a] = $line[0];
}
$pandata1[$b] = $line[3];
$pandata2[$c] = $line[6];
$a++;
$b++;
$c++;
}
if ( ! $c ) {
@pandata0[0] = `date "+%H:%M"`;
@pandata1[1] = 0;
@pandata2[2] = 0;
}
@data = (\@pandata0, \@pandata1, \@pandata2);
$my_graph->plot_to_gif( "/your/data/dir/sarb.gif", \@data );
, Mike
linux unix internet technology
Tuesday, January 22, 2008
Graphing System Paging Activity With Perl And Sar
Note: Click the above picture to see it at 800x600.
Hey There,
For today's post, we're following in the footsteps of yesterday's activity, but moving on to scripting out system paging activity on a server. This script goes in a little bit of a different direction than the previous two, as we'll be charting out these values with bars. There are only so many wavy lines a guy can take ;)
This script was written on a Solaris box, and the output from "sar -g" on RedHat is distributed slightly differently (so the array elements I use to plot the Graph values in this script wouldn't give you precise results if you just ran this on Linux. They might be interesting, though ;). The script below is, again, written in Perl and will, run on Solaris Unix and RedHat Linux (Linux, again, with slight modification - but we'll definitely hit on that in a follow-up post when we've covered all 4 major bases).
This script is written to be run on the host it's measuring, since it runs sar at the time you invoke it, like so:
./sarpagegraph.pl
This Graph didn't turn out to look very visually striking, but the use of bars to represent triplicate y values seemed appropriate. Charts like these cause eyestrain, and resulting headaches, commensurate with the amount of sampled data crammed into the picture ;)
Again, click on the picture above to see it at its actual resolution :)
Cheers,
This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License#!/usr/bin/perl
#
# Sarpagegraph.pl
#
# Graph Paging Activity In Almost Real Time
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/3.0/us/">Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License</a>
#
use GIFgraph::bars;
open(SARG, "/usr/sbin/sar -g|");
@sarg = <SARG>;
close(SARG);
$my_graph = new GIFgraph::bars(800,600);
$my_graph->set(
x_label => 'Time',
y1_label => 'Number',
y2_label => 'Number',
title => 'Paging Activity by Server - host',
two_axes => 1,
y_min_value => 0,
y1_max_value => 20,
y2_max_value => 20,
y_tick_number => 5,
long_ticks => 1,
x_ticks => 0,
legend_marker_width => 24,
line_width => 10,
bar_spacing => 0,
gifx => 800,
gify => 600,
transparent => 1,
dclrs => [ qw( red green blue black) ],
);
$my_graph->set_legend( qw( Pages_Paged_Out Pages_Added_To_Free_List Pages_Scanned ) );
$a = $b = $c = $d = 0;
foreach $line (@sarg) {
next if ( ($line !~ /:/) || ($line =~ /\//));
@line=split(" ", $line);
if ( ( $a % 12 ) != 0 ) {
$pandata0[$a] = undef;
} else {
$line[0] =~ s/:00$//;
$pandata0[$a] = $line[0];
}
if ( $line[2] == 0 ) {
$pandata1[$b] = 0;
} else {
$pandata1[$b] = $line[2];
}
if ( $line[2] == 0 ) {
$pandata2[$c] = 0;
} else {
$pandata2[$c] = $line[3];
}
if ( $line[2] == 0 ) {
$pandata3[$d] = 0;
} else {
$pandata3[$d] = $line[4];
}
$a++;
$b++;
$c++;
$d++;
}
if ( ! $c ) {
@pandata0[0] = `date "+%H:%M"`;
@pandata1[0] = 0;
@pandata2[0] = 0;
@pandata3[0] = 0;
}
@data = (\@pandata0, \@pandata1, \@pandata2, \@pandata3);
$my_graph->plot_to_gif( "/your/data/storage/dir/sarg.gif", \@data );
, Mike
linux unix internet technology
Monday, January 21, 2008
Free Memory Graph Generation With Perl And Sar
Note: Click the above picture to see it at 800x600.
Hey There,
For today's post, we're continuing from yesterday, but moving on to scripting out a graphological representation of the free memory usage on a server. Over the next few days I'll be posting the only other two variations I find that people really want to see all the time (and by people, I mean my managers). Then we'll go over setting them all up to run interactively in CGI so "people" will never bother you again. At least, not for that ;)
We'll also take a quick look, in retrospect, on how to "fix" all four scripts to run using Linux sar from the sysstat RPM. Maybe we'll just make this a graph generation script week. It won't compete with the Oscars, but it might resemble the closest thing we've ever come to a theme amongst a group of consecutive posts on this blog ;)
The script below is, again, written in Perl and will, run on Solaris Unix and RedHat Linux. The script was written specifically to run on Solaris and the output from "sar -r" on RedHat is in slightly different order (so the array elements I use to seed the Graph values in this script won't translate without some modification).
You can run it easily by just invoking it (as it is, it should be run on the host it's measuring, since it runs sar at the time you invoke it):
./sarmemgraph.pl
Note that there are some significant differences in this graph, compared to yesterday's post, including the use of multiple y values. Yay :)
Again, click on the picture above to see it at its actual resolution and enjoy :)
Enjoy,
This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License#!/usr/bin/perl
#
# sarmemgraph.pl
# Graph Free Memory in Pictorial Form
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/3.0/us/">Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License</a>
#
use GIFgraph::lines;
open(SARR, "/usr/sbin/sar -r|");
@sarr = <SARR>;
close(SARR);
$my_graph = new GIFgraph::lines(800,600);
$my_graph->set(
x_label => 'Time',
y1_label => 'MegaBytes',
y2_label => 'MegaBytes',
title => 'Free Memory By Server - host',
y_min_value => 0,
y_tick_number => 5,
long_ticks => 1,
x_ticks => 0,
legend_marker_width => 24,
line_width => 10,
bar_spacing => 0,
gifx => 800,
gify => 600,
transparent => 1,
dclrs => [ qw( red green blue ) ],
);
$my_graph->set_legend( qw( Real_Memory_Pages_Available Swap_Memory_Available ) );
$a = $b = $c = 0;
foreach $line (@sarr) {
next if ( ($line !~ /:/) || ($line =~ /\//));
@line=split(" ", $line);
if ( ( $a % 12 ) != 0 ) {
$pandata0[$a] = undef;
} else {
$line[0] =~ s/:00$//;
$pandata0[$a] = $line[0];
}
$pandata1[$b] = ($line[1]*8)/1000;
$pandata2[$c] = ($line[2]/2)/1000;
$a++;
$b++;
$c++;
}
if ( ! $c ) {
@pandata0[0] = `date "+%H:%M"`;
@pandata1[1] = 0;
@pandata2[2] = 0;
}
@data = (\@pandata0, \@pandata1, \@pandata2);
$my_graph->plot_to_gif( "/my/stats/directory/sarr.gif", \@data );
, Mike
linux unix internet technology
Sunday, January 20, 2008
Script to Generate CPU Graphs With Perl And Sar
Hey There, - Quick Note: Click on the above image to see it in its actual 800x600 resolution.
For today's post, I'll keep it short and sweet (due to the sleep deprivation suffered from my Friday/Saturday overnight work. We all need to sleep sometime. Anytime is good for me at this point ;)
Hope you enjoy the script below. It's written in Perl and will, theoretically, run on Solaris Unix and RedHat Linux.
Why only theoretically? Actually, it will work on both, but I wrote this script to run on Solaris and the output from "sar -u" on RedHat is in slightly different order (so the array elements I use to seed the Graph values in this script won't directly translate, but can be easily modified). You can run it easily by just invoking it (as it is, it should be run on the host it's measuring, since it runs sar at the time you invoke it):
./sarcpugraph.pl
Hope you enjoy this and can use the basic concept to help you generate all sorts of different graphs and charts from sar. I prefer the numbers, straight up, but some folks like to look at lines and bars and strangely colored pies. It's a free world ;)
The sample gif attached above is what your typical output will look like. Blogspot has trimmed it from the 800x600 resolution it was generated at. If you want to see it at that size, just click on it and the link should take you to the full version!
Enjoy,
This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License#!/usr/bin/perl
#
# Sarcpugraph.pl
# Make Fun CPU Graphs For People Who
# Like To See That Sort Of Thing
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#
use GIFgraph::lines;
open(SARU, "/usr/sbin/sar -u|");
@saru = <SARU>;
close(SARU);
$my_graph = new GIFgraph::lines(800,600);
$my_graph->set(
x_label => 'Time',
y1_label => 'Percentage',
y2_label => 'Percentage',
title => 'CPU Usage By Server - host',
y1_max_value => 100,
y2_max_value => 100,
y_min_value => 0,
y_tick_number => 5,
long_ticks => 1,
x_ticks => 0,
legend_marker_width => 24,
line_width => 10,
bar_spacing => 0,
gifx => 800,
gify => 600,
transparent => 1,
dclrs => [ qw( red green ) ],
);
$my_graph->set_legend( qw( CPU_Utilization ) );
$a = $b = 0;
foreach $line (@saru) {
next if ( ($line !~ /:/) || ($line =~ /\//));
@line=split(" ", $line);
if ( ( $a % 12 ) != 0 ) {
$pandata0[$a] = undef;
} else {
$line[0] =~ s/:00$//;
$pandata0[$a] = $line[0];
}
$pandata1[$b] = 100-$line[4];
$a++;
$b++;
}
if ( ! $c ) {
@pandata0[0] = `date "+%H:%M"`;
@pandata1[1] = 0;
}
@data = (\@pandata0, \@pandata1);
$my_graph->plot_to_gif( "/my/stats/directory/saru.gif", \@data );
, Mike
linux unix internet technology
Saturday, January 19, 2008
Installing System Boards on SunFire 6800 Servers
This week has been pretty intense (I'm actually in the middle of working another lovely Friday through Saturday maintenance window), so I thought I'd start out the weekend with a quick how-to regarding adding system boards to SunFire 6800 servers. Some of it is manual, but some can be scripted out to run on Solaris hands free. For the most part, when you're dealing with this stuff, I'd recommend not scripting it out. Odds are something important is on a machine that costs more than most people make in a year ;)
The first thing you'll want to do is connect to your system console, via SSH or Telnet (Or direct to the machine - your choice). You can find information on the basics of getting on to the 6800/6900 series machines in this previous post.
Here's a walkthrough of the commands you'd enter at the system console for "Domain B" if you were going to add, say, system board 3 (SB3).
First we'd start out by issuing the following commands at the "Domain B" system console, after having physically inserted the board. Note that all the showboards commands aren't absolutely necessary. I did include them however, because, even if the board you're adding never physically existed in your 6800/6900 SunFire box (Just a placeholder), it's almost always "owned" by a Domain, so it's best to delete the non-existent board before adding it. That doesn't seem to make a whole lot of sense, does it? Count yourself amongst the many ;)
host-sc0:B> showplatform -v
host-sc0:B> showboards -v
host-sc0:B> deleteboard SB3
host-sc0:B> showboards
host-sc0:B> addboard -d C SB3
host-sc0:B> showboards
host-sc0:B> poweron SB3
host-sc0:B> showboards
host-sc0:B> testboard SB3
host-sc0:B> showplatform -v
host-sc0:B> showboards -v
Now, that we've verified that the board is installed correctly and the system recognizes it, we get to the part that we could, theoretically, script out. There are so many variables here, that, maybe we should leave that up to Sun. If we all write them enough, maybe they'll include it in a future release of Solaris. It should raise the going price from an-unbelievable-amount-of-money to an-unbelievable-amount-of-money and 99 cents ;)
Now, we'll take a look at the following output - all should indicate your new system board (SB3) as active and available. This is done from the OS, as opposed to the "Domain Console"
host # usr/platform/sun4u/prtdiag -v
host # psrinfo
Then, we'll need to make sure that Solaris absolutely positively knows it's there by "configuring" it, like so:
host # cfgadm -c configure N0.SB3
This step can take upwards of 15 to 20 minutes to complete. If you've ever powered down or powered up a 6800, this will seem like the blink of an eye :P
The next step really is quick. We'll use cfgadm again to "connect" the board:
host # cfgadm -c connect N0.SB5
Then, again, we'll just verify that it's there and, to be doubly sure, log back into the system console and ensure that it's been installed without errors as well, with:
host # /usr/platform/sun4u/prtdiag -v
host # psrinfo
And, then connect to the system console and do:
host-sc0:B> showplatform -v
host-sc0:B> showboards -v
And you should be all set, with the board showing up as belonging to your Domain :)
In a future post, we'll look at how horribly things can wrong in this scenario. I'll hope and pray that you don't have to suffer that in this lifetime!
Cheers,
, Mike
linux unix internet technology
Friday, January 18, 2008
Script To Generate All Possible Passwords With Perl
In Yesterday's post, we took a look at how it's possible to use Perl to deal with generating random passwords, and that post was meant to link in with its preceding post. While it, technically, fulfilled that requirement, it took a while to elaborate on scripting out random passwords so I tried to keep on point, since it turned out to be a post's worth of information in itself.
With that in mind, in today's post, we're going to look at another part of the password puzzle: Generation of all possible passwords within a given numeric range. This might otherwise be referred to as brute-force password generation (which is the reason I wrote it using brute-force scripting :). What we're going to accomplish today is to create a simple "password generator" script that will allow us to generate all possible passwords (or all possible combinations of the 94 standard ASCII characters that make up valid passwords) up to an 8 character password. This has been tested on both Solaris Unix and RedHat Linux. One day, I'll stop compulsively typing that. If a script works in Perl, using that language's basics, its a given that it will work on most Unix/Linux systems you can install it on.
Please bear in mind that using this program is simple, but, depending on how you use it, it may take up all of your disk space and lots of your computer's time ;)
By way of explanation: This script produces one entry per line into a file, so that you could feed that file to a program like the one we looked at in this post.
Now, if we were to run it with the following arguments (we'll call it 8gen.pl for whatever goofy reason I pick the names I pick for my scripts ;)
host # ./8gen.pl 1 >OUTPUT <--- Note that, especially when running with larger number arguments, you should redirect the script output to a file, rather than "tee" it off, since viewing the tty/pty buffer as password combinations are generated could introduce a giant lag between completion of the script's execution and your getting a shell prompt back!
it would generate, in under a second, a file 94 lines long, with each line containing 1 of each of the 94 characters available (Please see the script's @94 array to check out all the variables. It's too insane to type over and over again ;), like this:host # wc -l OUTPUT
host # 94 OUTPUT
These numbers (both the size of the list in lines and the amount of time it takes to run the program) increase with each added level. They both increase exponentially which is more evident to the user if you're running an 8 character execution than if you're running a 2 character execution.
Let's say we decide we want to list out all possible 2 letter passwords. We would do this:
host # ./8gen.pl 2 >OUTPUT
And check out how big that gets (all combinations from aa to ??):host # wc -l OUTPUT
host # 8836 OUTPUT
Just to be sure we're right about this, let's check with the standard Linux and Unix "dc" utility by typing:
host # echo 94 2 ^ p | dc
host # 8836
and see that using 2 characters is actually the 94 original characters to the exponent of 2. This theory relates to every level you go up. So running it for all combinations of 4 letter passwords would be the 94 characters to the exponent of 4 as demonstrated here:host # echo 94 4 ^ p | dc
host # 78074896
host # ./8gen.pl 4 >OUTPUT
host # wc -l OUTPUT
host # 78074896 OUTPUT
And the time to wait for your list to generate? It may very well be exponentially longer. If you're going to run a 4 character execution, go grab a cup of coffee. If you're going to run an 8 character execution, depending on your machine, you might as well go home, get some rest and come back in to work the next day. Then it might be halfway done ;) I've never had the patience or desire to try and sneak this in at work and the boxes I have at home would take weeks to run this (94 to the 8th power will generate 6,095,689,385,410,816 unique passwords).
Example running on a SunFire v490 with 4 x 1350Mhz CPU and 32GB RAM:host # time ./8gen.pl 2 >OUTPUT
real 0m0.47s
user 0m0.04s
sys 0m0.38s
host # time ./8gen.pl 4 >OUTPUT
real 5m59.03s
user 5m37.10s
sys 0m2.86s
Once you've created a master password file, like so:
host # ./8gen.pl 8 >OUTPUT
you can use the OUTPUT file with the "pwdcheck" Perl script we introduced a few posts ago. Then, assuming you have the root privilege required to access and read your /etc/shadow file, you can set that on auto (perhaps trim the print statements so "pwdcheck" only prints out matches) and will eventually guess everyone's password. At this point, it's really only a matter of time, because you will be checking every possible combination of 94 possible characters in all 8 positions of the password. You can prove this to yourself simply by grepping any valid 8 character string from your OUTPUT file. It will be there (trust me)!
host # grep sN@gg3r$ OUTPUT
host # sN@gg3r$
...
Note that this also assumes that your password system is limited to an 8 character boundary, like Sun's standard. If you wanted to run up against more advanced password systems with better encryption and longer possible passwords, you'd just need to use your scripting abilities to modify both scripts slightly in order to achieve the same end result.
This Perl script should hopefully be a helpful tool in your constant fight against lame passwords. And, as always, though it can be used for less than ethical purposes, please understand that I only put this stuff out to try and help other admins like myself make their workplace more secure. Since any disgruntled lunatic can use these same methods to make your work-life miserable, you owe it to yourself to know how to do it, too.
Knowledge is power. If you know what your adversary knows, you're doing better than most :)
Enjoy,
This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License#!/usr/bin/perl
#
# 8gen.pl - generate all possible password
# combinations up to 8 characters
# Usage: 8gen.pl Password_Length (1 - 8)
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#
if ( $#{ARGV} < 0 ) {
print "\nUsage: $0 password_length\n";
print "Only 8 characters please. This is\n";
print "going to take a long time as it is!\n";
exit(1);
}
$pass_length = $ARGV[0];
@94 = qw(a b c d e f g h i j k l m n o p q r s t u v w x y z A B C D E F G H I J K L M N O P Q R S T U V W X Y Z ` 1 2 3 4 5 6 7 8 9 0 - = [ ] \ ; ' , . / ~ ! @ # $ % ^ & * ( ) _ + { } | : " < > ?);
if ( $pass_length < 1 || $pass_length > 8 ) {
print "Usage: $0 password_length\n";
print "Only 8 characters please. This is\n";
print "going to take a long time as it is!\n";
}
$a = $b = $c = $d = $e = $f = $g = $h = 94;
if ( $pass_length == 8 ) {
for ($a=0;$a<94;$a++) { for ($b=0;$b<94;$b++) { for ($c=0;$c<94;$c++) { for ($d=0;$d<94;$d++) { for ($e=0;$e<94;$e++) { for ($f=0;$f<94;$f++) { for ($g=0;$g<94;$g++) { for ($h=0;$h<94;$h++) {
printf("%s%s%s%s%s%s%s%s\n", $94[$a], $94[$b], $94[$c], $94[$d], $94[$e], $94[$f], $94[$g], $94[$h]);
} } } } } } } }
} elsif ( $pass_length == 7 ) {
for ($b=0;$b<94;$b++) { for ($c=0;$c<94;$c++) { for ($d=0;$d<94;$d++) { for ($e=0;$e<94;$e++) { for ($f=0;$f<94;$f++) { for ($g=0;$g<94;$g++) { for ($h=0;$h<94;$h++) {
printf("%s%s%s%s%s%s%s\n", $94[$b], $94[$c], $94[$d], $94[$e], $94[$f], $94[$g], $94[$h]);
} } } } } } }
} elsif ( $pass_length == 6 ) {
for ($c=0;$c<94;$c++) { for ($d=0;$d<94;$d++) { for ($e=0;$e<94;$e++) { for ($f=0;$f<94;$f++) { for ($g=0;$g<94;$g++) { for ($h=0;$h<94;$h++) {
printf("%s%s%s%s%s%s\n", $94[$c], $94[$d], $94[$e], $94[$f], $94[$g], $94[$h]);
} } } } } }
} elsif ( $pass_length == 5 ) {
for ($d=0;$d<94;$d++) { for ($e=0;$e<94;$e++) { for ($f=0;$f<94;$f++) { for ($g=0;$g<94;$g++) { for ($h=0;$h<94;$h++) {
printf("%s%s%s%s%s\n", $94[$d], $94[$e], $94[$f], $94[$g], $94[$h]);
} } } } }
} elsif ( $pass_length == 4 ) {
for ($e=0;$e<94;$e++) { for ($f=0;$f<94;$f++) { for ($g=0;$g<94;$g++) { for ($h=0;$h<94;$h++) {
printf("%s%s%s%s\n", $94[$e], $94[$f], $94[$g], $94[$h]);
} } } }
} elsif ( $pass_length == 3 ) {
for ($f=0;$f<94;$f++) { for ($g=0;$g<94;$g++) { for ($h=0;$h<94;$h++) {
printf("%s%s%s\n", $94[$f], $94[$g], $94[$h]);
} } }
} elsif ( $pass_length == 2 ) {
for ($g=0;$g<94;$g++) { for ($h=0;$h<94;$h++) {
printf("%s%s\n", $94[$g], $94[$h]);
} }
} elsif ( $pass_length == 1 ) {
for ($h=0;$h<94;$h++) {
printf("%s\n", $94[$h]);
}
}
, Mike
linux unix internet technology
Thursday, January 17, 2008
Perl Random Password Generator For Linux and Unix
As we noted in yesterday's post, we took a look at how it's possible to use Perl on Linux and Unix to try and find whether or not any given password exists within a system's /etc/shadow file.
In today's post, we're going to look at another part of the password puzzle: Password generation. This is slightly different than sequential password generation (which we'll look at tomorrow), in that the passwords that are generated are intended to annoy ;) In other words, the probability that any of these passwords could be found in a shadow password file is highly unlikely. And, if you did find one, I don't know if it would be grounds for penalty. That user would have an unbelievable secure password under any other circumstance!
This code has been tested on Solaris Unix and RedHat Linux and works almost every time (I can't speak to the infinite, but it hasn't failed me yet ;)
This script makes use of Perl's built-in rand function, which works in much the same way as the standard Unix or Linux random function does. The rand function is fed some slightly random data (certainly not good enough to seed any viable encryption) and generates random ASCII characters.
I wrote this up so that it can be invoked simply by its name, like so:
./pwdgen
and requires no argument. This script will pump out 8 character garbage passwords, like this:
O[W291,A
21&V-*4Q
('$\9@:<
YU="M1A>
;<<WG@(>
...
and it will go on forever until you either kill it or it finds that it has created a duplicate. Check it out, enjoy it and feel free to improve upon it. A simple change can make it so it only spits out a certain number of random passwords.
If you use it creatively, you can incorporate it with other Linux and Unix shell scripts to provide input for password cracking programs. Although, as I mentioned above, if I found any of these passwords among my users' passwords, I certainly wouldn't complain ;) Tomorrow we'll look at a way to generate real passwords (kind of like JTR does) that can be fed to the script we looked at yesterday to do some in-house brute-force password cracking against your shadow database file. Again; this sort of shell scripting is only recommended for security enhancement!
Best Wishes,#!/usr/bin/perl
#
# pwdgen - Create insane 8 character passwords to
# encourage users to beef up their password strength
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#
for ( $i = 1; $i > 0 ; $i++ ) {
$random = rand('netstat -a'*10000);
$count=0;
$newpassword = "";
while ( $count < 8 ) {
$tester = rand('netstat -a')*100;
if ( $tester < 33 ) {
$tester += 33;
} elsif ( $tester > 93 ) {
$tester -= 7;
}
$newpassletter = sprintf("%c", $tester);
$newpassword .= $newpassletter;
$count++;
}
$newpassword =~ s/ ?//g;
if ( exists $passwords{$newpassword} ) {
print "Found a pattern on step $i with password $newpassword\n";
exit;
} else {
$passwords{$newpassword} = $i;
}
print "$newpassword\n";
}
, Mike
linux unix internet technology
Wednesday, January 16, 2008
Perl Password Cracker For Linux and Unix
As you may recall, from a few postings ago, we took a look at wrapping a popular password cracker in Unix and/or Linux shell scripting for ease of use.
In today's post, we're actually going to do the password cracking ourselves!
This code has been tested on Solaris Unix and RedHat Linux and is 100 percent guaranteed to be somewhat entertaining ;) Again, I feel obligated to note that this software is for recreational use only and is actually intended to help system administrators find weak passwords in order to maximize security (In tomorrow's post, we'll look at a way to help with generating new passwords that will have everyone on your watch begging to be able to create their own ;)
This script makes use of Perl's built-in crypt function, which works in much the same way as the standard Unix or Linux crypt function does. Since we haven't yet figured out how to break the level of encryption you, hopefully, have on your systems, this program could also be considered a password guesser.
I wrote this up so that it can be invoked simply by its name, like so:
./pwdcheck
and requires no argument. This version prompts for input. It could be easily modified to accept command line arguments, but I left that alone for now, since it's not the main aim of the script. This script is also very simple, as I wanted to demonstrate the shell script concept of password guessing in its purest form. If you choose to try and find a single password you would type the name in when prompted. For a list of passwords, you would type in the name of the list (absolute or relative) with passwords in it (one to a line). Then the script will check your /etc/shadow file for any passwords that match your guess.
It's no lie that this method of password cracking is far less sophisticated than the methods used by products like John The Ripper, etc. However, it does demonstrate how the crypt function works and what really goes on at the guts of all of those types of programs (JTR, for instance, uses fastcrypt and does its own word manipulation outside of the "guessing" routine).
Hope you enjoy this and find some use for it. Tomorrow, we'll look at a script that will produce random passwords (great for helping out with system administration) that you can use for input to this script.
Cheers,
This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License#!/usr/bin/perl
#
# pwdcheck - Simple password checker.
# Works with password file input, one
# password per line, or one password
# at a time. Must be root to execute!
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#
#
print "Will you be checking one password or a list of them? [P or L]\n";
chomp($answer=<stdin>);
$answer=~ tr/A-Z/a-z/;
if ($answer eq "p") {
print "password to check?\n";
chomp($guess=<stdin>);
print "Do you keep your password entries in passwd,\n";
print "shadow, or another file? [P, S or O]\n";
chomp($reply=<stdin>);
$reply=~ tr/A-Z/a-z/;
if ($reply eq "s") {
open(SHADOW, "</etc/shadow");
} elsif ($reply eq "p") {
open(SHADOW, "</etc/passwd");
} elsif ($reply eq "o") {
print "What's the absolute pathname of the file?\n";
chomp($pathname=<stdin>);
if ( -f $pathname) {
open(SHADOW, "<$pathname");
} else {
print "I don't think it's there...\n";
exit;
}
} else {
print "Can't find that anywhere...\n";
exit;
}
@SHADOW=<SHADOW>;
foreach $LINE (@SHADOW) {
@line=split(/:/, $LINE);
$passwd{$line[1]} = $line[0];
}
foreach $PART (@SHADOW) {
@stab=split(/:/, $PART);
if (crypt($guess, $stab[1]) eq $stab[1]) {
print "Got $guess for user $passwd{$stab[1]}!\n";
$yes = 1;
}
}
if ( $yes eq 1) {
exit;
} else {
print "...Doesn't look like it...\n";
}
} elsif ($answer eq "l") {
print "Name of word file?\n";
chomp($passfile=<stdin>);
print "Absolute pathname of passwd, shadow or\n";
print "other relevant file?\n";
chomp($shadow=<stdin>);
open(SHADOW, "<$shadow");
@SHADOW=<SHADOW>;
foreach $LINE (@SHADOW) {
@line=split(/:/, $LINE);
$passwd{$line[1]} = $line[0];
}
close(SHADOW);
open(PWDFILE, "<$passfile");
@passfile=<PWDFILE>;
foreach $guess (@passfile) {
$guess=~ s/\s//g;
print "Checking for $guess ...\n";
open(SHADOW, "<shadow");
foreach $PART (@SHADOW) {
@stab=split(/:/, $PART);
if (crypt($guess, $stab[1]) eq $stab[1]) {
print "Got $guess for user $passwd{$stab[1]}!\n";
open(PD, ">>Passwords");
print PD "Login: $passwd{$stab[1]}\t\tPassword: $guess\n";
close(PD);
}
close(SHADOW);
}
}
} else {
print "Which part didn't you get?\n";
}
exit;
, Mike
linux unix internet technology
