Tuesday, November 18, 2008

Bash One-Liner Script To Produce Somewhat-Fancy Output Of Who's On Your Linux Or Unix Box

Hey there,

Today we're going to shoot out a quick one liner script that can come in handy from time to time. It doesn't fall under the "necessary" category (or, maybe not even the "useful" one ;), but it's nice to have for those times when you just don't feel like doing any extra typing or expending the effort required to separate the wheat from the chaff on a screen full of garbage output. That is, as I've always understood, one of the main reasons to script things out in the first place. The other reasons, like automation, efficiency, etc, are pretty much ignored here, even though we will be automating a repetitive action and that, by definition, will make our waste of company time more efficient. No matter how you look at them, automation and efficiency always seem to be a good thing ;)

This "script" is really just a simple pipe-slam so that we can figure out who's on the machine with us (my preference is for a script, but - as you'll soon see - this could just as easily be an alias, etc). Sure, we could just type "w" or "who," (and "w" does form the base of this bash one-liner script), but we'd like to have this whole thing look more personal. And by personal, I mean, we'd like to know who else is on other than us by UID or login name. The functionality (or relative merit) of this script depends heavily on the way in which users were created on your machine in the first place. If no "comment" (or "gecos") field is populated for users on your system, this command-chain is completely worthless.

For instance, assuming we had a user, with a login name of user1, logged in at the same time as us, with an entry that looked like this in /etc/passwd:

user2:x:32678:99::/home/user2:/bin/bash

and our /etc/passwd entry looked like this:

user1:x:32677:99:Mike The SysAdmin:/home/user1:/bin/bash

the command would pull no useful information for user2. So, if the output from w looked like this:

host # w
13:49:12 up 8 days, 3:41, 2 users, load average: 0.00, 0.14, 0.43
USER TTY LOGIN@ IDLE JCPU PCPU WHAT
user2 pts/0 08:44 5:03m 0.01s 0.01s -bash
user1 pts/1 09:05 0.00s 0.04s 0.00s w


our command would only produce output like this:

host # ./wdo.sh
Mike The SysAdmin


Whereas, if we added a comment field (perhaps via usermod -c "Bob The Guy Who Used To Be Anonymous" user2 or the chfn command), we would get nice output like the following:

host # ./wdo.sh
Mike The SysAdmin
Bob The Guy Who Used To Be Anonymous


and so forth, with users who had multiple sessions open being reported more than once. This script was written to give us a warm, fuzzy feeling, so seeing Bob's name five times is no big deal. If you want to use it for profiling purposes on a more heavily trafficed machine, there's always "sort."

Note, of course, that your version of Unix or Linux may produce slightly different output for the "w" command and xargs' -i flag may be "-I" (also, if, for some bizarre reason, your login names have spaces in them, just be sure to double quote the variable you define via xargs with the -i or -I flag. This should never happen). Basically, what I'm saying is, this script probably won't run correctly, exactly as it comes, on every single distribution of Linux and Unix out there :)

Enjoy the neighborhood watch :)

Cheers,

#!/bin/bash

#
# wdo.sh - More comments than code :)
#

w|awk '{print $1}' |xargs -ibing grep bing /etc/passwd|awk -F":" '{print $5}'



, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Monday, November 17, 2008

Bash Cable, Dish and Local TV Listings Script For Linux Or Unix

Hey There,

IMPORTANT NOTE: I'm beginning to think that this script should be put on my SourceForge page and maintained there. It's not that I mind doing these weekly updates, but I feel like I'm writing the same thing over and over again, where I could be offering you, the reader, more value by producing different scripts in its place. If you have a few seconds, please answer the short poll on the right hand side of the blog. If public opinion wants this project worked out on the blog, then so be it. If you're all getting sick of these updates, I can move this to my SourceForge page, as a project, and produce more original content for our Monday script posts.

For this week's Monday Linux/Unix bash shell script continuation, we're following up on our script from last week. If you liked that one, please revisit last week's cabletv.sh script, as this one introduces some features which (while nice) introduce a bit of extra run time. This week's improvements include making the script output more readable (Thanks to Michael Seeley for this suggestion) and, finally, fixing the am/pm (or 12 to 1) hump that would throw the results out of chronological order. As usual, please skip the following paragraph, unless you're interested in all the other Monday scripts we've cranked out to date.

Our previous web-based Bash scripts, in backward chronological order, include our posts on accessing Wikipedia, accessing the Farmer's Almanac, accessing the International Dictionary, checking out the world's weather, spewing out famous quotations on pretty much any subject, doing encyclopedia lookups, accessing the online Thesaurus, translating between different languages and, of course, using the online dictionary.

This week's improvements don't involve any actual extra functionality (as the, still in progress, configuration file loading and creation process will), but they do make things look better and read more sensibly, although this comes at a cost (see below).

The first improvement is in the readability of the script. Now, when you get your results, you'll be able to easily differentiate the channels from the program listings. The picture below shows a comparison of the older version (at the top) and the updated version (at the bottom):

Click on the picture below and be sure to have on your 3D specs ready ;)

old and new listings displays

The other improvements gets us over quiet a few errors involving the zero hour, incorrect times being auto-generated and ensuring correct chronological sorting of listings when crossing from 12 to 1 (am/pm or pm/am):

Click on the pictures below and enjoy the sensible passage of time ;)

chronological tv listings

Of course, getting the listings in chronological order has imposed a stiff penalty (although I'm sure it's because of the way in which I implemented it). Leaving everything in Unix time actually speeds up the sort process, but then running awk's strftime() call on each entry as it's printed has (to my way of thinking) disastrous repercussions. Of course, I'll be spending some time working this out. If you run "time" against the old version of the script (assuming my lineup with approximately 8 or 900 channels), the old version ran at this speed to produce all results in the maximum time frame (3 hours):

real 0m27.846s
user 0m6.835s
sys 0m4.725s


With all the extra frills, that number (for the exact same output - although with the fixes in place), runs at this pace:

real 3m30.538s
user 0m44.129s
sys 1m17.421s


a full 3 minutes longer! I've narrowed it down to the repetitive strftime() calls, and you can see that most of the processing time is in "sys," which means I'm forcing my kernel to work harder than it should. The implementation of date conversion in awk is the actual issue, and I'll need to convert that to simpler time-conversion via the Gnu date command, although this causes some issues with formatting that need to be worked out. I figured you'd want something that worked (if even more slowly) at this point, rather than something that ran fast and spewed garbage to your terminal.

And here's something that might make you feel better about the 3 minutes. Check out the time it takes to do every line using sed, grep and date without messing with the basic structure of the output:

real 19m31.859s
user 3m48.153s
sys 7m44.685s


Mamma mia! ;)

Until next time (possibly as a project on my SourceForge page :)

Cheers,


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/bash

#
# cabletv.sh - Get your local regular and HD Tv listings
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#
# TODO switch to enhanced getopt to deal with variable argument switches
# TODO add config file
# TODO check for bad provider code
# TODO rewrite unix to "pretty" time conversion to speed up returns
# ------------------------------
# DONE End Display Time For Previous day when start time is in zero hour
# DONE check for bad zip code
# DONE added optional time increments - up to 3 hours - for display
# DONE tightened up code using indirect variable references
# DONE date may now be entered in quotes numeric or alpha - e.g. 11/9/08 11/09/2008 "Nov 9 2008" "November 9th 2008" etc
# DONE time may now be entered as 10pm 10 (with am/pm defaulting to system time) 10:00am 23:00 etc
# DONE added error checking for invalid date and time entry - stop before querying online guide
# DONE fixed issue with 00:xx time causing script failure
# DONE added extra error checking
# DONE added better formatting of lineup for easier readability - Thanks to Michael Seeley for the suggestion!
# DONE You have entered an invalid ZIP or postal code
# DONE am to pm or pm to am ends up not in chronological order
# DONE 12 to 1 a/p ends up not in chronological order
#

function usage()
{
echo
echo "Usage: $0 ...all options are optional [-h for help]"
echo "[-z USZipCode] [-p LocalCableOrDishProviderID]"
echo "[-t Time] [-d date] [-n to leave out HD channel info]"
echo "[-s ListingTimeSpan 1 for just now, 2 for an additional half"
echo "hour, 3, 4, 5, 6, in half hour increments, and 7 for 3 hours"
echo "...defaults to 7 since this is the online listing default]"
echo
exit 1
}

while getopts t:z:hs:nd:p: option
do
case $option in
t)
opt_time="$OPTARG"
;;
d)
opt_date="$OPTARG"
;;
z)
opt_zip="$OPTARG"
;;
n)
opt_nohd=1
;;
p)
opt_provider="$OPTARG"
;;
s)
opt_timespan="$OPTARG"
;;
h)
usage
;;
*)
usage
;;
esac
done

wget=/usr/bin/wget
pager=/usr/bin/more

if [ ! "$opt_date" ]
then
nicedate=`date "+%m/%d/%y"`
else
nicedate=$opt_date
fi

if [ ! $opt_timespan ]
then
opt_timespan=7
elif [ $opt_timespan -lt 1 -o $opt_timespan -gt 7 ]
then
usage
fi

if [ ! $opt_time ]
then
hour=`date "+%I"`
minute=`date "+%M"`
ampm=`date "+%p"`
if [ $hour -lt 12 ]
then
if [ $minute -lt 30 ]
then
nicetime="${hour}:00 AM"
else
nicetime="${hour}:30 AM"
fi
else
if [ $minute -lt 30 ]
then
nicetime="${hour}:00 PM"
else
nicetime="${hour}:30 PM"
fi
fi
if [ $hour == "00" ]
then
cli_time=${hour}:${minute}
else
cli_time=${hour}:${minute}${ampm}
fi
else
cli_time=$opt_time
fi

date_ok=`date -d "$cli_time" +%s 2>&1`
if [[ "$date_ok" =~ "invalid date" ]]
then
echo
echo "INVALID TIME ENTERED: $opt_time"
echo "Be sure to include full am or pm"
echo "and do not add am or pm to 24 clock entries"
echo
usage
fi

ampm=`echo $opt_time|egrep 'a|p|m' 2>&1`
result2=$?
if [ $result2 -eq 0 ]
then
date_ok2=`echo $opt_time|egrep 'a|p'|grep m 2>&1`
result3=$?
if [[ $result3 -ne 0 ]]
then
echo
echo "INVALID TIME ENTERED: $opt_time"
echo "Be sure to include full am or pm"
echo "and do not add am or pm to 24 clock entries"
echo
usage
fi
fi

cli_time2=`date -d "$cli_time" +%s`
let fromtime1=$cli_time2*1000
let fromtime2=$fromtime1+1800000
let fromtime3=$fromtime2+1800000
let fromtime4=$fromtime3+1800000
let fromtime5=$fromtime4+1800000
let fromtime6=$fromtime5+1800000
let fromtime7=$fromtime6+1800000
if [ ! $opt_timespan ]
then
opt_timespan=7
fi

timespan_relay="$fromtime1"
timespan_counter=2
timespan_limit=$opt_timespan

while [ $timespan_limit -gt 1 ]
do
nextup_time=$(eval "echo \$$(echo fromtime${timespan_counter})")
timespan_relay="$timespan_relay $nextup_time"
let timespan_counter=$timespan_counter+1
let timespan_limit=$timespan_limit-1
done

from_timespan=$(eval "echo \$$(echo fromtime${opt_timespan})")
cli_time_end=`echo $from_timespan|awk '{printf("%20s",strftime("%I:%M %p - %x", $0 / 1000))}'`

if [ ! $opt_nohd ]
then
opt_nohd=0
fi

if [ ! $opt_zip ]
then
echo -n "Enter Zip Code: "
read zip_code
else
zip_code=$opt_zip
fi

$wget -nv -O - "http://tvlistings.zap2it.com/tvlistings/ZBChooseProvider.do?method=getProviders&zipcode=${zip_code}" 2>&1|sed -e :a -e 's/<[^>]*>/ /g;/</N;//ba' |sed -e '/^[ \t]*$/d' 2>&1|grep "You have entered an invalid ZIP or postal code" >/dev/null 2>&1
zip_ok=$?

if [ $zip_ok -eq 0 ]
then
echo
echo "INVALID ZIP CODE ENTERED: $opt_zip"
echo
usage
fi

if [ ! $opt_provider ]
then
$wget -nv -O - "http://tvlistings.zap2it.com/tvlistings/ZBChooseProvider.do?method=getProviders&zipcode=${zip_code}" 2>&1|sed 's/<.*&lineupId=\([^&]*\)\">/\1 \c/'|sed -e :a -e 's/<[^>]*>/ /g;/</N;//ba' |sed -e '/^[ \t]*$/d' |sed -e '1,/Choose Your Provider/{x;$!d;x;q;};x;$G' -e '/document.write/,$d' -e "s/'/'/" -e 's/&/\&/' -e 's/^[ \t]*//;s/[ \t]*$//'|sed '/\([A-Za-z0-9_][A-Za-z0-9_]*:[-X]\)/ {
N
s/ *\n/ \t/g
}'
echo "Enter Provider ID [e.g. \"IL57303:-\" \"4DTV:-\" \"IL12561:X]\""
read tv_provider
else
tv_provider=$opt_provider
fi

echo
echo "Television Listings for $nicedate from $cli_time to $cli_time_end"
echo

last=1

if [ $opt_nohd -eq 1 ]
then
for x in $timespan_relay
do
$wget -nv -O - "http://tvlistings.zap2it.com/tvlistings/ZCGrid.do?fromTimeInMillis=${x}&method=decideFwdForLineup&zipcode=${zip_code}&setMyPreference=false&lineupId=${tv_provider}" 2>&1|sed 's/<.*&sch=\([^&]*\)&[^>]*>/\1 /'|sed -e :a -e 's/<[^>]*>/ /g;/</N;//ba' |sed -e '/^[ \t]*$/d' |sed -e '1,/Forgotten password/d' -e '/isFavoritesAvailable/,$d'|sed -e '/[ECMP][SD]T/,+6d' -e "s/'/'/" -e 's/&/\&/' -e '/zc.getAdFrame/d' -e 's/^[ \t]*//;s/[ \t]*$//'| sed -n '/^[0-9][0-9]*$/,+2p'|sed 's/^\([0-9][0-9]*\)$/\n\1/'|sed '/^[0-9][0-9]*$/ {
N
N
s/ *\n/\t/g
}'|awk '{ if ( $1 ~ /^[0-9]/ ) {printf("%-4s %-8s ", $1,$2);for (i=4;i<NF+1;i++) {printf("%s ", $i)};printf("*** Start - %s", $3);print ""}}'
done|sort -t'-' -k1,1n -k2,2n | uniq|while read one two;do if [ $one -eq $last ]; then echo "$one $two"|awk '{aline=$NF;$NF="";printf("%s %s\n", $0, strftime("%x - %I:%M %p", aline/1000)) }'|while read one two three;do echo " $three";done;last=$one;else echo "$one $two"|awk '{aline=$NF;$NF="";printf("%s %s\n", $0, strftime("%x - %I:%M %p", aline/1000))}'|while read one two three;do echo " $one $two";echo " $three";done;last=$one;fi;done|$pager
else
for x in $timespan_relay
do
$wget -nv -O - "http://tvlistings.zap2it.com/tvlistings/ZCGrid.do?fromTimeInMillis=${x}&method=decideFwdForLineup&zipcode=${zip_code}&setMyPreference=false&lineupId=${tv_provider}" 2>&1|sed 's/<.*&sch=\([^&]*\)&[^>]*>/\1 /'|sed -e :a -e 's/<[^>]*>/ /g;/</N;//ba' |sed -e '/^[ \t]*$/d' |sed -e '1,/Forgotten password/d' -e '/isFavoritesAvailable/,$d'|sed -e '/[ECMP][SD]T/,+6d' -e "s/'/'/" -e 's/&/\&/' -e '/zc.getAdFrame/d' -e 's/^[ \t]*//;s/[ \t]*$//'| sed -n '/^[0-9][0-9]*\.*[0-9]*$/,+2p'|sed -e 's/^\([0-9][0-9]*\.*[0-9]*\)$/\n\1/'|sed '/^[0-9][0-9]*\.*[0-9]*$/ {
N
N
s/ *\n/\t/g
}'|awk '{ if ( $1 ~ /^[0-9]/ ) {printf("%-4s %-8s ", $1,$2);for (i=4;i<NF+1;i++) {printf("%s ", $i)};printf("*** Start - %s", $3);print ""}}'
done|sort -t'-' -k1,1n -k2,2n | uniq|while read one two three;do three_one=`echo "$three"|sed -e 's/.*\*\*\* Start - //' -e 's/^[ \t]*//;s/[ \t]*$//'`; let three_two=$three_one/1000; three_three=`echo "$three"|sed 's/^\(.*\*\*\* Start - \).*/\1/'`;three_four=`echo $three_two|awk '{printf("%s", strftime("%x - %I:%M %p", $0)) }'`;if [ $one -eq $last ];then echo " $three_three $three_four";last=$one;else echo "$one $two";echo " $three_three $three_four";last=$one;fi;done|$pager
fi

exit 0


, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Sunday, November 16, 2008

Fake Unix and Linux Advisory - The /dev/null Vulnerability

Hey again,

Here's a little something from 2003 (Remember way back then, when the future looked bright and almost anything seemed possible? ...me neither ;)

This is a humorous little fake CERT-like advisory concerning the implementation of /dev/null on all Unix and Linux operating systems and the disastrous effects of ignoring the problem. The bit I've put here was pulled from Ian's Humor Pages, which you might want to check out and find all the other funny stuff on there :)

Hope you enjoy it. I'm going away now to appease the "simple syndication" beast. Is this short enough for you, oh mighty and vainglorious Google RSS? ;)

Cheers,





From: lsloof@cirt.us (Dr. Lirpa Sloof)
Newsgroups: alt.security,comp.security.unix,comp.os.linux.security
Subject: CIRT Advisory CA-2003-0401: /dev/null Vulnerability
Date: 01 Apr 2003 08:21:28 GMT
Organization: Computer Incident Response Team (CIRT)
Message-ID: <advisory-ca-2003-0401@announce.cirt.us>
NNTP-Posting-Host: nimbus.thunder.net


CIRT Advisory CA-2003-0401 /dev/null Vulnerability
Computer Incident Response Team

Systems Affected

* Windows CE, ME, NT, 2000, XP, 98, 95
* Linux (all distributions)
* BSD-derived Operating Systems
* Solaris
* IRIX
* HP-UX
* Digital/Compaq Tru64 Unix
* AIX
* other Unix-compatible systems and Unix-compatibility libraries

Overview

There is a vulnerability in /dev/null on Unix systems, Unix-compatible
systems and those with Unix or POSIX compatibility libraries (including
Windows) that can be exploited to cause a denial-of-service condition
and could cause hardware damage to some systems in isolated cases.
Though rare, the possibility of hardware damage is the primary reason
why this advisory is being categorized as urgent.

I. Description

A vulnerability has been discovered with the algorithm most commonly
used to implement /dev/null on Unix and Unix-compatible systems which
can be used to cause damage to other software connected to it. In some
cases, this software damage can also trigger hardware damage. These
vulnerabilities can be exploited by a local user already logged into
the system for a denial-of-service. If used in conjunction with
remote exploit attacks, this could allow a remote attacker, worm or
virus to cause hardware damage on some systems. In even more rare
circumstances there are possibilities of bodily injury in bystanders.

II. Impact

The contemporary method of /dev/null drivers is described as the "high
suction algorithm" in comparison with the replacement that vendors have
made available for their systems. If a malicious user uses a program
with low-resistance logic to connect /dev/null back into itself,
the device goes critical and can be used for destructive purposes.
Once the /dev/null device driver enters a critical state, programs with
low-resistance logic will break, be consumed by /dev/null and expose
their standard input to the full force of /dev/null itself. Some examples
which have been verified in labs include the following:
* Programs which are consumed by /dev/null become permanent entry points
to /dev/null afterward.
* If standard input is redirected from any regular file, it will be
"sucked dry" and left empty. File permissions do not prevent loss
of data.
* If standard input is redirected from a directory, all the files and
directories within it will be sucked dry, recusrively removing an
entire directory tree.
* If standard input is redirected from a pipe or named pipe, it will
expose the full force of the critical state /dev/null to the program
on the other end of the pipe. As with direct linkage to /dev/null,
if the program contains logic too weak to resist the suction, it
will be consumed and permanently become a portal to /dev/null itself.
* If standard input is redirected from a keyboard device, the keyboard
will implode, crushing the keys. This has the possibility to cause
minor lacerations if anyone is typing on the keyboard at the time of the
implosion.
* If standard input is redirected from a mouse device, it will pop like
a weasel in a microwave oven.
* If standard input is redirected from a CRT monitor driver, the device
is unaffected because it already contains a vaccuum.
* If standard input is redirected from a disk driver, the drive will
be erased and the lubrication removed from the bearings. In lab tests
the disk platter has exited the drive at high speed in a random
direction. Note that if this occurs in a data center environment,
the platter is likely to embed itself in other computer hardware.
There is a risk of injury if a disk platter or other shrapnel from
the self-destructing disk drive should hit any person.
* If standard input is redirected from a network device, the results
have been very unpredictable. The effects appear to be mainly
confined within the local area network (LAN). In all cases, all
packets are sucked off the network.
* An ethernet hub is too weak to resist the suction and becomes a
vaccuum port for /dev/null.
* All ethernet switches and broadband network interfaces are immune
to the effects.
* In one home where a user-modified digital video recorder (DVR) device
was connected to the network, the existing recordings were erased.
But the device was running Linux so its own /dev/null created enough
suction that the pressure differential caused no further damage.
* In another home where a refrigerator was connected to the network,
all the food inside became freeze-dried.
* The most violent reaction was found where a home automation system
was connected to the network. The vaccuum came in contact with the
central air vents and sucked the air out of the house. Everyone
got out safely. But an infestation of termites in the house was
entirely suffocated.

III. Solution

Note that many of the mitigation steps recommended below may have
significant impact on your everyday home or office operations. Use
appropriate caution but also ensure that any changes made based on the
following recommendations will not unacceptably affect your ongoing use
of your computer and occupancy of your building unless you're certain
that a great danger exists in your circumstances.

Apply a patch from your vendor

Appendix A contains information provided by vendors for this advisory.
Please consult this appendix to determine if you need to contact
your vendor directly.

Appendix A. - Vendor Information

This appendix contains information provided by vendors for this
advisory.

Microsoft

The same patch as the fix for the March 17 buffer overflow in the
Windows Core DLL also fixes the high-suction algorithm. Note that
Windows systems are particularly vulnerable to the high-suction
algorithm because all the programs on Windows have weak logic and
are unable to resist the suction. Also, Microsoft has said that
there will be no patch for NT4 because they are unable to build a
new copy of the OS from source code. So NT4 not only sucks, but
it also blows.

Red Hat

Upgrade to kernel kernel-2.4.18-27.7.x (for RedHat Linux 7.x),
kernel-2.4.18-27.8.0 (for RedHat Linux 8.0) or later.

RedHat Linux 9.0 is not vulnerable.

SCO (Caldera)

Upgrade to OpenLinux 3.1.1, UnixWare 7.1.1, OpenUnix 8.0.0
or later.

Debian GNU/Linux

Upgrade to kernel kernel-image-2.4.18* or later for your system.

Mandrake Linux

Upgrade to kernel-2.4.19.32mdk-1-1mdk.* (for Mandrake Linux 9.0,
Corporate Server 2.1) or later.

Mandrake Linux 9.1 is not vulnerable.

SuSE Linux

Upgrade to kernel-2.4.19-20030324 updates (for SuSE Linux 8.1),
kernel-2.4.18-20030324 (for SuSE Linux 7.1-8.0) or later.

SuSE Linux 8.2 is not vulnerable.

Slackware Linux

For Slackware Linux, contact your vendor.

FreeBSD

Upgrade to FreeBSD 5.0 or later.

NetBSD

Upgrade to NetBSD-1.6.1 or later.

OpenBSD

Upgrade to OpenBSD 3.3 or later.

Solaris

Apply patch 114356-01 (Solaris 9), patch 114356-01 (Solaris 8)
or later.

IRIX

Apply update 20030201-01-P.

HP

Apply patch SSRT2322_2341_2384_2412_2439 (HP/Compaq Tru64 Unix).
For HP-UX, contact your vendor.

AIX

For AIX, contact your vendor.





, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Saturday, November 15, 2008

XKCD - Linux, Unix And Other Funny Cartoons

Hey there,

Hope your Saturday is going okay and that you're reading this in the evening or late afternoon (since, if you're like me, that's breakfast time ;) To start off this week's Linux and Unix humor section, I went looking around all 4 corners of the earth for something that made me laugh (at the very least, on the inside ;) and I ran back over a site that I'd actually used one cartoon from in a previous humor post. The site's name is XKCD and, after spending a while browsing through the tons of content on the site, I found that I liked way too many of the cartoons to include them all here. As some of you may have noticed, I have a tendency to try and include as much material as possible on my local page (with proper attribution to the content's creator and a link back to their site). This backfired on me last weekend when it made my RSS feed grow past the Google boundary and I had to muck around with that and start limiting the number of items to keep it alive (That's just for now. As soon as last weekend's humor posts fall off naturally, I'll be removing the size restriction so that the feed includes as much back-content as possible).

Anyway, I hope you enjoy the following comics. Some have to do with Linux and Unix, others have to do with computing concepts and still others have to do with... I have no idea ;) I have no problem admitting when I don't get a joke. It's actually really easy to tell because I don't even fake a laugh like I do for the bad ones that I understand but don't find amusing.

Have a safe Saturday, and we'll see you tomorrow with more "yuks" from the web that just happens to be as wide as the world ;)

Cheers,



Please click on any of the below images if you have trouble reading them. The increase in the laugh-factor should be commensurate with the relative picture growth :)













, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Friday, November 14, 2008

Basic Veritas Cluster Server Troubleshooting

Hey There,

For the end of the week, we're going to continue with the theme of sparse-but-hopefully useful information. Quick little "crib sheets" (preceding by paragraphs and paragraphs of stilted ramblings by the lunatic who pens this blog's content ;) For this Friday, we're going to come back around and take a look at Veritas Cluster Server (VCS) troubleshooting. If you're interested in more specific examples of problems, solutions and suggestions, with regards to VCS, check out all the VCS related posts from the past year or so. Hopefully you'll be able to find something useful in our archives, as well. These simple suggestions should work equally well for Unix as well as Linux, if you choose to go the VCS route rather than some less costly one :)

And, here we go again; quick, pointed bullets of info. Bite-sized bits of troubleshooting advice that focus on solving the problem, rather than understanding it. That sounds awful, I know, but, sometimes, you have to get things done and, let's face it, if it's the job or your arse, who cares about the why? Leave that for philosophers and academics. Plus, since you fix problems so fast, you'll have plenty of time to read up on the ramifications of your actions later ;)

The setup: Your site is down. It's a small cluster configuration with only two nodes and redundant nic's, attached network disk, etc. All you know is that the problem is with VCS (although it's probably indirectly due to a hardware issue). Something has gone wrong with VCS and it's, obviously, not responding correctly to whatever terrible accident of nature has occurred. You don't have much more to go on than that. The person you receive your briefing from thinks the entire clustered server set up (hardware, software, cabling, power, etc) is a bookmark in IE ;)

Now, one by one, in a fashion that zigs on purpose, but has a tendency to zag, here are a few things to look at right off the bat when assessing a situation like this one. Perhaps next week, we'll look into more advanced troubleshooting (and, of course, you can find lots of specific "weird VCS problem" solutions in our VCS archives)

1. Check if the cluster is working at all.

Log into one of the cluster nodes as root (or a user with equivalent privilege - who shouldn't exist ;) and run

host1 # hastatus –summary

or

host1 # hasum <-- both do the same thing, basically

Ex:

host1 # hastatus -summary

-- SYSTEM STATE
-- System State Frozen

A host1 RUNNING 0
A host2 RUNNING 0

-- GROUP STATE
-- Group System Probed AutoDisabled State

B ClusterService host1 Y N OFFLINE
B ClusterService host2 Y N ONLINE
B SG_NIC host1 Y N ONLINE
B SG_NIC host2 Y N OFFLINE
B SG_ONE host1 Y N ONLINE
B SG_ONE host2 Y N OFFLINE
B SG_TWO host1 Y N OFFLINE
B SG_TWO host2 Y N OFFLINE


Clearly, your situation is bad: A normal VCS status should indicate that all nodes in the cluster are “RUNNING” (which these are). However, it should also show all service groups as being ONLINE on at least one of the nodes, which isn't the case above with SG_TWO (Service Group 2).

2. Check for cluster communication problems. Here we want to determine if a service group is failing because of any heartbeat failure (The VCS cluster, that is, not another administrator ;)

Check on GAB first, by running:

host1 # gabconfig -a

Ex:

host1 # gabconfig -a
GAB Port Memberships
===============================================================
Port a gen 3a1501 membership 01
Port h gen 3a1505 membership 01


This output is okay. You would know you had a problem at this point if any of the following conditions were true:

if no port “a” memberships were present (0 and 1 above), this could indicate a problem with gab or llt (Looked at next)

If no port "h" memberships were present (0 and 1 above), this could indicate a problem with had.

If starting llt causes it to stop immediately, check your heartbeat cabling and llt setup.

Try starting gab, if it's down, with:

host1 # /etc/init.d/gab start

If you're running the command on a node that isn't operational, gab won't be seeded, which means you'll need to force it, like so:

host1 # /sbin/gabconfig -x

3. Check on LLT, now, since there may be something wrong there (even though it wasn't indicated above)

LLT will most obviously present as a crucial part of the problem if your "hastatus -summary" gives you a message that it "can't connect to the server." This will prompt you to check all cluster communication mechanisms (some of which we've already covered).

First, bang out a quick:

host1 # lltconfig

on the command line to see if llt is running at all.

If llt isn't running, be sure to check your console, system messages file (syslog, possibly messages and any logs in /var/log/VRTSvcs/... - usually the "engine log" is worth a quick look) As a rule, I usually do

host1 # ls -tr

when I'm in the VCS log directory to see which log got written to last, and work backward from there. This puts the most recently updated file last in the listing. My assumption is that any pertinent errors got written to one of the fresher log files :) Look in these logs for any messages about bad llt configurations or files, such as /etc/llttab, /etc/llthost and /etc/VRTSvcs/conf/sysname. Also, make sure those three files contain valid entries that "match" <-- This is very important. If you refer to the same facility by 3 different names, even though they all point back to the same IP, VCS can become addled and drop-the-ball.

Examples of invalid entries in LLT config files would include "node numbers" outside the range of 0 to 31 and "cluster numbers" outside the range of 0 to 255.

Now, if LLT "is" running, check its status, like so:

host # lltstat -wn <-- This will let you know if llt on the separate nodes within the cluster can communicate with one another.

Of course, verify physical connections, as well. Also, see our previous post on dlpiping for more low-level-connection VCS troubleshooting tips.

Ex:

host1 # lltstat -vvn
LLT node information:
Node State Link Status Address
0 prsbn012 OPEN
ce0 DOWN
ce1 DOWN
HB172.1 UP 00:03:BA:9D:57:91
HB172.2 UP 00:03:BA:0E:F1:DE
HB173.1 UP 00:03:BA:9D:57:92
HB173.2 UP 00:03:BA:0E:D0:BE
1 prsbn015 OPEN
ce3 UP 00:03:BA:0E:CE:09
ce5 UP 00:03:BA:0E:F4:6B
HB172.1 UP 00:03:BA:9D:5C:69
HB172.2 UP 00:03:BA:0E:CE:08
HB173.1 UP 00:03:BA:0E:F4:6A
HB173.2 UP 00:03:BA:9D:5C:6A


host1 # cat /etc/llttab <-- pardon the lack of low-pri links. We had to build this cluster on the cheap ;)

set-node /etc/VRTSvcs/conf/sysname
set-cluster 100
link ce0 /dev/ce:0 - ether 0x1051 -
link ce1 /dev/ce:1 - ether 0x1052 -
exclude 7-31
host1 # cat /etc/llthosts
0 host1
1 host2
host1 # cat /etc/VRTSvcs/conf/sysname
host1


If llt is down, or you think it might be the problem, either start it or restart it with:

host1 # /etc/init.d/llt.rc start

or

host1 # /etc/init.d/llt.rc stop
host1 # /etc/init.d/llt.rc start


And, that's where we'll end it today. There's still a lot more to cover (we haven't even given the logs more than their minimum due), but that's for next week.

Until then, have a pleasant and relaxing weekend :)

Cheers,

, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Thursday, November 13, 2008

Getting CPU Information From Various Flavours Of Linux And Unix

Hey there,

Today, in keeping with yesterday's theme of covering a fairly specific topic and trolling it around the seamy underbelly of the world of Unix and Linux, we're going to take a look at how to grab CPU information from your Unix or Linux box. I try to cover the major distros here, but (of course) my resources are limited and (except for the HP-UX example below) I've personally run and verified all of the results of the commands put out in this post today. If you crave additional information about your particular Operating System and the CPU(s) on it, check out some of our earlier posts on CPU usage, monitoring, etc, for all sorts of Unix/Linux varieties. Hopefully, you'll find some useful information in the growing archives of this sprawling morass of letters and numbers (by which I, of course, mean this blog ;)

On to today's business! For each OS, we'll be listing the OS name, followed by a practical line of code that you should, hopefully, be able to cut-and-paste onto your own system to get your specific information. Also, for each, we'll list the actual output that my trials produced, except for the HP-UX exception already noted.

1. Listing out CPU information on Solaris (Note that these "virtual" CPU's are actually real):

host # psrinfo -v| awk 'BEGIN{cpua=0;cpub=0}{if ( $0 ~ /Status.*processor/ ) {print $0;cpua=1} else if ( cpua == 1 ) {printf"%s ", $1;cpua=0;cpub=1} else if ( cpub == 1 ) {penult=NF-1;print $penult,$NF;cpub=0}}'|sed 's/,$//'
Status of virtual processor 0 as of: 11/12/2008 13:53:47
on-line 650 MHz
Status of virtual processor 1 as of: 11/12/2008 13:53:47
on-line 650 MHz


2. Listing out CPU information on SUSE and/or RedHat Linux:

host # cat /proc/cpuinfo|egrep 'processor|model name'|awk -F":" 'BEGIN{i=0}{if ( ! i%2 ) {printf"%s %d: ", $1,$2;i++} else {print $2;i=0}}'

processor 0: Intel(R) Xeon(TM) CPU 3.40GHz
processor 1: Intel(R) Xeon(TM) CPU 3.40GHz
processor 2: Intel(R) Xeon(TM) CPU 3.40GHz
processor 3: Intel(R) Xeon(TM) CPU 3.40GHz


3. Listing out CPU information on AIX 5.x:

host /# pmcycles -m
CPU 0 runs at 1200 MHz
CPU 1 runs at 1200 MHz
CPU 2 runs at 1200 MHz
CPU 3 runs at 1200 MHz
CPU 4 runs at 1200 MHz
CPU 5 runs at 1200 MHz
CPU 6 runs at 1200 MHz
CPU 7 runs at 1200 MHz


4. Listing out CPU information on HP-UX (as generally as possible). This bit of code is sewn together from exemplary commands on SysDigg's HP-UX CPU Info Page, since I can't get my hands on an HP-UX box right now, and the information gathering process is more convoluted than any of the preceding distro's (We're not even going to begin to talk about the new Itanium processors and/or differentiate between virtual and physical CPU's/Cores, etc. This is for a HP9000/800 model box):

host # pc=`ioscan -k |grep processor |wc –l `
host # pt=$(grep -i $(model |tr "/" " " \|awk '{print $NF}') \/usr/sam/lib/mo/sched.models |awk '{print $NF}')
host # ps=`echo "itick_per_usec/D" | adb /stand/vmunix /dev/mem | tail -1`
host # echo "Processor count: $pc - CPU Type: $pt - Speed: ${ps}Mhz"
Processor count: 2 - CPU Type: PA8700 - Speed: 750Mhz


Phew... Hopefully, the HP-UX information is correct (if not, please write in as I'd be glad to credit anyone with the correct answer if they can get to an HP-UX machine before I can :)

And, that's that. Quick and painless today ;) I promise to chew your virtual ear off tomorrow as I rant and rage for no apparent reason other than to possibly foment passive-aggressive revolution amongst the working class. Actually, whatever I have to write about tomorrow probably won't be "that" moving ;)

Cheers,

, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Wednesday, November 12, 2008

Broad Usage Of ipcs And ipcrm On Linux And Unix

Hey there,

A wise man once said to me: "Mike, " he said; "If you always remember to keep your mind focused and follow the simple and logical path of progression in every area of your life, you will accomplish many great things." As I pondered his words, he added the following punctuation, which has since become a part of the definition of "me." He added: "Of course, you'll eventually die with the realization that you missed out on 99% of your life and you'll most probably be miserable and bored at the end of both the literal and figurative day. Well before you reach your expiration date your milk will have already turned into cottage cheese. Leave me now and go do something either irrational, contrary or both, but just go. And have fun!" He was probably the most upbeat nihilist I've ever met ;)

And today, to commemorate his entertaining, and often confusing, apocalyptic outlook on life, I'm going to make our post about something that has almost no relation whatsoever to any of our posts so far this week, this month or this year, and attack two commands that vary in options and output on almost every distro of Linux or Unix you'll ever pick up: ipcs (Inter-Process Communication Status) and ipcrm (Inter-Process Communication message queue, semaphore set or shared memory segment Remover). I salute, and dedicate this post to, you, Captain Trips (he never told me his real name. It was more than he could bear ;)

Hopefully this attempt to capture the essence of using ipcs and ipcrm (along with reading the man pages on your system) will make the whole experience easy to get into and fun to customize :) If you're at all interested, we did a post a while back on cleaning up Solaris shared memory that looks at a very specific use of IPC commands on a single Unix distribution.

As a special reminder, since we're covering such broad ground and trying to make this post accessible to users of the widest variety of ipcs and ipcrm versions out there, distro-specific extended options will not be covered in this post!

1. ipcs:

Basic Usage: This command is used to identify currently active (or hung in stasis) message queues, semaphore sets and/or shared memory segments. It best serves as a way to interpret possible system performance problems (commonly referred to as sluggishness ;) and can be very useful. You'll generally want to call it in one of four or five ways (for the most generic purposes):

host # ipcs -q <-- To query the message queue
host # ipcs -m <-- To query shared memory segments
host # ipcs -s <-- To query semaphore sets
host # ipcs -a <-- To query all three (like -qms)
host # ipcs <-- To query all three the same as with "-a". Use this if you're not sure if you have the "-a" option. It should be safer.

General Listing Information by column (Note that these may appear in different forms on your machine. For instance, the type (T) might be listed in a banner across the top of the output and columns like ID might be invocation specific, like shmid for "ipcs -m," etc):

Note: To avoid carpal tunnel syndrome, I'll be referring, from this point on, to messages queues as MQ's, shared memory segments as SHM's and Semaphore Sets as SEM's.

Columns generally shared amongst all object types:

T - The type of listing: q for MQ's, m for SHM's and s for SEM's
ID - The ID number of an MQ, SHM or SEM
KEY - The hexadecimal or, more often, octal key used to identify an MQ, SHM or SEM
MODE - The permissions of the particular MQ, SHM or SEM. These work almost exactly like regular file permissions, with a few exceptions:

1. The first two columns may be either of the following (or they may be backward). Some options may not exist on your version:

Usually column 1:

S : an ipcs object waiting for a msgsnd() call
- : the exact opposite of S if in the same column

Usually column 2:

R : an ipcs object waiting on a msgrcv() call
C : an SHM that will be cleared on next SHM attach operation
D : an SHM that has been removed but won't disappear until the last process attached to it detaches
- : the opposite of any or all of the above, or (in other words) none of the above

Columns 3 through 9: Owner, Group and Other permissions. These may consist of r (read), w (write) and a (alter) permissions. the "-" character indicates that the permission is not granted. Every permissions set (from left to right: Owner, Group and Other) is made of up three columns (r, w and a, with optional "-"'s in any column):

Ex:
SDrwarw-r--


Columns specific to MQ's:

CREATOR: The username or UID that created the listed MQ
CGROUP: The group of the MQ's CREATOR
CBYTES: The number of bytes outstanding in the MQ
QNUM: The number of messages outstanding in the MQ
QBYTES: The maximum number of bytes allowed in messages outstanding in the MQ
LSPID: The last PID to execute a msgsnd call to the MQ
LRPID: The last PID to receive a msgrcv call from the MQ
STIME: The last time a msgsnd call was sent to the MQ
RTIME: The last time a msgrcv call was received from the MQ

Columns specific to SHM's:

CREATOR / SID: The username or UID that created the listed SHM
CGROUP: The group of the SHM's CREATOR
NATTACH: The number of processes attached to the SHM
SEGSZ: The size of the SHM
CPID: The PID of the SHM CREATOR
LPID: The last PID to attach to, or detach from, the SHM
ATIME: The time the last attach operation was executed on the SHM
DTIME: The time the last detach operation was executed on the SHM

Columns specific to SEM's:

CREATOR: The username or UID that created the listed SEM
CGROUP: The group of the SEM's CREATOR
NSEMS: The number of semaphores in the SEM
OTIME: The time the last semaphore operation was completed on the SEM

Other Column headers you may see, that can be easily matched up with their equals in the lists above:

Additional possible MQ columns:

------ Shared Memory Segments --------
key shmid owner perms bytes nattch status


Additional possible SHM columns:

------ Shared Memory Segments --------
key shmid owner perms bytes nattch status


Additional possible SEM columns:

------ Message Queues --------
key msqid owner perms used-bytes messages


2. ipcrm:

Basic Usage: This command is used to remove currently active (or hung in stasis) MQ's, SHM's and/or SEM's. In a perfect world, you'd never have to run this command, but (if you work with IPC a lot) you'll probably use it quite a bit (by which I'm, in no way, implying that this world is anything less than perfect ;) You'll generally want to call it in one of three to six ways (I've found that quite a few distro's do not allow removal by "key," as illustrated below):

These three work on every version I've tested:

host # ipcrm -q MQ_ID <-- To remove an MQ by ID
host # ipcrm -m SHM_ID <-- To remove an SHM by ID
host # ipcrm -s SEM_ID <-- To remove a SEM by ID

These three work on most Linux distro's, but not most Unix that I've tested:

host # ipcrm -Q MQ_KEY <-- To remove an MQ by hexadecimal or Octal KEY
host # ipcrm -M SHM_KEY <-- To remove an SHM by hexadecimal or Octal KEY
host # ipcrm -S SEM_KEY <-- To remove a SEM by hexadecimal or Octal KEY

And that, my friends, concludes the short version of this attempt to reasonably cover every possible widely-portable ipcs and ipcrm option (with result-layouts explained). As I mentioned at the beginning, be sure to check the man pages for your Unix or Linux distro's specific implementation of ipcs and ipcrm. You're liable to find a lot more options and/or how some options vary slightly from what has been posted here (For instance, Solaris 10 - with all the fancy zone functionality - has quite a few options most other OS's don't ;)

Enjoy and have a safe and peaceful morning, afternoon, evening and night :)

, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Tuesday, November 11, 2008

Determining Volume Group Disk Usage On AIX Unix Using Sh And Awk

Hey there,

Today's topic, and accompanying script, have to do with a more specific function already described in our series on working with AIX LVM (Actually, that link points to the last entry in the series. It's a strange thing about serialized content; the only way I can hook you up with a link that will get you links from part 4 to parts 3, 2 and 1 is to give you the last link. If only I could predetermine my following day's post URL without having to actually write the post... Actually, it probably can be done... I'll remind myself to work that into a post here soon. Hopefully I'll send myself an email to remind me before I forget ;)

This post has, loosely, to do with work. And I only mention this because I read in the online news that Google just canned some guy for writing about work in his blog! I'm not going to b.s. you and pretend like I read the entire thing, but, hopefully, he was posting company secrets. I'd hate to think what kind of an oppressive environment the corporate world would become if you could get your walking papers for ruminating over your day in an online diary. A lot of people have to work so much, they don't have much else to write about. Especially at Google. Why do you think they advertise all the games and cafeteria's and cots? Because it's a great place to work? They say; but perhaps it's because you're never ever going home again. The circle of madness has to be broken at some point ;)

In any event, working on an old AIX box, I was presented with a problem I hadn't considered for a while (on 5.x, anyway - I've been stuck in 3 and 4.x land for far too long ;) and, as I looked around for answers, I noticed that the information required to complete a pretty basic piece of work (and very useful, as well) just wasn't out there anywhere. Of course, the work involved using 2 (count 'em, 2 ;) separate commands, but... nothing. And that brings us to here.

The situation: Given a number of physical volumes, which have striped volume groups written across them, how can we figure out how much disk space (on how many different physical volumes and altogether) does that take up. I'm being intentionally vague, since the value we arrive at isn't as consequential as just being able to arrive at the value. Let's just say that we need to know because we need to replicate the disk space requirements elsewhere.

The answer: Surprisingly simple, once you get to it (although, perhaps because of my in-and-out AIX experience, confusing from time-to-time along the way). In order to extract this information, we just need to cruise through the following steps:

1. List out all available volume groups with the lsvg command:

host # lsvg
rootvg
number1vg
number2vg
...


2. Find out what physical volumes are associated with those volume groups using the same command and the "-p" switch. This time we'll need to name one of the volume groups. Steps from this point out would repeat for each individual volume group (and, later, physical volume) that comprise the logical and physical disk we're interested in (ellipses will be used to shorten lines for readability's sake):

host # lsvg -p rootvg
rootvg:
PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIBUTION
xdisk0 active ...
xdisk1 active ...


3. Then, get the number of physical partitions, and the set physical partition size for each physical volume listed out with the "lsvg -p" command above (The only two lines we really need from this output are the only two that aren't cropped with more ellipses), using the lspv command:

host # lspv xdisk0
PHYSICAL VOLUME: xdisk0...
PV IDENTIFIER: 000...
PV STATE: acti...
STALE PARTITIONS: 0 A...
PP SIZE: 64 megabyte(s) LOGICAL VOLUMES: 11
TOTAL PPs: 542 (34688 megabytes) VG DESCRIPTORS: 2
FREE PPs: 3....
USED PPs: 1....
FREE DISTRIBUTION: 1....
USED DISTRIBUTION: 0....


and we're all set :) Now, we just have a big sloppy mess of output to deal with. Lots and lots of lines of garbage, with the numbers we need sandwiched in, that we can manually add together. So, we'll move on to the natural next phase and script this out. For that volume group "rootvg" (for instance), rather than seeing all the above times 2, 3 or however much it is, this is much nicer to look at:

host # ./pvinfo.sh
Volume Group: rootvg
-----------
Physical Volume: xdisk0
Number of Physical Partitions: 542 Physical Partition Size: 64 Segment Total 34688 Mb
-----------
Physical Volume: hdisk1
Number of Physical Partitions: 542 Physical Partition Size: 64 Segment Total 34688 Mb
-----------
Physical Volume Grand Total : 69376 Mb
...


and so forth for all of our volume groups. Now we'll know how much disk space is allocated to our volume group, in total, and how much is allocated from each physical volume associated with the volume group. Feeling much better now ;)

Hope this helps you out some!

Cheers,


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/sh

#
# pvinfo.sh - get your AIX physical volume totals here!
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

for x in `lsvg|awk '{print $1}`
do
echo "Volume Group: $x"
echo "-----------"
lsvg -p $x|sed 1d|while read one two three
do
if [ $two == "active" ]
then
echo "Physical Volume: $one"
lspv $one|awk 'BEGIN{sum1=0;sum2=0}{ if ( $1 ~ /TOTAL/ && $2 ~ /PPs/ ){ sum1 += $3 } else if ( $1 ~ /PP/ && $2 ~ /SIZE/ ) { sum2 += $3 }}END{sum3=sum1*sum2; print "Number of Physical Partitions: " sum1 " Physical Partition Size: " sum2 " Segment Total " sum3 " Mb";print "-----------"}'
fi
done
lsvg -p $x|sed 1d|while read one two three
do
if [ $two == "active" ]
then
lspv $one|awk 'BEGIN{sum1=0;sum2=0}{ if ( $1 ~ /TOTAL/ && $2 ~ /PPs/ ){ sum1 += $3 } else if ( $1 ~ /PP/ && $2 ~ /SIZE/ ) { sum2 += $3 }}END{sum3=sum1*sum2;print sum3}'
fi
done|awk '{ sum4 += $1 }END{print "Physical Volume Grand Total : " sum4 " Mb";print "-----------"}'
done


, Mike




Chris Gibson wrote in with this helpful shortcut. He's got quite a wealth of AIX info, and we've posted additional comments from him on our LVM Series page :)


BTW, you can also do the thing same with the lsvg command:

$ lsvg rootvg | grep 'TOTAL PPs'
VG PERMISSION: read/write TOTAL PPs: 1150 (73600 megabytes)

In the example above, we have 73600Mb in rootvg.

Cheers.

Chris


Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Monday, November 10, 2008

Updated Bash Script To Get Cable, Dish and Local TV Listings

Hey There,

For this week's Monday Linux/Unix bash shell script, we're following up on our script from last week, with more user suggested improvements, that makes it so you can get your local TV listings from the command line with bash. This week's script is an update to include a lot of error checking and (from the suggestion I received the most often :) the ability to pick a time span. This script's invocation will allow you to get listings from "right now" (or whatever you set the "time" variable to) to the regular 3 hour time span in half-hour increments. The next paragraph is a blatant homage (read: shameless self promotion) to "other scripts we've done before that follow along the lines of this one." Please skip it and move on to next paragraph unless you enjoy reading lists for the sake of list reading :)

Our previous web-based Bash scripts, in backward chronological order, include our posts on accessing Wikipedia, accessing the Farmer's Almanac, accessing the International Dictionary, checking out the world's weather, spewing out famous quotations on pretty much any subject, doing encyclopedia lookups, accessing the online Thesaurus, translating between different languages and, of course, using the online dictionary.

This script still gets its content from the TV Listings At Zap2It.com and includes a lot of error fixes. Most importantly, though, it allows you to pick your time slot (assuming you don't want the full 3 hour spread), shown in the usage message like this:

host # ./cabletv.sh -h

Usage: ./cabletv.sh ...all options are optional [-h for help]
[-z USZipCode] [-p LocalCableOrDishProviderID]
[-t Time] [-d date] [-n to leave out HD channel info]
[-s ListingTimeSpan 1 for just now, 2 for an additional half
hour, 3, 4, 5, 6, in half hour increments, and 7 for 3 hours
...defaults to 7 since this is the online listing default]


Much of the error checking was done so that the script would capture bad zip codes, invalid time specifications and make it so you can set your date in a more free manner (output below does not include actual listings to make the picture smaller):

Click on the picture below for a graphic depiction of some of the code fixes ;)

cable tv script errors

After that, check out the next few pictures below, which show successful executions and the beginning of the TV listings. The first picture shows the output from a "right now" execution and the other is from an "2 hour" execution. The default time span is 3 hours, since that's the way it is online:

Click on the pictures below and prepare for the mediocrity ;)

tv listings for right now

2 hours of tv listings

I'd like to thank everyone who wrote in with comments and suggestions (which I'll continue to post online, as possible). The script is starting to get better, but it's still far from perfect. Hopefully you find it more helpful now :)

I still need to put out those movie/favorites filters. Soon... Soon... ;)

Cheers,


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/bash

#
# cabletv.sh - Get your local regular and HD Tv listings
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#
# UPDATE FOR 11/9/2008
# TODO You have entered an invalid ZIP or postal code
# TODO am to pm or pm to am ends up not in chronological order
# TODO 12 to 1 a/p ends up not in chronological order
# TODO switch to enhanced getopt to deal with variable argument switches
# TODO add config file
# TODO check for bad provider code
# DONE check for bad zip code
# DONE added optional time increments - up to 3 hours - for display
# DONE tightened up code using indirect variable references
# DONE date may now be entered in quotes numeric or alpha - e.g. 11/9/08 11/09/2008 "Nov 9 2008" "November 9th 2008" etc
# DONE time may now be entered as 10pm 10 (with am/pm defaulting to system time) 10:00am 23:00 etc
# DONE added error checking for invalid date and time entry - stop before querying online guide
# DONE fixed issue with 00:xx time causing script failure
# DONE added extra error checking
#

function usage()
{
echo
echo "Usage: $0 ...all options are optional [-h for help]"
echo "[-z USZipCode] [-p LocalCableOrDishProviderID]"
echo "[-t Time] [-d date] [-n to leave out HD channel info]"
echo "[-s ListingTimeSpan 1 for just now, 2 for an additional half"
echo "hour, 3, 4, 5, 6, in half hour increments, and 7 for 3 hours"
echo "...defaults to 7 since this is the online listing default]"
echo
exit 1
}

while getopts t:z:hs:nd:p: option
do
case $option in
t)
opt_time="$OPTARG"
;;
d)
opt_date="$OPTARG"
;;
z)
opt_zip="$OPTARG"
;;
n)
opt_nohd=1
;;
p)
opt_provider="$OPTARG"
;;
s)
opt_timespan="$OPTARG"
;;
h)
usage
;;
*)
usage
;;
esac
done

wget=/usr/bin/wget
pager=/usr/bin/more

if [ ! "$opt_date" ]
then
nicedate=`date "+%m/%d/%y"`
else
nicedate=$opt_date
fi

if [ ! $opt_timespan ]
then
opt_timespan=7
elif [ $opt_timespan -lt 1 -o $opt_timespan -gt 7 ]
then
usage
fi

if [ ! $opt_time ]
then
hour=`date "+%I"`
minute=`date "+%M"`
ampm=`date "+%p"`
if [ $hour -lt 12 ]
then
if [ $minute -lt 30 ]
then
nicetime="${hour}:00 AM"
else
nicetime="${hour}:30 AM"
fi
else
if [ $minute -lt 30 ]
then
nicetime="${hour}:00 PM"
else
nicetime="${hour}:30 PM"
fi
fi
if [ $hour == "00" ]
then
cli_time=${hour}:${minute}
else
cli_time=${hour}:${minute}${ampm}
fi
else
cli_time=$opt_time
fi

date_ok=`date -d "$cli_time" +%s 2>&1`
if [[ "$date_ok" =~ "invalid date" ]]
then
echo
echo "INVALID TIME ENTERED: $opt_time"
echo "Be sure to include full am or pm"
echo "and do not add am or pm to 24 clock entries"
echo
usage
fi

ampm=`echo $opt_time|egrep 'a|p|m' 2>&1`
result2=$?
if [ $result2 -eq 0 ]
then
date_ok2=`echo $opt_time|egrep 'a|p'|grep m 2>&1`
result3=$?
if [[ $result3 -ne 0 ]]
then
echo
echo "INVALID TIME ENTERED: $opt_time"
echo "Be sure to include full am or pm"
echo "and do not add am or pm to 24 clock entries"
echo
usage
fi
fi

cli_time2=`date -d "$cli_time" +%s`
let fromtime1=$cli_time2*1000
let fromtime2=$fromtime1+1800000
let fromtime3=$fromtime2+1800000
let fromtime4=$fromtime3+1800000
let fromtime5=$fromtime4+1800000
let fromtime6=$fromtime5+1800000
let fromtime7=$fromtime6+1800000
if [ ! $opt_timespan ]
then
opt_timespan=7
fi

timespan_relay="$fromtime1"
timespan_counter=2
timespan_limit=$opt_timespan

while [ $timespan_limit -gt 1 ]
do
nextup_time=$(eval "echo \$$(echo fromtime${timespan_counter})")
timespan_relay="$timespan_relay $nextup_time"
let timespan_counter=$timespan_counter+1
let timespan_limit=$timespan_limit-1
done

from_timespan=$(eval "echo \$$(echo fromtime${opt_timespan})")
cli_time_end=`echo $from_timespan|awk '{printf("%20s",strftime("%I:%M %p - %x", $0 / 1000))}'`

if [ ! $opt_nohd ]
then
opt_nohd=0
fi

if [ ! $opt_zip ]
then
echo -n "Enter Zip Code: "
read zip_code
else
zip_code=$opt_zip
fi

$wget -nv -O - "http://tvlistings.zap2it.com/tvlistings/ZBChooseProvider.do?method=getProviders&zipcode=${zip_code}" 2>&1|sed -e :a -e 's/<[^>]*>/ /g;/</N;//ba' |sed -e '/^[ \t]*$/d' 2>&1|grep "You have entered an invalid ZIP or postal code" >/dev/null 2>&1
zip_ok=$?

if [ $zip_ok -eq 0 ]
then
echo
echo "INVALID ZIP CODE ENTERED: $opt_zip"
echo
usage
fi

if [ ! $opt_provider ]
then
$wget -nv -O - "http://tvlistings.zap2it.com/tvlistings/ZBChooseProvider.do?method=getProviders&zipcode=${zip_code}" 2>&1|sed 's/<.*&lineupId=\([^&]*\)\">/\1 \c/'|sed -e :a -e 's/<[^>]*>/ /g;/</N;//ba' |sed -e '/^[ \t]*$/d' |sed -e '1,/Choose Your Provider/{x;$!d;x;q;};x;$G' -e '/document.write/,$d' -e "s/'/'/" -e 's/&/\&/' -e 's/^[ \t]*//;s/[ \t]*$//'|sed '/\([A-Za-z0-9_][A-Za-z0-9_]*:[-X]\)/ {
N
s/ *\n/ \t/g
}'
echo "Enter Provider ID [e.g. \"IL57303:-\" \"4DTV:-\" \"IL12561:X]\""
read tv_provider
else
tv_provider=$opt_provider
fi

echo
echo "Television Listings for $nicedate from $cli_time to $cli_time_end"
echo

last=1

if [ $opt_nohd -eq 1 ]
then
for x in $timespan_relay
do
$wget -nv -O - "http://tvlistings.zap2it.com/tvlistings/ZCGrid.do?fromTimeInMillis=${x}&method=decideFwdForLineup&zipcode=${zip_code}&setMyPreference=false&lineupId=${tv_provider}" 2>&1|sed 's/<.*&sch=\([^&]*\)&[^>]*>/\1 /'|sed -e :a -e 's/<[^>]*>/ /g;/</N;//ba' |sed -e '/^[ \t]*$/d' |sed -e '1,/Forgotten password/d' -e '/isFavoritesAvailable/,$d'|sed -e '/[ECMP][SD]T/,+6d' -e "s/'/'/" -e 's/&/\&/' -e '/zc.getAdFrame/d' -e 's/^[ \t]*//;s/[ \t]*$//'| sed -n '/^[0-9][0-9]*$/,+2p'|sed 's/^\([0-9][0-9]*\)$/\n\1/'|sed '/^[0-9][0-9]*$/ {
N
N
s/ *\n/\t/g
}'|awk '{ if ( $1 ~ /^[0-9]/ ) {printf("%-4s %-8s ", $1,$2);for (i=4;i<NF+1;i++) {printf("%s ", $i)};printf("%20s %-20s", "*** Start",strftime("%x - %I:%M %p",$3 / 1000));print ""}}'
done|sort -t'-' -k1,1n -k2,3n | uniq|while read one two three;do if [ $one -eq $last ];then echo " $three";last=$one;else echo "$one $two $three";last=$one;fi;done|$pager
else
for x in $timespan_relay
do
$wget -nv -O - "http://tvlistings.zap2it.com/tvlistings/ZCGrid.do?fromTimeInMillis=${x}&method=decideFwdForLineup&zipcode=${zip_code}&setMyPreference=false&lineupId=${tv_provider}" 2>&1|sed 's/<.*&sch=\([^&]*\)&[^>]*>/\1 /'|sed -e :a -e 's/<[^>]*>/ /g;/</N;//ba' |sed -e '/^[ \t]*$/d' |sed -e '1,/Forgotten password/d' -e '/isFavoritesAvailable/,$d'|sed -e '/[ECMP][SD]T/,+6d' -e "s/'/'/" -e 's/&/\&/' -e '/zc.getAdFrame/d' -e 's/^[ \t]*//;s/[ \t]*$//'| sed -n '/^[0-9][0-9]*\.*[0-9]*$/,+2p'|sed -e 's/^\([0-9][0-9]*\.*[0-9]*\)$/\n\1/'|sed '/^[0-9][0-9]*\.*[0-9]*$/ {
N
N
s/ *\n/\t/g
}'|awk '{ if ( $1 ~ /^[0-9]/ ) {printf("%-4s %-8s ", $1,$2);for (i=4;i<NF+1;i++) {printf("%s ", $i)};printf("%20s %-20s", "*** Start",strftime("%x - %I:%M %p",$3 / 1000));print ""}}'
done|sort -t'-' -k1,1n -k2,3n | uniq|while read one two three;do if [ "$one" == "$last" ];then echo " $three";last=$one;else echo "$one $two $three";last=$one;fi;done|$pager
fi

exit 0


, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Sunday, November 9, 2008

Unix And Linux Humor - Cult Satire

Hey There,

For this Sunday's humor section, I found a cool little piece of future-nostalgia on a site called Hillside Systems. There's a very interesting article on their site (recently updated - Something from 2008! Almost unheard of ;) regarding The Unix Cult that's pretty enjoyable, and funny, reading if you like your satire straight up - no emoticons ;) <-- Like that winking thing I do every 5 seconds...

Enjoy, as we travel, now, into the future, to reflect upon (and attempt to understand) the past, which is probably happening as I type :)

Happy Sunday :)





The UNIX cult









Peter Collinson (local name)

AS/103/108/121/110 (Galactic designation)


Galactic Cult Investigation Team,

Canterbury, Kent

(Small Island off Continent 3, Sol 3)


Abstract

Notes from some recent archeological findings on the birth of the
UNIX cult on Sol 3 are presented.
Recently discovered electronic records have shed considerable light
on the beginnings of the cult.
A sketchy history of the cult is attempted.

Background

The UNIX
cult is widespread across the Galaxy now and the
surprise discovery of some ancient files in the archives of
Intergalactic Brain Machines on Sol 3 triggered the dispatch of
an inter-disciplinary investigation team.
The files are extremely
extensive, occupying all of a small island off the coast of
Continent 3.
It transpires that the island was
taken over by Intergal in the aftermath of
the Corporate Wars which plagued Sol 3 some centuries after the
birth of the cult.

The team were asked to find out the original meaning of some of
the incantations used in UNIX religious practice
and also to shed some light on what it all meant at the start.

We should take this opportunity to use the ancient prayer:

UNIX is a trademark of AT&T in the USA and other countries.

Earlier versions of this prayer do seem to exist, it is unclear why the
form of words altered.
`AT&T'
was the Corporation where the Creators of the cult worshiped.
The Corporation
totally disappeared in the wars and many of its original records were
either destroyed or altered by the victor in an attempt to `re-write'
history.
The placement of the country USA
on the four continents has been lost.

The Gurus

There seems to be still no trace of the original Creators of the UNIX
cult, so we start our examination of the records with a group calling
themselves the Gurus .
The etymology of this word is not quite clear
but it does have associations with religious teaching and the High Priests of
UNIX are given that title today.

Extract from electronic phone tap of someone nicknamed
`dsw', believed to be Daniel Stuart Wilson:


``Of course, we were all isolated in the Version 6 days.
When we changed things in the kernel we were on our own.
There was no-one to phone to scream for help, in many cases
there was no-one on the same site who you could discuss the problems
with.
We just had to get down and read that C.
Sorting things out yourself was painful but you benefited in the long
run.
You became a better software engineer.
After a bit you became a Guru and could lead others.
This is largely bluffing - given a new situation and a knowledge of
UNIX you could extrapolate the problems easily and seem to be very clever.
It didn't actually matter what you had or had not done, it mattered that you
appeared
to have done a lot and could talk confidently about
RK05's,
PDP-11/45's,
typing chdir rather than cd and a myriad other things
which showed that you were a Guru.''

This extract shows clearly that the cult grew in small
pockets across the globe.
In each centre, a few individuals were given the task
of installing the paraphernalia of the UNIX
cult and of converting others to its use.
In the beginning,
it seems that these individuals had little or no contact with each other;
curiously, this appears to have strengthened their ability rather than weakening it.
Other extracts from the archive indicate that
the early practices of the cult were small and simple making it easier for
one person to grasp their full meaning.
As the cult grew, the practices became more complicated and the understanding
harder.

The term
software engineer refers to a person whose task it was to feed instructions or
programs
into the primitive versions of the Overlords which were extant at the time.
Judging from the emphasis made in the records about the
need to generate `correct' programs, this was obviously an
artistic task requiring considerable expertise.
The reference to `C' implies that there was some official form of speech
or perhaps a special religious language which was used to convey instructions.

It is not clear whether the
RK05 and the PDP-11/45
were the names of the Overlords or some ancillary equipment associated
with them.
However,
the word cd
(Pronounced `see-dee')
meaning `to move from place to place'
is still used in some arborial societies on the planet US/115/110/105/120.

The User Group

In current UNIX
religious practice, the term
User Group
is used to refer to the congregations at the Hologram Services.
It seems that after a while
the early practitioners of the cult
began to have meetings (where people actually appeared together
in the same room).
Why these were called `User Group' meetings is unclear, since
the early meetings were attended by Gurus and not by Users.
At the time, a User seems to have been a derogatory term for
the un-initiates.
However, the head Guru at a site was given the title `Super-User';
perhaps this was to hide the evangelical nature of the Guru's task.

The reason for the early meetings was mostly to allow
Gurus to inform each other
how best to perform religious conversion and how to get the most converts
the fastest by improving the Service to Users.

As the word spread,
the meetings grew in size, expense and quality of surroundings.
They proved to be an exceptionally good way of upgrading
low level Novices into higher level Gurus.
This is a reflection on the early religious books which were
aimed at Gurus and were often way above the head of the Novices.
Novices were encouraged to attend the meetings to gain
by word of mouth what they could not gain from the literature.
At these later meetings, the nuts and bolts of UNIX
practice were still discussed because many of the Novices
were either acting
in a support role to Gurus or planning to become evangelists themselves.
After a while, the Gurus got bored with discussing
inodes (The meaning of this is totally lost).
and other topics began to creep in.

The meetings gradually altered in character, with the Gurus attending
because they wished to see the other Gurus; and more and more Novices
attending in the vain hope that they would learn something.
The cult had spread so far by now that it was profitable for
Corporations to become involved in the selling of UNIX
paraphernalia and religious goods.
Initially, these Products were advertised widely at the meetings.
However, the Corporations often sent attendees with prepared scripts
who were sometimes not even Users
and who had little or no knowledge of intricacies of UNIX
practice.
The Gurus and Novices were dismayed.

On their side, Corporations began to see that the word which
they were spreading was falling on somewhat stony ground.
This gave rise to the Great Split with a rival organisation being
set up primarily aimed at selling and the original User Group
concerning itself more with the cerebral activities of the cult.
In the end, this was a good thing because the Gurus were able
to start deriving benefit from the meetings again since
there were now spare slots for educational talks.
In fact,
the rival organisation was wiped out in the Corporate wars
because it had allied itself strongly with AT&T.

The User Group, then, provided some important functions.
It supplied a forum for discussion of the practices of the cult.
It provided a meeting place for the widespread Gurus who initially
met to discuss their work but as time passed they went just to meet
each other.
It spread the word to Novices; and
as the cult grew, it provided a place where new ideas on the
direction which things should take could be discussed.

Another more hygienic method of worldwide communication
grew out of the cult with the formation of the Network.
This is discussed in the next section.

The Communicators

When the cult had grown to worldwide proportions, we begin to see
the emergence of a global communication system - the `Network'
or `Net'.
The activities on the Network can be deduced from the
electronic archives but the material is so vast that
scanning it for relevant information is proving difficult.

The majority of traffic on the network seems to have been
communication between Overlords describing various error conditions.
Here is a sample:




Various repeated unintelligible lines
From: MAILER-DAEMON (Mail Delivery Subsystem)
To: <uucp@a4los.uucp>
Subject: Returned mail: Service unavailable
----- Transcript of session follows -----
>>> DATA
<<< 554 sendall: too many hops (30 max)
554 <megd> ... Service unavailable: Bad file number
----- Unsent message follows -----
More of the same



There are very many other examples of the same type of message.
However, these messages do place things in context - we now know
that the communication system was called Mail and from this we
infer that the mechanism was intended to permit communication between
people.
In amongst all the Overlord messages we do find some files
which appear to emanate from one person and be addressed to another.
A high proportion of the sampled files were intended to probe the
capabilities of the Network, often provoking an Overlord error message.
We know this because the
destination address is the same as the source; in some cases the
subject is `Just testing' or something akin to that.
This technique was perhaps used to investigate the ability
of the Network to transfer files.
We are left with a much smaller
number of what might be termed `useful' messages.
In many cases, these consist of personal trivia showing that
the Network was supplying the useful social function of allowing
people to make and maintain contact.

Other messages consist of hieroglyphics containing many
braces `{' and brackets `('.
This type of message shows some form of regularity in structure but
syntax analysis is hampered by the presence of so many exceptions.
It is possible that these messages contain portions of religious
ceremonial or it has been suggested that this is the
ritual language `C' and the exceptions are what were called `bugs'.
This last suggestion (by a post graduate student on the team) has
been greeted with a certain amount of derision.

Mail messages have a recognisable format and can be distinguished from
another type of message which occurs in much greater volume.
It appears that these messages are part of a system called `The News'
broadcast to many sites.
The News permitted individuals to send a single message to many
people across the world.
A sample of the contents imply that the News took over as
the main method of spreading the UNIX
word when the User Group meetings ceased to be totally useful
in this role.

Judging from the beginning of many News files,
the News was split into many separate subject headings.
Over large periods of time, we see the subjects come and go with
abrupt changes in title occurring from time to time.
The subjects do not seem to be confined to discussion of topics
with direct relevance to the cult.
There are many headings which just carry `talk' on various subjects.
An analysis of the topics is being prepared as a background
document since they fall outside the remit of the investigation.

It might be thought that the News would provide an excellent vehicle
for the Super Gurus who must have existed by this time - but
far from it - very few messages contain what might be termed
confident information.
Most seem to carry opinion which is contradicted in later messages.
The evidence here is that most authors were Novices.
The Gurus had either lost interest in spreading the word, or were simply
too busy to wade through what must have been daily oceans of verbiage.
It is also possible that Gurus just used the Mail to communicate, perhaps
not wishing to impose their definitive opinions on the discussion
with the thought that discussion is a healthy academic tool.

The Network spread slowly over most of Sol 3 with some areas
being exempt because either they resisted conversion to the UNIX
cult or were deliberately omitted for ideological reasons.
A single News message from `kremvax', an otherwise silent site,
seems to have been met with a storm of protest.
This shows that the Network crossed political boundaries and proves
the contention that Sol 3 was split into separate economic entities
predating the rise of the Corporations.

The Vendors

The Corporations who were involved in the propagation of UNIX
paraphernalia and religious goods were known as
Vendors.
The Vendors had a different view of the world from the Gurus, and
this difference lead to many schisms in the cult.
The Vendors and Gurus tried to maintain a separate identity at all times.
For example, when the Vendors attended the User Group meetings
they provocatively wore different religious vestment from the Gurus.
They did this to make it easy for other Vendors to identify them
and to allow their easy differentiation from the Gurus and Novices.

At the centre of the clash of opinion was a fundamental difference
in the perception of `the User'.
The Vendors were always complaining that UNIX
religious practices were not `User friendly'.
By this, they meant that the act of worship was hard to learn, and
the rituals were cryptic.
They wished their Users to only deal in simple concepts and never
to learn more than the basics of the rituals.
The Gurus objected to this view because they believed that the learning
of cryptic rituals allowed the worship to proceed faster and
that limited exposure to only the simple concepts restricted the
Users in a way which was not desirable.
The Vendors believed that the Users were fundamentally stupid and
without any hope of redemption; and the Gurus believed that the
Users were fundamentally stupid but might be saved given the correct
tuition.

The early Vendors were peopled by Gurus who had often to undergo the
necessary clothing transformation to demonstrate that they had
switched camps.
These Vendors were responsible for the spreading of the cult
to a much wider and naive User population.
Some of the Vendors tried to set up their own breakaway cults to avoid
the central control imposed by AT&T,
the Corporation where the Creators worshiped.
These attempts failed.
Other Vendors created Sects which specialised in worshiping the
many minor Overlords which had appeared about this time.
These Sects were partially successful and converted Users
to their way of thinking.
The Sects created by this method had special names
related syntactically to the word UNIX.
Only XENIX
has survived as an example of this.

Then seeing all this activity, AT&T
decided to become a Vendor.

The Gurus were horrified when the Marketing Staff, regarded
today as the front line fighting force of all Corporations, were put
in charge of the development and promulgation of the cult.
The Creators were no longer allowed to directly influence the
development, instead they were to pass their ideas to the Marketeers
who would decide what was acceptable or not.
The Marketeers spent a lot of time producing a new religious tome
to help to guide the developers.
The book was known as the Svid and laid out the central
tenets of the practice.
The Marketeers were determined that the Svid should be elevated to
the level of all the other religious books.
To this end
they tried to make the Svid as cryptic as all the others, and succeeded.

After some initial shock, the Vendors accepted the central control
which AT&T
imposed because they saw that it
made their Products accessible to more Users.
`Consider it Standard' became the watchword in a Crusade designed
to eliminate the undesirable Sects which wished to differ from the Svid.

The Users of the Vendors Products
were certainly pleased with AT&T's
decision to become a Vendor.
It meant that they were no longer tied to the Products of one
particular Vendor but could pick and choose without altering their
worship.
The Gurus were less pleased until they realised that
AT&T were unable to change the cult to eliminate them.
Every Overlord where the UNIX
cult was practiced still required a Guru
to perform essential tasks.

The Sects

The UNIX
cult was always noted for its propensity to split into separate Sects.
In the early days, the Creators had a release policy which
made sure that all cult members possessed all the relevant facts
in order to fully comprehend the implications of the worship.
It was said that Users aspiring to be Gurus
needed all the facts because the religious books were written for Gurus,
and Users could make no sense of them.
Unfortunately, access to the information
was much abused because the Gurus immediately used
the knowledge to alter things and many
minor and major Sects of the Cult sprang into life.
The ability of each site to generate its own Sect was somewhat
curtailed by the cunning ploy of re-issuing the rituals and
practices in a slightly altered form from time to time.
The Gurus were soon tired of altering the same things
every time a new set of rituals came though the door and they began
to leave things alone.
Also, by this time, the Vendors had made an appearance and because they
believed in the Svid Crusade, they stuck with the orthodox mainstream
AT&T view of the cult.

Even so, at the end of the period being researched there seems to have been
two Sects with differing practices and rituals.
The main rival to the orthodox view
was a Guru lead Sect called the `Berkeley System Devotees'.
This had sprung into existence in the early days of the cult,
taking advantage of the knowledge imparted to them by the Creators.
However, the early Berkeley Gurus
cleverly distributed their rituals and practices
in the much the same way as the Creators and this ensured a wider
following.
The prominence was noted by a higher power and
they were chosen to master
the revisions of the Cult which were needed to permit worship on
some new Overlords.

These new Overlords had managed to conquer the restrictive memory sizes
which plagued many of their forbears.
It was said by many that the new Overlords had not got this right but at least
they did it.
As a result the new Overlords had the potential to allow bigger and
more expansive practices.
The Berkeley Gurus grasped this opportunity with the
objective of creating `The Perfect Ritual'.
The rival Sects referred to this as `creeping featurism';
the precise meaning of this obscure phrase is under investigation.

The Perfect Ritual was defined as one where all the letters of the
alphabet were used as a `parameter' specifying a distinct action
or phase in the worship.
The Berkeley Gurus never managed to create the Perfect Ritual,
but they came close.

The Berkeley Gurus distributed their rituals
and these became popular because the Marketeers inside AT&T
were so busy creating the Svid that they failed to notice that
Berkeley practices were slowly being adopted on all the new bigger Overlords.
The Berkeley rituals were also liked by Gurus because they were
nearer the `Old Religion' laid down by the Creators.
In a fit of pique, the AT&T
Marketeers decided that they would no longer
support the new Overlords and branded the Berkeley Gurus as heretics.

As heretics, the Berkeley Gurus decided to go one step further in
altering their Sect.
They proposed and executed a fundamental change of direction which
was to become a `De-facto Standard'.
The new Sect was revolutionary because it allowed Overlords to talk
to each other, but to do this a whole new litany had to be created.
New words entered the vocabulary and new concepts were introduced.
For many, worship in the new Sect was slow and unwieldy in comparison
to what had gone before.
But the new practices meant the easy
ability to interconnect Overlords and this was demanded by the Overlords
themselves.

Unfortunately for the
participants in the Svid Crusade, many Users actually insisted
on being able to use some of the Berkeley rituals.
The pressure from the Users was such that we begin to see Vendors
announcing their Products as being `with Berkeley enhancements'.
Finally, the AT&T
Marketeers were forced to incorporate some features of the
rituals which did not conflict with the teachings in the Svid.
At this time, we also begin to see specially created
Overlords which could be used to worship in the practices of either Sect simultaneously,
this was known as the `Universe Concept'.

The Berkeley Gurus were so broken by the gestation of the new Sect that many
left, some to worship the sun and
some to seek salvation in the noble task of Pixel Creation.
It was thought that the new Sect would be the last to
sally forth from hallowed halls of Berkeley because
the staff were demoralised and without joy.
The Svid Crusaders were pleased,
``All we have to do is wait and do nothing'', they said.
Since they weren't noted for doing much anyway, this wasn't difficult.

However, much to the dismay of the Crusaders,
many Novices amongst Berkeley group were promoted to Gurus,
and these new Gurus worked to consolidate and strengthen the new rituals.
The label of `slow and unwieldy' was not to be applied again.
Time and motion studies were performed on the rituals
for the first time in the recorded history of the
UNIX cult and a little more than lip service was paid to the notion
of efficiency of worship.

The Standardisers

One way of defeating the degeneration of the pure
UNIX cult into Sects was by the creation of a `Standard'.
As we have seen, the Svid is one example of this.
However, the Svid differed from other Standards because only the
AT&T marketeers were in a position to generate a Standard without reference
to anyone else.
They seemed especially keen that no taint of the Berkeley heresy should
appear in their work and so did not consult the Users.

In order that the Svid could qualify as a Standard, the
AT&T Marketeers had to promise that it would not alter.
They agreed to this because
they were determined to ensure that the Svid gained religious significance.
This was a good thing for other Vendors who were treating the
Svid as a Standard but it is possible that the slavish adherence
was detrimental to the fortunes of
AT&T in the long run because they were unable to update the rituals and
practices to keep up with the demand for change created by the Users.

All the other standards were either created by groups of Vendors
working together and ignoring the Users;
or by groups of Users working together and ignoring the Vendors.
Gurus were rarely involved; they were either too busy and
important to sit on committees or just plainly could not see the need for
conformity.

The Standards rarely reflected the
UNIX practices and rituals which were in use at the time of their creation.
All of them seemed to have a speculative element, as if the
Standardisers themselves tried to develop or perhaps rationalise the
rituals in some way.
As a result, the Standards were never standard.

However, the various Standards did give the Users an idea of what
was expected of them if they desired to move from one Sect to another.
Users who did this often, the so-called
Portables, learned to use
the minimum of ritual and to localise the Sect dependent areas of
their worship.

The Portables might have been helped by a Standard for the religious
language, C.
They were surprised to learn that Standard C was, in fact, a different
language from the original and almost no-one had an Overlord
which could understand it.
The changes were no doubt desirable but came from treating C as a
`high-level language' rather than using it in the way which the
Creators intended, a method of communicating intimately with
the Overlords.

Conclusion

The archives are still being searched for other interesting material
but enough has been found to demonstrate the fervent activity which
followed the creation of the
UNIX cult.
We have found no trace of the Creators and barely a hint of the
disciples who followed them inside AT&T .
We hope that
more research may provide some answers and respectfully
ask for more funding.




On to Sun Expert Articles








Peter Collinson Last change: 7th September 2008





, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.