Hey There,
For our "Lazy Sunday" Post this week, we're going to look at some more, hopefully, useful and easy html creation, like we've done many times in the past with posts on listing out Linux or Unix crontabs and a few others. Today's Bash shell script will produce an index.html file (and a whole lot of others) that should end up as your one-stop-shop for the documentation that came with your system's Perl distribution. It's nothing fancy (although you can make it that way if you want) but it should serve its purpose well enough right out-of-the-box.
Of course, you should edit the POD_HTML_DIR variable, at the least, so that the script can find your Perl modules and use pod2html to convert them. Other than that, running it straight-up, like so:
host # ./pods.sh
should produce output like below, in the following two pictures (click on them to make them larger if you need to :)
Hope you enjoy the script and it brings you some convenience! Note that I'm 100% aware that there is no error handling done for files that can't be created (i.e. when pod2html fails, a blank html page is hyperlinked to rather than no hyperlink, or html, being created), but that should make the whole experience more educational if you want to add that. You can refer to our earlier posts on Bash's implementation of errno and the Bash-specific PIPESTATUS variable to help handle those errors and do what you want with them ;)
Happy Sunday,
This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License#!/bin/bash
#
# pods.sh - make perdocs into one html repository
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#
trap 'cd $PWD;rm -rf $POD_HTML_DIR;exit 2' 1 2 3 15
PWD=`pwd`
PERL_MANPAGE_BASE=/usr/lib/perl5/5.8
POD_HTML_DIR=podhtml
INDEX_PAGE=index.html
if [ -d $POD_HTML_DIR ]
then
rm -rf $POD_HTML_DIR
fi
mkdir podhtml
echo -n "Working on it..."
find $PERL_MANPAGE_BASE -name "*.pm"|while read line do
do
echo -n "."
output=`echo $line|sed 's/^.*\/\(.*\)$/\1/'`
pod2html $line >podhtml/${output}.html 2>/dev/null;
done
cd $POD_HTML_DIR
echo "<html><head><title>Perl Module Document Repository</title></head><body>" >$INDEX_PAGE
ls |grep html|while read line
do
echo "<a href=\"$line\">$line</a><br>" >>$INDEX_PAGE
done
echo "</body></html>" >> $INDEX_PAGE
echo "done"
, Mike
Sunday, June 29, 2008
Fast Perl HTML POD Creation On Linux and Unix
Friday, April 25, 2008
Determining Signal Definitions In Linux and Unix
Greetings,
Today, we're going to take a quick look at simple return codes (or errno values) that get returned by the bash and ksh shells in Linux (and Solaris Unix). This complements a post we did a while back on trapping signals in Perl and the shell, and adds a bit more to understanding of the fork-and-exec process introduced in our post on running a login shell on a network port.
While these signals are subject to change, the POSIX standard is fairly consistent between most Unix and Linux systems. If you ever need to find out what a signal means, you have a number of options at your disposal to determine that information.
1. At the process level on Solaris, you can figure out what signals are associated with what processes by using the "psig" command. This command will list out all signals associated with a given process (assuming you know its process ID). A sample of this process-finding process would be something like:host # ps -ef|grep "[l]pd"
daemon 2073 1 0 Mar 21 ? 0:00 /usr/local/sbin/lpd
host # psig 2073
2073: /usr/local/sbin/lpd
HUP caught sigacthandler
...
ILL default
...
PIPE ignored
...
basically, you'll get a ton of output. And none of it may make any sense to you. That's okay for now :)
2. At the process level on Linux, you can get approximately the same functionality from the "crash" command ( used interactively, if available ) or, even better, this publicly available script written specifically to emulate Solaris' psig functionality, for use on Linux. It's called psig.sh for Linux and is an excellent tool for replicating the psig experience for Linux users who are used to Solaris' proc commands.
3. On either Linux or Unix (any flavor), you can determine what signals are handled by your machine, what their names are and what their signal numbers are, by checking your system include files. On Solaris, the header file that contains this information is almost guaranteed to be named /usr/include/sys/iso/signal_iso.h (at least for Solaris 9 and 10). In Linux, it will generally be very specific to the architecture of your build system (something like: /usr/include/asm-x86_64/signal.h)
In any event, you can follow a simple process to figure out where your system's signal definitions are loaded, no matter what flavor of Unix or Linux you're running. It all comes down to derivation and, luckily, you'll always have the same starting point (I may be wrong on this, but I've never experienced it ;).
Here's a quick command line walkthrough of two different ways to figure out where your signal definitions are listed (that is, the include/header file that defines the signal numbers, their abbreviated signal names and definitions) that can be used on almost any *nix system. As a starting point, /usr/include/signal.h is probably the best, since it exists on most systems. We'll also use SIGHUP (the standard HangUp signal and signal number 1) to guide our search:
The painful way:host # grep SIGHUP /usr/include/signal.h|grep -w 1
host # grep include /usr/include/signal.h
#include <sys/feature_tests.h>
#include <sys/types.h>
...
host # grep include /usr/include/signal.h|sed 's/^.*<\([^>]*\)>/\1/'|xargs -ivar grep SIGHUP /usr/include/var
host # grep include /usr/include/signal.h|sed 's/^.*<\([^>]*\)>/\1/'|xargs -ivar grep "^#include" /usr/include/var|grep sig
#include <sys/iso/signal_iso.h>
#include <sys/siginfo.h>
...
... and so on, and so on, until you find the right file. Basically, just follow the includes until you reach the end. This can be tedious and time consuming.
The incredibly easy way:
<--- This is our file :)Solaris_9_or_10_host # find /usr/include|xargs grep SIGHUP /dev/null|grep -w 1
/usr/include/sys/iso/signal_iso.h:#define SIGHUP 1 /* hangup */
or
<--- Note that all 3 of these entries are valid, with this header file being the most generic. Our next command can narrow down the most "specific" correct result.Linux_host # find /usr/include|xargs grep SIGHUP /dev/null|grep -w 1
/usr/include/bits/signum.h:#define SIGHUP 1 /* Hangup (POSIX). */
<--- Now we know that, if we want to be extra sure, the "/usr/include/asm-x86_64/signal.h" is the signal definition file we should be looking at to determine our signal information./usr/include/asm-x86_64/signal.h:#define SIGHUP 1
/usr/include/asm-i386/signal.h:#define SIGHUP 1
Linux_host # uname -pr
2.6.5-7.286-smp x86_64
And that's all there is to it :) As an interesting bit of trivia, if you receive an errno value from a process that's higher than 128, you can easily deduce what signal stopped that process. All you need to do is deduct 128 from the return code, like this:
host # sleep 2
host # echo $?
0
host # sleep 200
^C
host # echo $?
130
In this case, when run normally, our sleep command exited with a return code of 0 (success). When we did a control-C during the sleep command, it returned a code of 130. Of course, you won't find this signal number defined in any of your include files, but, utilizing the shell's native reporting of signal interrupts, all you need to do is subtract 128 from 130 and you now know that the sleep command was killed by a signal 2, which is defined as:
SIGINT 2 /* interrupt (rubout) */ <--- on Solaris
and
SIGINT 2 /* Interrupt (ANSI). */ <--- on the Linux box I'm using.
Both answers, although formatted differently, indicate the same signal. Nice :)
Below, I've listed the standard signals for your convenience (from a fairly generic file - signum.h) and take no credit for writing them myself ;)
Enjoy,
Some Basic signals from /usr/include/bits/signum.h on Linux (Note that I'm only listing the basic signals and not I/O polling signals, etc)#define SIGHUP 1 /* Hangup (POSIX). */
#define SIGINT 2 /* Interrupt (ANSI). */
#define SIGQUIT 3 /* Quit (POSIX). */
#define SIGILL 4 /* Illegal instruction (ANSI). */
#define SIGTRAP 5 /* Trace trap (POSIX). */
#define SIGABRT 6 /* Abort (ANSI). */
#define SIGIOT 6 /* IOT trap (4.2 BSD). */
#define SIGBUS 7 /* BUS error (4.2 BSD). */
#define SIGFPE 8 /* Floating-point exception (ANSI). */
#define SIGKILL 9 /* Kill, unblockable (POSIX). */
#define SIGUSR1 10 /* User-defined signal 1 (POSIX). */
#define SIGSEGV 11 /* Segmentation violation (ANSI). */
#define SIGUSR2 12 /* User-defined signal 2 (POSIX). */
#define SIGPIPE 13 /* Broken pipe (POSIX). */
#define SIGALRM 14 /* Alarm clock (POSIX). */
#define SIGTERM 15 /* Termination (ANSI). */
#define SIGSTKFLT 16 /* Stack fault. */
#define SIGCLD SIGCHLD /* Same as SIGCHLD (System V). */
#define SIGCHLD 17 /* Child status has changed (POSIX). */
#define SIGCONT 18 /* Continue (POSIX). */
#define SIGSTOP 19 /* Stop, unblockable (POSIX). */
#define SIGTSTP 20 /* Keyboard stop (POSIX). */
#define SIGTTIN 21 /* Background read from tty (POSIX). */
#define SIGTTOU 22 /* Background write to tty (POSIX). */
#define SIGURG 23 /* Urgent condition on socket (4.2 BSD). */
#define SIGXCPU 24 /* CPU limit exceeded (4.2 BSD). */
#define SIGXFSZ 25 /* File size limit exceeded (4.2 BSD). */
#define SIGVTALRM 26 /* Virtual alarm clock (4.2 BSD). */
#define SIGPROF 27 /* Profiling alarm clock (4.2 BSD). */
#define SIGWINCH 28 /* Window size change (4.3 BSD, Sun). */
#define SIGPOLL SIGIO /* Pollable event occurred (System V). */
#define SIGIO 29 /* I/O now possible (4.2 BSD). */
#define SIGPWR 30 /* Power failure restart (System V). */
#define SIGSYS 31 /* Bad system call. */
#define SIGUNUSED 31
#define _NSIG 65 /* Biggest signal number + 1
, Mike
linux unix internet technology
Monday, February 11, 2008
Bit Shifting In Perl To Get Errno Values From The System Function
Hey there,
My two main reasons for putting together this little Perl script are:
1. To account for the unlikely event when mail queue monitoring commands don't work correctly, or you suspect that your system is so munged up that your OS commands aren't giving you correct information. Most Linux and Unix setups are far more robust than Windows, etc, and can take a severe beating before they go overboard. But, just like anything else, if pushed way too hard, they'll all eventually crack. Common symptoms of a total meltdown include obviously incorrect memory, swap and cpu statistics.
2. To demonstrate how to get back the actual errno result from a system command in Perl without using backticks.
This Perl script can, basically, be used on any directory and I set it up so that it takes a directory name as its only argument (so that it will work with any mail software no matter where it keeps its mail queue). You can call it simply, like this:
host # ./mailkue.sh /var/spool/clientmqueue <--- Substitute your mail queue location
We're making use of Perl's stat() and localtime() functions to parse a time-sorted directory listing. This should help you determine if there really are tons of old expired emails in your mail queue that you may or may not want to delete.
Also, probably more notably, we're using Perl's bit shift operator to extract the correct errno value from the system function. Normally, the system function only returns a scalar value indicating whether or not it executed the subshell sucessfully. If you run a program within the system function, like so:
host # $return = system("touch a");
The return value from that system command would be the numerical value of the success or failure of system's ability to invoke the shell.
If you did the same thing with backticks, your return value would be the output from that command. So you could do something as easy as this to get the real errno value:
host # $return = `touch a;echo $?`;
If you're not interested in the output, as we're clearly not in the above examples, but just want the proper return value from the command executed by the system function (as opposed to the return value of the system function... getting confusing and convoluted, here ;), you can get that by changing the above example, like so:
host # system("touch a");
host # $return = $? >> 8
And you'll know whether or not your touch was successful, not just if the system function worked. And it's guaranteed to work on Linux, Unix or any *nix system that Perl will run on.
Of course, backticks are generally simpler to use, but, under certain circumstances the above trick can come in very handy. Here's hoping it helps you out with your Perl scripting in the future!
Cheers,
This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License#!/usr/bin/perl
#
# mailkue.sh - check your mail
# queue - or anywhere else :)
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#
if ( $#ARGV < 0 ) {
print "Usage: $0 directory\n";
exit(1);
}
%months = qw(0 Jan 1 Feb 2 Mar 3 Apr 4 May 5 Jun 6 Jul 7 Aug 8 Sep 9 Oct 10 Nov 11 Dec);
chdir $ARGV[0];
system("ls -t >/dev/null 2>&1");
$status = $? >> 8;
if ( ! $status ) {
@nfiles = `ls -t`;
foreach $nfile (@nfiles) {
chomp($nfile);
$count = 1;
print "FILE: $nfile\n";
@files = `/usr/bin/find $ARGV[0]/$nfile -print|xargs -I {} ls -d1 "{}"`;
foreach $file (@files) {
chomp($file);
($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size,$atime,$mtime,$ctime,$blksize,$blocks) = stat($file);
($csec,$cmin,$chour,$cmday,$cmon,$cyear,$cwday,$cyday,$cisdst) = localtime($mtime);
$year = $cyear + 1900;
printf("%-2d: %-45s\t%10d %-3s %2d, %-4d\n", $count, $file, $size, $months{$cmon}, $cmday, $year);
$count++;
}
}
} else {
print "No mail in queue\n";
}
, Mike
linux unix internet technology
Thursday, December 20, 2007
Getting Error Values From The Middle Of A Pipe Chain In Bash
This is something very interesting I found out not too long ago, while hashing out some work with a colleague. As most administrators (or users) who do a fair amount of shell scripting know, the error status or return code (Generally referred to as "errno" in all the man pages) of a process is a fairly common method to use in determining the process flow of a script.
The one thing about the value of "errno" (or, literally, the variable "$?" in most shells) is that it's erased with each consecutive process that gets run. So if you were to run a series of command lines that echoed the return value of the grep command, the following example would be accurate (assuming the string "bob" can't be found in /home/myfile):
host # grep bob /home/myfile >/dev/null 2>&1
host # echo $?
host # 1
while this one would give you misleading information:
host # grep bob /home/myfile >/dev/null 2>&1
host # touch /home/myfile
host # echo $?
host # 0
So, on the first set of command lines, you're actually getting the return code of 1 from grep (indicating that it can't find the string "bob" in /home/myfile), while the second one gets you the return code of 0 from the touch command. "errno" always contains the return value of the last-executed command.
Which brings us around to the topic indicated in the title of this post (I promise to tie in the whole introduction about "errno" at the end; it wasn't a complete waste of your time ;). While it's easy enough to trap "errno" in any series of disconnected commands (for instance, in the second example above, if we'd echoed $? before running touch, it would have given us the correct output), I had always thought it was impossible to grab the correct value from a command in the middle of a pipe chain, like this:
host # grep bob /home/myfile 2>&1|Grep joe|xargs echo
host # echo $?
host # 0
You'll note that I purposefully capitalized the G in grep so that it would return an error code that didn't indicate success, yet - since this is a chain of commands all connected by pipes - "errno" returns the value of the xargs command, since it was the last one executed. Which means I've spent a lot of time jumping through hoops to "reword" any pipe chain so that I could extract the information I needed.
Now (and I'm almost positive this wasn't the case a few years back) the bash shell has actually taken on this predicament and come up with a nice workable solution for it(I'm waiting for it to pop up in sh and ksh, since they've been burned into my psyche over the last decade or so). In bash, if you run a series of piped-together commands, you can actually extract the value of "errno" from any command in the chain by using the shell built-in PIPESTATUS array, like so:
host # grep bob /home/myfile 2>&1|Grep joe|xargs echo
host # echo ${PIPESTATUS[@]}
host # 1 127 0
How nice is that? :) Now you can easily tell the return value of every process in a pipe-chain. The initial grep returns 1 because the string "bob" isn't in /home/myfile, the misspelled Grep returns 127 because the command can't be found and the final xargs returns 0. That solves a lot of problems and can potentially save you lines upon lines of convoluted code.
The one thing about it that can be frustrating is that it behaves in much the same way as "errno" (See, I told you I'd bring it back around ;). If you don't capture the output immediately (or dish it off into another variable), the array will zero out and contain no values as soon as you enter your next command, like so:
host # grep bob /home/myfile 2>&1|Grep joe|xargs echo
host # touch /home/myfile
host # echo ${PIPESTATUS[@]}
host # 0
At this point, after we've executed the touch command, the PIPESTATUS array has been cleared out, just like "errno" gets written over, even though we haven't executed another pipe chain. Its behaviour is basically identical. Below, we show that, once the array has been written over, its size gets reduced to 1 ( The single return value of the last executed command) and we further prove that the array really has been clipped down to one variable by attempting to print the first and second values; the second of which doesn't exist. Continued from above:
host # echo ${#PIPESTATUS[@]} <--- Here we ask bash for the size of the PIPESTATUS array
host # 1
host # echo ${PIPESTATUS[0]} <--- Here we check the first variable in the PIPESTATUS array
host # 0
host # echo ${PIPESTATUS[1]} <--- Here we check the second variable, which now doesn't exist
host #
This is easy enough to get around, however, since - just like "errno" - you can assign that array to another array before you execute another command, like so:
host # grep bob /home/myfile 2>&1|Grep joe|xargs echo
host # new_array=${PIPESTATUS[@]}
host # touch /home/myfile
host # echo ${PIPESTATUS[@]}
host # 0
host # echo ${new_array[@]}
host # 1 127 0
If you knew this already, I envy you the convenience you continue to enjoy. For the rest of us; a pleasant surprise :)
Best Wishes,
, Mike
linux unix internet technology
