Wednesday, December 31, 2008

New Year,s Eve News: Unix And Linux Users Beware

Hey there,

As part of a non-existent tradition, composed of this year's "New Year's Eve post" (being written as I type ;) and last year's zany and over-the-top network port querying script post, we're going to make this year's EOY post short, to the point, hopefully fun and somewhat worthwhile, so you can leave work early without getting it ok'ed and begin making bad decisions as soon as possible ;)

In all seriousness, if you go out tonight, have one (or maybe three or four ;) on me, but try to stay safe. According to a recent study by law enforcement and the National Association of Situations You'll Regret Later (NASYRL), lots of people get into, sometimes fatal, accidents and/or end up in jail for driving under the influence during the holidays. Everyone here wishes you a safe, happy and fun New Year's Eve and highly encourage you to partake of your legal drug of choice in quantities that will make you question your prior state of mind tomorrow (we can't officially condone the other stuff, but would recommend - if we were in the business of minding yours ;)- that you do whatever you want to do in moderation. Although certain substances have a higher lethality-rate than others, excess is the enemy. Basic food and water will demonstrate this very well any day of the year ;).

And, if possible, if you must go home at the end of the night, please find someone sober to drive you. If you're that sober person, please do what you can to round up a few rummies that need a lift ;)

And here's a little cautionary tale from 3 years ago, ripped from today's headlines! (wait... no... that's this week's episode of Law & Order ;) This piece is actually from LinuxReviews.org regarding the dangers of using a text web browser, especially during times of crisis and/or excess. Hopefully, it will scare the cr@P out of you and you'll begin doing what you're told. If not, and you either enjoy it, hate it or feel absolutely nothing for it, yet still refuse to walk the line, we salute you!

Have fun this New Year's Eve! Best wishes for your safety and maximum enjoyment from all of us :)

Cheers,

NOTE: We left in all the spelling mistakes from the original articles to maintain the integrity of the sources and, also, to simulate drunken journalism in keeping with the night's theme ;)



Solaris-user arrested by British Police for using Lynx


Lynx is a popular text-based browser user by Unix and Linux users. British Telecom, who are the web site administrator for a Tsunami aid site mistook a Lynx web browser user who donated money for a hacker because of the strange user agent the browser leaves in the logs and reported the user to the police.



The police, apparently equally incompetent, raided the private citizen who made the donation swat-style while he was having a quiet lunch.



BBC reported this as Officers from the Metropolitan Police's Computer Crime Unit have begun an inquiry after BT blocked the attempt on New Year's Eve. A 28-year-old man from east London was arrested and released on bail in connection with alleged offences. and this is a good example of how state media reports violent crimes against private citizens when done by the same stats police force.



The persons only crime was to use the non-standard Internet browser Lynx.



From a mailing list:






For donating to a Tsunami appeal using Lynx on Solaris 10. BT (British Telecom) who run the donation management system misread an access log and saw hmm thats a non standard browser not identifying it's type and it's doing strange things. Trace that IP. Arrest that hacker.



Armed police, a van, a police cell and national news later the police have gone in SWAT styley and arrested someone having their lunch.



Out on bail till next week and preparing to make a lot of very bad PR for BT and the Police....



So just goes to show if you use anything other than Firefox or IE and you rely on someone else to interogate access logs or IDS logs you too could be sitting in a paper suit in a cell :(






What is Lynx?



Lynx, Links, Dillo and other rare browsers are used by thousands browsers are used by thousands of Linux and Unix users world wide, but they only make up a very small percentage of visits to mainstream entertainment sites. This may because many such sites frequently use javascript, flash and other garbage non-standard elements that make these sites impossible to render in standard-compliant browsers.



Average Entertainment Website, Browser market share Desember 2004



























202932 visitsBrowser
53.54%Internet Explorer 6.0 (Windows)
27.10%Gecko 1.7.x (Mozilla/Firefox/Galeon/Epiphany/etc)
4.21%Internet Explorer 5.0
4.14%Safari (OSX)
2.54%Opera




The picture is very different for Linux related sites.



Linux Reviews, Browser market share Desember 2004



























56360 visitsBrowser
36.63%Gecko 1.7.x (Mozilla/Firefox/Galeon/Epiphany/etc)
17.58%Internet Explorer 6.0 (Windows)
10.43%Internet Explorer 5.0
3.37%Opera
2.83%Konqueror




Lynx was not in the entertainment site numbers and had 0.53% share at Linux Reviews.



SourceForge, Slashdot, Freshmeat, Lxer and other sites like these have regular visitors who use rare browsers, but if you are using Lynx then you may want to stay clear of official UK government web sites and perhaps also other government web sites in general.






, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Tuesday, December 30, 2008

Unix And Linux Easter Eggs For The Wrong Holiday

Hey there,

Today, since it's just past Christmas and almost New Year's, I figured this would be a great time to trot out some Linux and/or Unix Easter eggs. Actually, it doesn't make sense at all, but if you can put aside your burnt-in sense of the chronological order of the holidays, these can still be fun ;)

I found all of the Easter Eggs for today at a site with the very strange name Eeggs.com. I don't know what an eegg is, and I'm not sure that I want to know, but they have a great collection of Easter Eggs for all manner of OS' ;) I spent most of my time in their Linux section, but you could spend hours on other sections of their site and only occasionally be reminded that you're still at work. Of course, in all seriousness, if you're at work, the thought of driving home as soon as possible is keeping you aware of your location at all times ;)

The following are a few of the cooler ones I ran across (AND could personally verify). If you get a chance, drop by Eeggs.com and submit a support email asking why "eegs" isn't in the dictionary when "ain't" is ;)

1. Fun with PHP. This has worked with every site I've tested it against. The key here is just to find a php-enabled site, and navigate to a php page. Then, all you need to do is pass the php page a few arguments on the browser command line to find these four gems.

For a working example, we'll look at linuxandunixupdates.com's index.php page. Using that URL, we can add the following four strings and get the following four easter eggs. All of the links in this section are set to open up in new windows, so you can click on the link above and add the strings manually, or you can just click on any of the links below. I've also included a picture of the outcome of running those commands below each "magic string" just in case you're worried that I might be luring you into clicking on a redirected link or something else I don't have the time to invest in doing properly right now ;) You should be able to replicate this on any php page on any site anywhere. I haven't been able to fully test the veracity of that claim, but it appears to be true so far!

a. Add ?=PHPE9568F34-D428-11d2-A769-00AA001ACF42 to the end of your URL to see this picture:

php logo

b. Add ?=PHPE9568F35-D428-11d2-A769-00AA001ACF42 to the end of your URL to see this picture:

zend engine 2 logo

c. Add ?=PHPE9568F36-D428-11d2-A769-00AA001ACF42 to the end of your URL to see this picture:

squiggly php logo

d. Add ?=PHPB8B5F2A0-3C92-11d3-A3A9-4C7B08C10000 to the end of your URL to see the PHP Credits. This page looks exactly like the standard info.php page, but lists all the developers who worked on each component. I haven't included it here because it's incredibly long and there are more Easter Eggs to get to before we all forget why we're here :)

2. MAGIC reboot times in the Linux Kernel. This one is interesting, and a bit of a puzzle, since the original entry only gives the answer to the first time (they're all significant to Linux in some way). In any event, you can find these times by looking in /usr/include/linux/*.h and grepping for LINUX_REBOOT_MAGIC. As you can see, below, in our includes, they're all in reboot.h:

host # grep LINUX_REBOOT_MAGIC /usr/include/linux/*.h
/usr/include/linux/reboot.h:#define LINUX_REBOOT_MAGIC1 0xfee1dead
/usr/include/linux/reboot.h:#define LINUX_REBOOT_MAGIC2 672274793
/usr/include/linux/reboot.h:#define LINUX_REBOOT_MAGIC2A 85072278
/usr/include/linux/reboot.h:#define LINUX_REBOOT_MAGIC2B 369367448
/usr/include/linux/reboot.h:#define LINUX_REBOOT_MAGIC2C 537993216


MAGIC2 (as well as the MAGIC2A, B and C) is where you'll find the Easter Egg. If you take any of those values and convert them into regular time (using Perl, for instance), they resolve to an important date in Linux history.

host # perl -e 'print localtime(672274793). "\n";'
Sun Apr 21 17:59:53 1991
host # perl -e 'print localtime(85072278). "\n";'
Mon Sep 11 10:11:18 1972
host # perl -e 'print localtime(369367448). "\n";'
Mon Sep 14 21:04:08 1981
host # perl -e 'print localtime(537993216). "\n";'
Sun Jan 18 12:33:36 1987


Sun Apr 21 17:59:53 1991 is supposedly (and I'm not using the word "supposedly" to cast any more doubt than any reasonable human being would have. I'm not sure if the following is true, so I can only "suppose" that the folks who submitted these Easter Eggs aren't just prepping a new Wikipedia page. Just kidding, of course. Everything in Wikipedia is true ;)) the date Linus Torvalds first began writing Linux. The rest is left up to us to figure out. Something tells me the answers are all somewhere in this Linux Online Timeline.

3. And lastly, so there's plenty more left for you to check out at Eeggs.com, I really enjoyed this last one (actually there were a few others I'm dying to try, along the same lines, but don't have the proper OS' to validate right now) since I'm a "huge" fan of Douglas Adams, even beyond the HitchHiker's Series (although lots and lots of people got really upset over Mostly Harmless when he chose to wrap up the HitchHiker's Trilogy (with the 5th book in the series) in a manner that, apparently, was extremely dissatisfying to ardent fans of the series. I don't begrudge them their opinions. I dug it. I'm only sorry that he passed away and that we'll never know if the The Salmon of Doubt was going to be the sixth HitchHiker's book (answering the fan's complaints, at worst) or the next Dirk Gently novel.

Back to planet earth ;) If you open up vim, and type the following:

host # vim
[esc]:help 42


with the [esc]: being the actual "escape" or "esc" key, followed by the colon (:)

You'll, sadly, not get an explanation of the answer to the meaning of life, the universe and everything, but the payoff's just as pleasant :)

What is the meaning of life, the universe and everything? *42*
Douglas Adams, the only person who knew what this question really was about is
now dead, unfortunately. So now you might wonder what the meaning of death
is...

==============================================================================

Next chapter: |usr_43.txt| Using filetypes
...


Hope you all enjoyed those Easter Eggs and, should you decide to look for more, happy hunting :)

Cheers,

, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Monday, December 29, 2008

Finding Your Yahoo Search Index Rank From The Unix Or Linux CLI

Hey There,

Today we're going to continue in our ongoing quest to rank highly in search engine results while simultaneously messing with them a lot ;) Previously, we've put out scripts to find your MSN search index rank from the CLI and to find your Google search index rank from the CLI. This is, of course, another script that, although it fits in the same category as the other two, is distinctive in several ways:

1. This time we're scouring Yahoo's search results.

2. The search results are being parsed differently, so that it makes it easier for the script to detect the point at which Yahoo cuts you off and won't let you do another search for a good 10 or 15 minutes.

3. The parsing code has been minimized, somewhat, so that it may actually be readable by humans who aren't pipe-chain-sed-awk-regular-expression fanatics ;)

4. See the continuation below for a very interesting analysis of Yahoo's robot-tolerance.

BUT FIRST, THIS VERY IMPORTANT NOTE: If you use wget (as we are in this script), or any CLI web-browsing/webpage-grabbing software, and want to fake the User-Agent, please be careful. Please check this online article regarding the likelihood that you may be sued if you masquerade as Mozilla, from the folks who maintain wget themselves.

This Yahoo script, of course, is only slightly different than the original Google and MSN scripts, although the differences are significant enough that most of the core was rewritten completely. The script, itself, operates the same way our Google and MSN search index page rank scripts do, insofar as executing it from the command line goes. There are, at least, three different ways you can call it. The most basic being:

host # ./yrank www.yourdomain.com all these key words

It doesn't matter if they're enclosed in double quotes or not. If you "really" want to get the double quote experience, you just need to backslash your double quotes.

host # ./yrank www.yourdomain.com \"all these key words\"

Other ways include creating files with the URL and keyword information (same format as the command line) and feeding them to the script's STDIN:

host # cat FILE|./mrank
host # ./mrank <FILE


Point 4 (continued from above): Yahoo robot search tolerance as compared with Google. This is actually quite interesting since, I believe, the general assumption is that Google is far less tolerant of seemingly-human interaction with its search than Yahoo is. However, in this case (and we've repeated this experiment over and over again) the opposite is, in fact, true. Check it out! :)

The setup is that we've created a simple file called "searchterms" to feed to both the grank and yrank scripts. It contains the following information:

host # cat searchterms
linuxshellaccount.blogspot.com unix linux
linuxshellaccount.blogspot.com linux unix
linuxshellaccount.blogspot.com unix and linux
linuxshellaccount.blogspot.com linux and unix
linuxshellaccount.blogspot.com unix
linuxshellaccount.blogspot.com linux
linuxshellaccount.blogspot.com perl script
linuxshellaccount.blogspot.com shell script


Then, we put each search engine to the test. Each grabbing results at 100 per page. You'll notice that the Google search engine makes it through the entire bunch without kicking us to the curb ;)

The image below was captured with a rear-view mirror and is, therefore, actually larger than it appears. Click below to see it in "life size" ;)

Google Robot Tolerance Test

And here is the exact same experiment; this time run against Yahoo's search engine. It ...just ...barely ...nope. It doesn't make it, again ;)

This image was taken in "Wallflower-Vision" - sometimes referred to as the "Shrinking-Violet Protocol." To see it in its natural, and unabashedly large, state, just click on it below and it will almost definitely come out of its corner ;)

Yahoo Robot Tolerance Test

Our initial suggestion is to change this line in the script (decreasing the number that you divide RANDOM by will increase the maximum wait time between tries):

let random=${RANDOM}/600


The value of 600 roughly approximates between 0 and 60 seconds of wait time. Reducing that number to 300 will roughly approximate wait times between 0 and 120 seconds, etc. The numbers generated by bash's RANDOM variable may vary depending upon your OS, system architecture, etc.

Another possibility, which we didn't have time to fully test (so it's not included in the script) is that Yahoo may actually object to the direct manipulation of the GET request and would probably respond more favorably if we extracted the URL value for our next successive request from the "Next" button on the search page, rather than moving on to the next valid (although coldly calculated) GET string to bring up the next set of 100 results. Time will tell. Experimentation is ongoing.

Hope you enjoy it, again, and you're still enjoying the holiday's. Even if none of them apply to your religious or moral belief-system, at least you get some paid time off of work :)

Cheers,


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/bash

#
# yrank - Get yer Yahoo's Out ;)
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

if [ $# -lt 2 -a $# -ne 0 ]
then
echo "Usage: $0 URL Search_Term(s)"
echo "URL with or with http(s)://, ftp://, etc"
echo "Double Quote Search If More Than 1 Term"
exit 1
fi

if [ $# -eq 0 ]
then
while read x y
do
url=$x
search=$y
$0 $x "$y"
done
exit 0
else
url=$1
shift
search=$@
fi

search_terms=`echo $search|sed 's/ /+/g'`
start=1
count=1

echo "Searching for URL $url with search terms: $search"

results=`wget -O - http://search.yahoo.com/search?p=${search_terms}\&ei=UTF-8\&fr=yfp-t-501\&pstart=1\&b=$start 2>/dev/null|sed -n 2p 2>&1|sed 's/^.* of \([0-9,]*\) for .*$/\1/'`

while [ $start -lt 1001 ]
do
wget -O - http://search.yahoo.com/search?p=${search_terms}\&ei=UTF-8\&fr=yfp-t-501\&pstart=1\&b=$start\&n=100 2>&1|grep "error 999" >/dev/null 2>&1
screwed=$?
if [ $screwed -eq 0 ]
then
echo
echo "You have been temporarily barred due to excessive queries."
echo "Please change the \"random\" variable in this script to a"
echo "lower value, to increase wait time between queries, or take"
echo " 5 or 10 minutes before you run this script again!"
echo
exit 1
fi
wget -O - http://search.yahoo.com/search?p=${search_terms}\&ei=UTF-8\&fr=yfp-t-501\&pstart=1\&b=$start\&n=100 2>/dev/null|sed -n 2p 2>&1|sed 's/^.* of [0-9,]* for //'|sed 's/<[^>]*href="\([^"]*\)"[^>]*>/\n\1\n/g'|sed -e :a -e 's/<[^>]*>//g;/</N;//ba'|grep "^http"|sed '/^http[s]*:\/\/[^\.]*\.*[^\.]*\.yahoo.com/d'|sed '/cache?ei/d'|uniq|while read line
do
echo "$line"|grep $url >/dev/null 2>&1
yes=$?
if [ $yes -eq 0 ]
then
echo "Result $count of approximately " $results " results for URL:"
echo "$line"
exit 1
else
let count=$count+1
fi
done
end=$?
if [ $end -eq 1 ]
then
exit 0
else
let start=$start+100
let count=$count+100
let new_limit=$start-1
let random=${RANDOM}/600
echo "Not in first $new_limit results"
echo "waiting $random seconds..."
sleep $random
fi
done


, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Sunday, December 28, 2008

The NetSlave Quiz - Unix And Linux Humor

Hey There,

Todays Unix and Linux humour find brings back a lot of memories from the dotcom days for me. Humungous Stock Option incentives, constant excitement about a possible IPO (Coupled with constant worry about getting laid off next week ;) and, of course, the frequent (and often mandatory) drinking on the job. Yes, there was a "valid" reason software upgrades were published before the QA phase back then ;) This is, strangely enough, a practice that continues to this day... except without the booze. In today's fast paced world, managers are expected to make bad decisions with a clear head ;)

And, before I get myself in too much "more" trouble, here's the bit I found on softpanorama.org called the NetSlave Quiz. Of course, the full version is available on their site, along with a quit a bit more!

Hope you enjoy it and, if you're like me, don't be ashamed if you start feeling a little bit misty ;)

Out of respect for the original publisher's rights, I have not modified any of the links within the piece. They don't look like affiliate links to Amazon.com, but, if they are, he (or she) will be getting paid if you buy. I have a strict no-hijacking policy (more of a moral/ethical thing than any worry about the legality of it. I just wouldn't want anyone to do it to me. Kind of like that "golden rule" everyone keeps talking about ;)

Cheers :)



Softpanorama Open Source Humor Archive



A variation of Steve Baldwin's

 NETSLAVE QUIZ


Adapted from the Amazon.com  NetSlaves
page by Nikolai Bezroukov


PART I: YOUR WORK LIFE


1. AT YOUR LAST JOB INTERVIEW, YOU EXHIBITED:


A. Optimism

B. Mild Wariness

C. Tried to overcome headache. I was really tied

D. Controlled Hostility


2. DESCRIBE YOUR WORKPLACE:


A. An enterprising, dynamic group of individuals laying the groundwork for
tomorrow's economy.

B. A bunch of geeks with questionable social skills.

C. An anxiety-ridden, with long hours and a lot of stress because of
backbiting bunch of finger-pointers.

D. Jerks and PHB


3. DESCRIBE YOUR HOME:


A. Small, but efficient.

B. Shared and dormlike.

C. Rubble-strewn and fetid.

D. I have a personal network at my home with three or more connected
computers and permanent connection to the Internet


PART II: YOUR RELATIONSHIP WITH TECHNOLOGY


4. MY PERSONAL COMPUTER:


A. Maximizes my productivity.

B. Is my best friend.

C. Lets me work all kinds of crazy hours at home.

D. My computers can heat my apartment.


5. MY ALPHANUMERIC PAGER:


A. Is a convenient way to stay in touch with my office.

B. I do not like when it sometimes goes off at night

C. Has been temporarily disabled and I receive warning because of that

D. I have it all the time and always wear a cell phone on my belt.


PART III: YOUR INTERNET KNOWLEDGE


6. WHAT DOES THE "I" IN IPO STAND FOR?


A. Initial

B. Imaginary

C. Impossible

D. Insolvent


7. WHAT DOES THE "E" IN E-COMMERCE STAND FOR?


A. Electronic

B. Exciting

C. Error-prone

D. Exploitation


8. WHAT ARE THE "THREE C'S OF THE INTERNET?"


A. Content, Community and Commerce

B. Cults, Chats and Clutter

C. Cheetos, Chinese Food and Coke

D. Crap, Costs and Chaos


9. WHAT IS THE "WINDOWS ENVIRONMENT"?


A. A graphical operating system.

B. A ploy for world domination.

C. A bug-ridden, proprietary disaster. Open source is the way to go.

D. Linux rulez.


10. WHO IS THE REAL "FATHER OF THE INTERNET"?


A. Vice-president Gore

B. Vinton Cerf

C. Linus Torvalds

D. George Orwell


11. WHY WAS THE INTERNET CREATED?


A. To simplify a way to bid out jobs to an immigrant workers.

B. To boost the Dow over 10,000

C. To create a new bunch of billionaires

D. To create an IPO gold rush.


PART IV: YOUR HOPES AND FEARS FOR THE FUTURE


12. WHY ARE YOU WORKING IN THE INTERNET BUSINESS?


A. To get rich.

B. To get by.

C. Linux rulez

D. Nobody else will take me.


13. DESCRIBE YOURSELF POLITICALLY:


A. Republican

B. Democrat

C. Anarchist

D. Luddite


14. DESCRIBE YOUR RELIGION:


A. Monotheist

B. Atheist

C. Open Source

D. Linux


15. WHAT WILL HAPPEN IN 2000?


A. It's revenge time for brick-and-mortars.

B. Your girlfriend may dump you for a doctor or Venture Capitalist.

C. You are feared that your health insurance may run out and you may not
have new contracts to renew it.

D. You are feared that you will be fired or outsourced with no equity.


HOW TO SCORE YOUR RESULTS



  • If you answered “A” to any of the above questions, your IEQ
    ("Internet Exploitation Quotient") is zero – you’re not a
    Net Slave. Reading The Cathedral and Bazaar can be beneficial

  • If you selected "B," you’ve probably just started working in
    the technology business or lead some sort of charmed (mythical?) existence
    the rest of us only dream about.  Reading NetSlaves
    and Critique
    of CatB
    recommended

  • For those answering mostly  “C” with some D -- you are on your
    way to becoming NetSlave. Reading NetSlaves
    strongly recommended.

  • For those who answered mostly  “D,” -- you are Net Slave. Also
    you should probably contact a mental health professional immediately,
    reading NetSlaves
    probably will not help much, but you can try.





, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Saturday, December 27, 2008

Unix and Linux WebTech Humor: Bourne Shell Server Pages

Happy Saturday :)

Here's a little humour, from way back when, that's as true today as it was whenever it was originally published. It's a joke (or is it?) that puts forth a bold and visionary alternative to more complicated server page technologies, like ASP, JSP, etc. As the author suggests, it may all be a bunch of sh##, but that's by design ;)

This shining example of a Linux and Unix joke that you can actually put into practice, comes from Mark Lindner's Public Domain Software Site (It's actually his software, that he developed... not a bunch of unclaimed stuff that he put together in one place ;). To check out the original version of this page (including extra specifications, quotations and a bunch of other sh## <-- All this scatological prose will tie in nicely when you read the article, I promise ;), check out the SHSP Project Home Page. For this alone, Mark deserves a contribution. Check out his main page and, if you like this sh##, consider funding his future ...stuff ;)

Enjoy and remember to be kind to animals (even other humans ;)



Bourne Shell Server Pages



Introduction



Every work of art begins with a sudden flash of inspiration. In
my case, it all started when I received the following email from
my friend, Steve:






Hey Mark:

I just came up with a totally genius idea: Bourne Shell Server
Pages.

Why should shell languages be the only ones left out of the
${YOUR_LANGUAGE_HERE} Server Pages fad?

The way I see it, the advantages of SHSPs are as follows:


  1. runs everywhere (or at least everywhere on Unix and since I consider
    Windows to be nowhere...)
  2. slo-o-ow
  3. goofy

Now most rational people would consider points 2 & 3 to be drawbacks,
but as far as I can tell they are flat-out requirements for any popular
Web-based technology.

SHSPs are the Wave of the Future, so line up and sign up today!

Tune in next week when we invent 6502 Assembler Server Pages.


—Steve





Even as I was reading this email, my mind was racing,
subconsciously piecing together the implementation details. The
Bourne shell, it turns out, is quite well suited for this type of
task...much more so, I dare say, than Java or C. The solution turned out to be both simple and elegant.



This document chronicles my foray into the not-so-fascinating
world of ${YOUR_LANGUAGE_HERE} Server Pages
technology. All of the code I developed is available for free
download, so you can use Bourne Shell Server Pages to build your
very own killer Web application.



Implementation





The basic idea behind all server page technologies is this:
rather than writing code that generates an HTML document
on-the-fly by writing it out as a series of print
statements, you start with a "skeleton" HTML document and embed
the code right inside it. Voila! Instead of having a tangled,
unreadable, unmaintainable mess of HTML embedded in source code,
you have a tangled, unreadable, unmaintainable mess of source
code embedded in HTML.



Bourne Shell Server Pages are ordinary ASCII text files, with
the special extension .shit, which denotes
"Shell-Interpreted Template." The result of invoking the page
compiler on a .shit file, is, naturally, a shell
script. (It occurred to me that this file extension might seem
objectionable to some, but since it quite accurately—if
unintentionally—conveyed my sentiments toward Web technology in
general, I decided that it should be left unchanged.)



There are three basic elements that can be embedded in HTML:
blocks of code, expressions, and include directives. Each of
these is indicated by special HTML-like tags which are processed
by the page compiler to produce an executable entity (a Java servlet in
the case of JSP, for example), which, when run, produces the
resulting HTML document. I'll discuss each of these elements in
turn.


Code Blocks

Arbitrary blocks of code can be
inserted within an HTML document. Bourne Shell Server Pages
provides the <$ ... $> tags for this
purpose. Shell code that appears between these tags will be
executed, and the output of that code will be inlined in the
HTML in its place. For example:





<html>
<body>
<h1><$ echo "Hello, world!" $></h1>
</body>
</html>



The code can of course span multiple lines, and can be of
arbitrary complexity. What's more, all of the code blocks in a
given Bourne Shell Server Page are in reality part of the same
shell script, so they share a global state: for instance, a
block can reference variables that were assigned in an earlier
block.



Expressions
Arbitrary Bourne Shell expressions
can be inserted into HTML using the <` ... `> tags. These tags serve the same purpose as
do backticks in the Bourne Shell: the expression is
evaluated and the result is inlined in its place. For example,
to set the title of the document based on the contents of the
shell variable $title, one might do this:





<title><` $title `></title>



The above example assumes that a value was assigned to the
variable $title earlier in the page, namely, within
a <$ ... $> block.



Includes
Server pages can be composed of other
server pages using the include directive. The page compiler
evaluates pages recursively, so the nesting can be arbitrarily
deep. In Bourne Shell Server Pages, the tag <^ ... ^> indicates an include. For example the tag:





<^ more.shit ^>




will be replaced with the result of evaluating the Bourne
Shell Server Page more.shit.




The Bourne Shell Server Pages page compiler (shspc.sh),
which is tasked with converting .shit files into shell
scripts, actually consists of a series of sed
commands. These commands perform the following substitutions:


  • All contiguous segments of HTML are wrapped within here
    documents
    .

  • Include directives are replaced with recursive calls to
    shspc.sh to process the included Bourne Shell Server Pages.

  • Expression tags are replaced with a call to the shell's
    eval command.



The result is a script which, when interpreted by the Bourne
Shell, produces the desired HTML page to standard output.



Most other server page technologies rely on the presence of a Web
server, which is itself a complex piece of software that often
requires a substantial amount of maintenance. Bourne Shell Server
Pages has no such requirement, as it includes its own Web server,
implemented as a small shell script (in.httpd.sh) that is launched
from inetd. While certainly not a full-featured Web server,
it does quite an admirable job of serving HTML documents, images,
and of course, Bourne Shell Server Pages.



So What About Web Services?



How does the Bourne Shell Server Pages technology fit into the
bigger picture of Web Services? It's a legitimate question. For
that matter, what the hell are "Web Services" anyway?



I've read quite a bit about Web Services, and have had some
in-depth, first-hand experience with the technologies that form
their underpinnings. To the best of my knowledge, here is an
accurate definition of the term:




Web Services  noun  A software development
meme that espouses the notion of tying together disparate software
components via a crude, non-typesafe, remote procedure call (RPC)
mechanism that consists of sending and receiving data encoded in
an excessively verbose, plaintext format (XML) over a largely
inelegant, stateless file transfer protocol (HTTP).





Whew. That doesn't sound glamorous or exciting at all. It's not
even object oriented. There must be more to this than just
inferior reinterpretations of old ideas? Sadly, there isn't.



One particularly curious aspect of Web Services is that all
communication between components must take place over TCP port
80. The other ports (all 65,534 of them) constitute a veritable
Pandora's Box of perceived dangers, horrors and evils, and so
Thou Shalt Not Bind Them. It's painfully clear that Web
Services exists along three distinct axes: a technical one, an
emotional one, and a decidedly religious one.



It turns out that aside from the fact that it employs HTTP, the
network protocol of the World Wide Web, Web Services really has
nothing to do with the Web. "HTTP Services" would probably be a
more appropriate name for the technology. (Sure, that sounds
pretty silly...but it is what it is.)



The answer to the thesis of this section is thus very simple:
Bourne Shell Server Pages doesn't fit into Web Services,
any more than does any other ${YOUR_LANGUAGE_HERE} Server
Pages technology.


That being said, Bourne Shell Server Pages is a compelling
approach to the problem of providing Web access to existing
software components, which incidentally happens to be the major
focus of Web Services. Other Web technologies require complex
software plumbing in the form of adapters, bridges, and frameworks
to expose legacy software systems to the Web. However, due to the
radically expressive and flexible nature of the Bourne Shell,
Bourne Shell Server Pages makes this problem all but disappear:
any program on the system can be invoked from within a Bourne
Shell Server Page, its output becoming immediately available on
the Web.


For example, providing a Web-based search front-end to a large
CSV-data file is almost trivial with Bourne Shell Server Pages,
since one can rely on the presence of powerful utility programs
for manipulating data, e.g.:






<table>
<tr>
<th>Last Name</th>
<th>First Name</th>
<th>Phone</th>
<th>State</th>
</tr>

<$ IFS=','; grep -i $pattern datafile.csv | \
( while read lname fname phone state;
do
$>

<tr>
<td><` $lname `></td>
<td><` $fname `></td>
<td><` $phone `></td>
<td><` $state `></td>
</tr>

<$ ;; done ) $>

</table>





The true power of Bourne Shell Server Pages stems from the fact
that any program can be executed from within a Server Page,
with no intermediate "glue" logic required. Think of SHSP as "Web
Services without all the layers upon layers of crap."



Downloads





You, too, can build exciting dynamic Web content with Bourne
Shell Server Pages.

There are no pre-requisites for deploying
the technology other than a working Bourne Shell and a decent
operating system to run it on. (Those modest requirements,
unfortunately, rule out these products.)




shspc.sh - page compiler (710 bytes)
in.httpd.sh - Web server (834 bytes)




Installation is simple. Place the above files in
/usr/local/bin, and then install this href="http">http file in /etc/xinet.d (or the
equivalent). Then send a HUP signal to xinetd.

Securing the installation is left as an exercise for the reader.



Credits



Web technology is certainly a controversial topic. Despite the
fact that building Web applications is about as enjoyable as
having one's private parts caught in a cement mixer, it continues
to be an extremely popular software development paradigm. It's
been said that although there is a certain class of problems
for which Web technology is the appropriate solution, your problem
is almost certainly not among them, no matter what your problem
is
. But one would be hard-pressed to convince the industry of
it.



As do most belief systems, the Web technology religion has its
share of rabid followers. In order to avoid their wrath, I remain,
humbly and respectfully yours, Anonymous.



Some astute and enterprising readers have graciously submitted
bugfixes and improvements for SHSP. I have included these into the
official release.




, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Friday, December 26, 2008

The Twelve Days Of Unix

Hey There,

Thanks to the staggered posting system we have here, this joke is post-dated. Despite that, however, we here would like to have wished you and yours a Merry Christmas and/or Happy Holidays ;) Like the falling snow out here in Illinois (which will, by the way, never EVER stop until we're all buried alive) you can expect plenty of fluff over the holiday season, up to, and especially including New Years Day. ...okay, and almost definitely the night before that ;)

Here's a little something for you from Junauza.com from their Unix and Linux Christmas Jokes Page. Be sure to check it out for even more festive funnies and other assorted humor.

Hope you all enjoy it and Happy belated Holidays, again (or, perhaps, soon ;),



The Twelve Days of UNIX
Lyrics: Evan Leibovitch [evan@telly.on.ca]

On the first day I left it, my Unix gave to me:
A burnt-out V.D.T.

On the second day I left it, my Unix gave to me:
Two faulty tapes;
And a burnt-out V.D.T.

On the third day I left it, my Unix gave to me:
Three heads crashed;
Two faulty tapes;
And a burnt-out V.D.T.

On the fourth day I left it, my Unix gave to me:
Four bad blocks;
Three heads crashed;
Two faulty tapes;
And a burnt-out V.D.T.

On the fifth day I left it, my Unix gave to me:
Five core dumps;
Four bad blocks;
Three heads crashed;
Two faulty tapes;
And a burnt-out V.D.T.

On the sixth day I left it, my Unix gave to me:
Six bad controllers;
Five core dumps;
Four bad blocks;
Three heads crashed;
Two faulty tapes;
And a burnt-out V.D.T.

On the seventh day I left it, my Unix gave to me:
Seven blown partitions;
Six bad controllers;
Five core dumps;
Four bad blocks;
Three heads crashed;
Two faulty tapes;
And a burnt-out V.D.T.

On the eighth day I left it, my Unix gave to me:
Eight gettys dying;
Seven blown partitions;
Six bad controllers;
Five core dumps;
Four bad blocks;
Three heads crashed;
Two faulty tapes;
And a burnt-out V.D.T.

On the ninth day I left it, my Unix gave to me:
Nine floppies frying;
Eight gettys dying;
Seven blown partitions;
Six bad controllers;
Five core dumps;
Four bad blocks;
Three heads crashed;
Two faulty tapes;
And a burnt-out V.D.T.

On the tenth day I left it, my Unix gave to me:
Ten ports a-jamming;
Nine floppies frying;
Eight gettys dying;
Seven blown partitions;
Six bad controllers;
Five core dumps;
Four bad blocks;
Three heads crashed;
Two faulty tapes;
And a burnt-out V.D.T.

On the eleventh day I left it, my Unix gave to me:
Eleven chips a-smoking;
Ten ports a-jamming;
Nine floppies frying;
Eight gettys dying;
Seven blown partitions;
Six bad controllers;
Five core dumps;
Four bad blocks;
Three heads crashed;
Two faulty tapes;
And a burnt-out V.D.T.

On the twelfth day I left it, my Unix gave to me:
Twelve boards a-blowing;
Eleven chips a-smoking;
Ten ports a-jamming;
Nine floppies frying;
Eight gettys dying;
Seven blown partitions;
Six bad controllers;
Five core dumps;
Four bad blocks;
Three heads crashed;
Two faulty tapes;
And a burnt-out V.D.T.





, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Thursday, December 25, 2008

Happy Holidays - More ASCII Art

Happy Holidays to you all and Merry Christmas To Some Of You :)

Not much more needs to be said. I hope that you, like me, are spending some time with family and friends or, at least, writing a festive shell script ;)

As usual, click the picture below and it will appear larger than it is!


Happy holidays to one and all

Cheers,

NOTE: ASCII art copy write "Joan Stark" from her ASCII Art Christmas page

echo -ne "\n"
echo -ne " | | . , |_ . ,_ \_/ _,_, |__) _ _ _ _ / \,_\n"
echo -ne " |_|_|_|_/_)| |_|_| |_\_| | (_) |_|_ | _(/_(_|_(__(/_ \__/| |_\n"
echo -ne " __/\n"
echo -ne " __ ___\n"
echo -ne " (_ _ ,_-|-|_ _ ,_ _| | ,_ \_/ _,_, ,_ |__| _ _ ,_-|-\n"
echo -ne " (__(_|_|(_|_| |_ (_|_| |_(_|_ _|_| |_ | (_) |_|_|(_ | |_(/_(_|_|(_|_\n"
echo -ne "\n"
echo -ne " _..._\n"
echo -ne " _..._ .' _ '.\n"
echo -ne " .' _ './ (a) .-'-.\n"
echo -ne " / (e) .-'-. '-,-'\`\n"
echo -ne " | '-,--'\` /\n"
echo -ne " ; / ; |\n"
echo -ne " _ / |/ .\ _\n"
echo -ne " (_\ __..;__ \ ::\ ,/_)._\n"
echo -ne " _;\;\`\`\"\"----...___...---:\"\`\` .-'_ \`'. \ '::.()\__)\n"
echo -ne " (\())(\`._'- \`-._ ,:-\" '. '. - : ; '::\--;__\n"
echo -ne " _\()\`''----...__ \`-._ '-. '._ .' | .:::|()\_)\n"
echo -ne " (_/\`;-. jgs _ ()\`-. \`\"\"--...____..-'\` ;_ .:::| \`.\n"
echo -ne " \_) _;/_)/).;()'._ (\`\.-();/_).__ .::::/().-./\`)\n"
echo -ne " (_\\\`;(()(_/(()/(. __.--;();\`\_)()\__) .::()--;__:()\n"
echo -ne " '-;---()_;\_/()\"-._ (_/ (/ /()';();--._\_::'\`.();\_)\`\n"
echo -ne " ,() (_.-(); \._ \`\`\"\"(_.();_().(\`\.-();__)\`(_/ \`--.__\n"
echo -ne " (_/ (_/() \_) __/(_/|\();__\_)\` \_)\n"
echo -ne " \` (_/ \_)\`-._);\n"


, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Wednesday, December 24, 2008

Taking Full Advantage Of "Who" On Solaris

Hey there,

First things first: Happy Christmas Eve :) Our apologies to our readers who celebrate different festivities this time of year. It's not that we don't want you to enjoy them (although our wishes are probably belated for some), just that we almost forgot about our own... What does that say about us? We don't know either ;) In the spirit of "it's never too late" - Happy Holidays (whatever they may be) from all of us :)

This post is more of an expansion, than a follow-up, to our May 2008 post on using who to find out what and when on Linux and Unix (picking only the options that appeared in most distro's version of "who"). It's also a bit more restrictive than our previous post, as we're restricting the discussion to newer Solaris' (8/9/10) implementation of the classic command (Okay, "according to Hoyle" (and I'm using that idiom loosely, since this post has nothing to do with the game of Whist ;), it can't be a classic for another 12 years, but if TNT can televise "new classics" that just came out last year, the meaning of the word has either been completely devalued or we're making liberal use of artistic license. We prefer the latter justification ;)

Solaris' implementation of who is a great deal better than in older versions, and turns what used to be a "mostly" informational command into a command that you can use to really get to the bottom of things. Following, a list of Solaris who's usage parameters and a brief discussion of each. One thing that should be noted is that the Solaris who command has no "help" switch (-h, --help, nothing), and it will produce output if you just run it without arguments. We usually go with "who -h" (since -h isn't an option) to get usage/help output. You'll have to subject yourself to the humiliating error message, but it's generally worth it ;)

Ladies and gentleman, The Options :)

1. Who straight-up (what better way to start a list of options than with the no-option option? This is what you get if you settle. Four columns: User ID (NAME), terminal (LINE), date (TIME) and local-screen/IP-address:

host # who
user001 console Dec 8 09:48 (:0)
user001 pts/3 Dec 8 09:50 (:0.0)
user001 pts/5 Dec 23 08:56 (10.99.99.99)


2. -a. This option is fairly obvious. It stands for "all" and is the equivalent to using "-Abdlprtu" which, the astute among you may have noted, does not comprise the entirety of the options available ;) We'll go over all the options that are lumped together in this option shortly (If I had more time to proofread, I wouldn't have made the word option a tag for this post ;).

host # who -a
. system boot Dec 8 09:46
. run-level 3 Dec 8 09:47 3 0 S
zsmon . Dec 8 09:47 old 307
LOGIN console Dec 8 09:47 old 331
user001 + console Dec 8 09:48 old 663 (:0)
user001 + pts/3 Dec 8 09:50 old 1015 (:0.0)
user001 pts/4 Dec 8 10:37 old 0 id= /4 term=0 exit=0
user001 + pts/5 Dec 23 08:56 . 17914 (10.99.99.99)
user001 pts/4 Dec 11 16:15 old 28003 id=ts/4 term=0 exit=0
user001 pts/6 Dec 19 10:39 old 11406 id=ts/6 term=0 exit=0
user001 pts/7 Dec 18 16:40 old 9997 id=ts/7 term=0 exit=0
user001 pts/8 Dec 16 14:05 old 5804 id=ts/8 term=0 exit=0


3. -b. This option will show you the last time the system booted. This can be very helpful (especially when used in combination with "last -x" to make use of last's full potential):

host # who -b
. system boot Dec 8 09:46


4. -d. This option purports to list out "dead" processes. The reason we phrase it in that way is that, generally, these processes (or pseudo terminals) may have been used before, but this doesn't mean that they're hanging around like a bunch of zombie processes. For instance, the following output lists 5 "dead" pseudo terminals, although none of them can be found in the output of either "ps" or "lsof" (???) In any event, it's a cool feature :)

host # who -d
user001 pts/4 Dec 8 10:37
user001 pts/4 Dec 11 16:15
user001 pts/6 Dec 19 10:39
user001 pts/7 Dec 18 16:40
user001 pts/8 Dec 16 14:05


5. -H. This one is a fantastically fun joyride through the land of the obvious. It forces who to print out the header for each column it reports on (although it does forget about the display/IP column noted in straight-up "who" output from point 1):

host # who -H
NAME LINE TIME
user001 console Dec 8 09:48 (:0)
user001 pts/3 Dec 8 09:50 (:0.0)
user001 pts/5 Dec 23 08:56 (10.99.99.99)


6. -l. Using who with the -l option lists out only "login" processes. Basically, it will only report on logins that are logged in (or appear to be logged in) to the localhost directly (no external pseudo terminals):

host # who -l
zsmon . Dec 8 09:47 old 307
LOGIN console Dec 8 09:47 old 331


7. -q. This will perform a quick who (only showing the NAME field), and is the only option that the -n option works with. If you use -n with -q, you can specify the number of returned processes you want to see per line of output, at most. By default, who -q tries to return as many results as possible on a single line:

host # who -q
user001 user001 user001

who -q -n 2
user001 user001
user001


8. -r. This option will let you know what run level your system is currently at (again, check out our previous post on using who for more specifics on all of the output "who -r" produces:

host # who -r
. run-level 3 Dec 8 09:47 3 0 S


9. -s. This option is considered the "short form," since it doesn't report any "time since last login," session activity status or PID output. who, run with this option alone is actually the default output. The one time this comes in handy is when you're using it with -a (and -H for the headers, if you want), and want to trim that output a bit. Otherwise, using this option wouldn't make sense, since you'd have to specify the flags to print the two fields you want removed ;)

host # who -s
user001 console Dec 8 09:48 (:0)
user001 pts/3 Dec 8 09:50 (:0.0)
user001 pts/5 Dec 23 08:56 (10.99.99.99)

host # who -asH

NAME LINE TIME
. system boot Dec 8 09:46
. run-level 3 Dec 8 09:47 3 0 S
zsmon . Dec 8 09:47
LOGIN console Dec 8 09:47
user001 + console Dec 8 09:48 (:0)
user001 + pts/3 Dec 8 09:50 (:0.0)
user001 pts/4 Dec 8 10:37
user001 + pts/5 Dec 23 08:56 (10.99.99.99)
user001 pts/4 Dec 11 16:15
user001 pts/6 Dec 19 10:39
user001 pts/7 Dec 18 16:40
user001 pts/8 Dec 16 14:05


10. -t: This option will show you all the times that your system clock was reset (and, yes, sometimes this output can be empty, for reasons that require no explanation ;)

host # who -t
host #


11. -T. This flag shows your tty status (referred to also, above, as session activity status). The + symbol indicates that the tty's status is "writable," the - symbol indicates that the tty's status is "not writable" and the ? symbol indicates general confusion ;) It just means that the system has no idea what the tty's status is, which generally means that it's hung:

host # who -T
user001 + console Dec 8 09:48 old 663 (:0)
user001 + pts/3 Dec 8 09:50 old 1015 (:0.0)
user001 + pts/5 Dec 23 08:56 . 17914 (10.99.99.99)


12. -u. This flag lists out (and I'm quoting from the "usage" output) "useful information." That isn't to say that any other output you can get from who is completely useless. Although, the terminology does seem to cast a shadow... ;)

host # who -u
user001 console Dec 8 09:48 old 663 (:0)
user001 pts/3 Dec 8 09:50 old 1015 (:0.0)
user001 pts/5 Dec 23 08:56 . 17914 (10.99.99.99)


13. -m. This flag limits the information to the current terminal session only. As you can see below, we're logged in using pseudo tty /dev/pts/5:

host # who -m
user001 pts/5 Dec 23 08:56 (10.99.99.99)


14. And, to begin the wrap-up, Solaris' who makes up for the fact that it doesn't have a built in handler to deal with being called as "whoami" by providing two different options to get that same information. In an alarming show of disregard for proper capitalization, both of these versions work ;) Note that this output is almost always exactly the same as the output from "who -m":

host # who am i
user001 pts/5 Dec 23 08:56 (10.99.99.99)
host # who am I
user001 pts/5 Dec 23 08:56 (10.99.99.99)


follow that up with the question that, statistically, follows "who am I?" most often, give it a little bit of "Talking Heads" flavour, and you've got yourself a command that's completely useless ;)

host # my God, what have I done?
-bash: my: command not found


15. Back off the Road To Nowhere (David Byrne, again. Make him stop!! ;)... Lastly (no pun intended, as you'll understand by the end of this paragraph), you can use who, using any combination of options (with the exception of -n, which only works with -q), and follow it all up with a different utmpx file (if, for instance, your old one got to big and you copied it off somewhere). Straight up who on Solaris makes use of /var/adm/utmpx, but you can tell who to use any utmpx-like file (including wtmpx, which can make the "who" command emulate the "last" command to a basic degree):

host # who /var/adm/wtmpx
root console Nov 6 13:39
root console Nov 6 13:48 (:0)
root pts/3 Nov 6 13:51 (:0.0)
root pts/3 Nov 6 14:38 (:0.0)
root pts/4 Nov 6 14:39 (:0.0)
user001 sshd Nov 7 09:48 (host.subnet.domain.com)
user001 pts/4 Nov 7 09:48 (host.subnet.domain.com)
user001 sshd Nov 7 09:54 (host1.subnet2.domain3.com)
user001 pts/5 Nov 7 09:54 (host1.subnet2.domain3.com)
...


Here's hoping today's post help shed a bit more light on Solaris' who options than the standard usage screen does, pointed out a number of reasons it can be a great tool to have in your troubleshooting arsenal and (perhaps) taught you a trick or two :)

Cheers,

, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Tuesday, December 23, 2008

We'll Be Moving Soon: Unix and Linux Lame Encryption Decoded

Hey there,

There are still several hours left to go on the poll (which may be closed at the time of publishing) as I write this post, but (unless something insane happens) it looks as though our desire to remove ourselves from blogspot agrees with the opinion of about 77 percent of the folks who took the time to vote. I won't go through all the reasons things will be better once we get our own URL, although I will provide a handy link back to the original post where we laid out our reasons for moving on to our own host. Once things get ironed out (which should be well before we actually "do" move), the switch will be made. Hopefully, we'll be able to either maintain both sites at once or Google will be good enough to allow us to keep this domain and redirect from it for a while (for free or for a fee; however they do it).

NOTE: If you're one of the potentially thousands of people who stop by here every once in a while (and you haven't written in yet), we'll be putting up a limited subscription email form (you can always email us via the "Send Me A Comment" link at the top right of every page) as soon as possible. To be 100 percent clear, the "limited" part means that it won't be up forever AND that the subscription (as is so often referenced on many white/black/grey-hat marketing sites on the net) is "limited" in that you'll get one email (announcing our new address, and any other information pertinent to the move), after which you will be automatically unsubscribed. This is a one-shot email deal and your email address will not be sold or traded in any way. You always have the option of just following the site, as we should be posting information about the move, as we get it, right here. We probably have something to gain by collecting a bunch of email addresses, but if we ever want to sell you something, we'll let you know that we're trying to sell you something. Wanna buy a bridge? ;)

More on that, as it comes. It appears as though we have some work to do over the holidays. In the end, this move should result in a better experience for both the reader and everyone here, since we'll be able to avail ourselves of conveniences not possible under our current setup. We thank Google for helping us get our start for zero dollars per month, but now it's time for us to move on.

And, to wrap up, since nobody replied regarding our previous confusion and lame encryption treasure-hunt post, we got lucky and don't have to pony-up the prize right away ;) Although, rest assured, on our new site, there will be revision upon revision of our cable TV script, in real-time, as we're able to update it in our CVS repository. We have a personal stake in seeing this through because, after having it work so well for a month, many of us have become entirely dependant on it. Zap2it may have changed their format (and will probably do so again), but we can modify how we extract the information. Also, TVGuide would be a good alternate source of information for getting instant television listings.

BTW, the original script was our Google search index rank script, compressed and goofed around with (although not made unworkable) via methods posted in our series on security through obfuscation and, finally, made even more confusing using our script to do lame encryption using od. Check it out. Why would we lie ;)

In closing, we're looking forward to the move and will put up a mail form as soon as possible. If you prefer, again, just send us email via the "Send Me A Comment" link at the top right of every post if you wish to be notified when the terms of the move are finalized, or you just have a gripe ;)

Cheers :)

, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Monday, December 22, 2008

Finding Your MSN/Live Index Rank From The Unix Or Linux CLI

Hey There,

Some of you may recall, a while back, when we did a post on how to find your Google search index rank from the CLI. In that post we took a look at another aspect of site SEO monitoring, where (rather than looking at Google's proprietary "PR" ranking system) we just went out and tried to fake being a human being entering keywords into the search page and then figuring out what result our desired URL actually showed up at (first - we stopped after the first positive result returned, since Google - like most places - will cut you off for, at least, a little while if they suspect you are a non-human ;).

Today's script is a variant on that, although we're going to be trudging through the landscape of MSN/Live's search engine.

IMPORTANT NOTE: Although this warning is on the original Google search rank index page, it bears repeating here and now. If you use wget (as we are in this script), or any CLI web-browsing/webpage-grabbing software, and want to fake the User-Agent, please be careful. Please check this online article regarding the likelihood that you may be sued if you masquerade as Mozilla.

This MSN script, of course, is only slightly different than the original Google script, although the differences are significant enough that I rewrote entire portions so I wouldn't have to back-translate code and then convert it and hope it worked ;) The script, itself, operates the same way our Google search index page rank script works, insofar as executing it from the command line goes. There are, at least, three different ways you can call it. The most basic being:

host # ./mrank www.yourdomain.com all these key words

It doesn't matter if they're enclosed in double quotes or not. If you "really" want to get the double quote experience, you just need to backslash your double quotes.

host # ./mrank www.yourdomain.com \"all these key words\"

Other ways include creating files with the URL and keyword information (same format as the command line) and feeding them to the script's STDIN:

host # cat FILE|./mrank
host # ./mrank <FILE



There is a little bugger that I need to remove from this script that incorrectly returns position #1 as #0 sometimes, but that should be easy to fix. I'd do it if I had the time. In fact, when I do, I'll repost the script and just include a notice in whatever that day's post is.

Now for the pictures :) Following are a few shots of MSN index rank checking as compared to Google index rank checking. Although most people will tell you it should be the opposite, it seems that we have a much better presence on Google than on MSN, after being in business (read: shamelessly self-promoting by answering question in forums, writing articles and submitting a billion sitemaps) for a year or so.

Following those, an EVEN BETTER set of pictures that made me scratch my head and mutter "WTF?" You'll see why. Let's just say that this site ranks lower on MSN for a certain keyword phrase than other sites that point to the post AND the first time our site comes up in that "double quoted" exact-string search, it's for an entirely unrelated post (???) Who has the time to wonder? ;) After that, way down at the bottom, we've somehow managed to remember to tack on the script.

By clicking any of the pictures below you are consenting to the use of our custom shrink ray, which will make each picture appear larger than normal, until you come back to the site ;)

mapquest cli search

adobe cli search

zombie search

And here's the MSN "managing swatch output" double-quote search that I still don't get.

swatch cli search

swatch search not our urls

And here we are, down a bit farther than any references to our post, and listing a page that doesn't contain the post, but probably has it listed in the blog archive on the sidebar ;)

swatch search our url

And, at long last, here's the script. Enjoy and have fun re-tooling it. Just be sure to double-check MSN to make sure you're not trying to fix a problem with your index ranking that exists by design ;)

Cheers,


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/bash

#
# mrank - Get your MSN Live Search Ranking Index
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

if [ $# -lt 2 -a $# -ne 0 ]
then
echo "Usage: $0 URL Search_Term(s)"
echo "URL with or with http(s)://, ftp://, etc"
echo "Double Quote Search If More Than 1 Term"
exit 1
fi

if [ $# -eq 0 ]
then
while read x y
do
url=$x
search_terms=$y
$0 $x "$y"
done
exit 0
else
url=$1
shift
search_terms=$@
fi

base=0
num=0
start=1
multiple_search=0
not_found=0

for x in $search_terms
do
if [ $multiple_search -eq 0 ]
then
search_string=$x
multiple_search=1
else
search_string="${search_string}+$x"
fi
done

echo "Searching For MSN Index For $url With Search Terms: $search_terms..."
echo

num_results=`wget -q --user-agent=Firefox -O - http://search.msn.com/results.aspx?q=${search_string}\&first=${start}|awk '{ if ( $0 ~ /of [0-9,]* results/ ) print $0 }'|sed 's/^.*of \([0-9,]*\) results.*$/\1/'`

while :;
do
if [ $not_found -eq 1 ]
then
break
fi
wget -q --user-agent=Firefox -O - http://search.msn.com/results.aspx?q=${search_string}\&first=${start} 2>&1|sed 's/<a href=\"\([^\"]*\)\"[^>]*>/\n\1\n/g'|sed -e :a -e 's/<[^>]*>/ /g;/</N;//ba'|sed '1,/See all.../d'|grep http|egrep -v 'cc.msnscache.com' 2>&1|sed '/search.live.com/d' |sed '/search.msn.com/d'|sed '/ocid=/,$d'|sed '/Developers | Help | Feedback/,$d'|awk -v num=$num -v base=$base '{ if ( $1 ~ /^http/ ) print base,num++,$NF }'|awk '{ if ( $2 < 10 ) print "MSN Index Number " $1 $2 " For Page: " $3; else if ( $2 == 10 ) print "MSN Index Number " $1+1 "0 For Page: " $3;else print "MSN Index Number " $1 $2 " For Page: " $3 }'|grep -i $url
if [ $? -ne 0 ]
then
let start=$start+10
let nexthopper=$start-1
if [ $nexthopper -ge 100 ]
then
not_found=1
if [ $not_found -eq 1 ]
then
break
fi
fi
let base=$base+1
num=0
else
break
fi

let sleep_time=${RANDOM}/600
echo "Not In Top $nexthopper Results: Sleeping $sleep_time seconds..."
sleep $sleep_time
done

if [ $not_found -eq 1 ]
then
echo "Not Found In First 100 Index Results!"
echo
fi

echo "Out Of Approximately $num_results Results"
echo
exit 0


, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Sunday, December 21, 2008

Deep Thoughts: What Does This Humor Have To Do With Linux Or Unix?

A glorious Sunday too you :)

Well, it's the end of yet another week, which means I'm slacking off and showcasing funny stuff from other places :) Today, even though I can't make the connection between this piece, Linux and/or Unix, I bring you the (long, but, hopefully, not forgotten) musings of Jack Handey. I first saw this stuff on Saturday Night Live, but, since the folks at PacketStorm Security have this in their humour archives, there's some connection between these hilarious one liners and the world of computing ;)

The original text is available at PacketStorm's Humor Archive.

Read, laugh and enjoy :)



Deep Thoughts
-------------
By Jack Handey

1) Dad always thought laughter was the best medicine, which I guess is why several of us died of tuberculosis.

2) Maybe in order to understand mankind, we have to look at the word itself: "Mankind". Basically, it's made up of two separate words - "mank" and "ind". What do these words mean ? It's a mystery, and that's why so is mankind.

3) I hope if dogs ever take over the world, and they chose a king, they don't just go by size, because I bet there are some Chihuahuas with some good ideas.

4) It takes a big man to cry, but it takes a bigger man to laugh at that man.

5) I guess we were all guilty, in a way. We all shot him, we all skinned him, and we all got a complimentary bumper sticker that said, "I helped skin Bob."

6) I bet the main reason the police keep people away from a plane crash is they don't want anybody walking in and lying down in the crash stuff, then, when somebody comes up, act like they just woke up and go, "What was THAT?!"

7) The face of a child can say it all, especially the mouth part of the face.

8) Ambition is like a frog sitting on a Venus Flytrap. The flytrap can bite and bite, but it won't bother the frog because it only has little tiny plant teeth. But some other stuff could happen and it could be like ambition.

9) I'd rather be rich than stupid.

10) If you were a poor Indian with no weapons, and a bunch of conquistadors came up to you and asked where the gold was, I don't think it would be a good idea to say, "I swallowed it. So sue me."

11) If you define cowardice as running away at the first sign of danger, screaming and tripping and begging for mercy, then yes, Mr. Brave man, I guess I'm a coward.

12) I bet one legend that keeps recurring throughout history, in every culture, is the story of Popeye.

13) When you go in for a job interview, I think a good thing to ask is if they ever press charges.

14) To me, boxing is like a ballet, except there's no music, no choreography, and the dancers hit each other.

15) What is it that makes a complete stranger dive into an icy river to save a solid gold baby? Maybe we'll never know.

16) We tend to scoff at the beliefs of the ancients. But we can't scoff at them personally, to their faces, and this is what annoys me.

17) Probably the earliest flyswatters were nothing more than some sort of striking surface attached to the end of a long stick.

18) I think someone should have had the decency to tell me the luncheon was free. To make someone run out with potato salad in his hand, pretending he's throwing up, is not what I call hospitality.

19) To me, clowns aren't funny. In fact, they're kind of scary. I've wondered where this started and I think it goes back to the time I went to the circus, and a clown killed my dad.

20) As I bit into the nectarine, it had a crisp juiciness about it that was very pleasurable - until I realized it wasn't a nectarine at all, but A HUMAN HEAD!!

21) Most people don't realize that large pieces of coral, which have been painted brown and attached to the skull by common wood screws, can make a child look like a deer.

22) If trees could scream, would we be so cavalier about cutting them down? We might, if they screamed all the time, for no good reason.

23) Better not take a dog on the space shuttle, because if he sticks his head out when you're coming home his face might burn up.

24) You know what would make a good story? Something about a clown who make people happy, but inside he's real sad. Also, he has severe diarrhea.

25) Sometimes when I feel like killing someone, I do a little trick to calm myself down. I'll go over to the persons house and ring the doorbell. When the person comes to the door, I'm gone, but you know what I've left on the porch? A jack-o-lantern with a knife stuck in the side of it's head with a note that says "You." After that I usually feel a lot better, and no harm done.

26) If you're a horse, and someone gets on you, and falls off, and then gets right back on you, I think you should buck him off right away.

27) If you ever teach a yodeling class, probably the hardest thing is to keep the students from just trying to yodel right off. You see, we build to that.

28) If you ever fall off the Sears Tower, just go real limp, because maybe you'll look like a dummy and people will try to catch you because, hey, free dummy.

29) Anytime I see something screech across a room and latch onto someones neck, and the guy screams and tries to get it off, I have to laugh, because what is that thing?

30) He was a cowboy, mister, and he loved the land. He loved it so much he made a woman out of dirt and married her. But when he kissed her, she disintegrated. Later, at the funeral, when the preacher said, "Dust to dust," some people laughed, and the cowboy shot them. At his hanging, he told the others, "I'll be waiting for you in heaven--with a gun."

31) The memories of my family outings are still a source of strength to me. I remember we'd all pile into the car - I forget what kind it was - and drive and drive. I'm not sure where we'd go, but I think there were some trees there. The smell of something was strong in the air as we played whatever sport we played. I remember a bigger, older guy we called "Dad." We'd eat some stuff, or not, and then I think we went home. I guess some things never leave you.

32) If a kid asks where rain comes from, I think a cute thing to tell him is "God is crying." And if he asks why God is crying, another cute thing to tell him is "Probably because of something you did."

33) Contrary to what most people say, the most dangerous animal in the world is not the lion or the tiger or even the elephant. It's a shark riding on an elephant's back, just trampling and eating everything they see.

34) As we were driving, we saw a sign that said "Watch for Rocks." Marta said it should read "Watch for Pretty Rocks." I told her she should write in her suggestion to the highway department, but she started saying it was a joke - just to get out of writing a simple letter! And I thought I was lazy!

35) One thing kids like is to be tricked. For instance, I was going to take my little nephew to DisneyLand, but instead I drove him to an old burned-out warehouse. "Oh, no," I said, "DisneyLand burned down." He cried and cried, but I think that deep down he thought it was a pretty good joke. I started to drive over to the real DisneyLand, but it was getting pretty late.

36) If you saw two guys named Hambone and Flippy, which one would you think liked dolphins the most? I'd say Flippy, wouldn't you? You'd be wrong, though. It's Hambone.

37) Laurie got offended that I used the word "puke." But to me, that's what her dinner tasted like.

38) We used to laugh at Grandpa when he'd head off and go fishing. But we wouldn't be laughing that evening when he'd come back with some whore he picked up in town.

39) I wish a robot would get elected president. That way, when he came to town, we could all take a shot at him and not feel too bad.

40) As the evening sky faded from a salmon color to a sort of flint gray, I thought back to the salmon I caught that morning, and how gray he was, and how I named him Flint.

41) If you're a young Mafia gangster out on your first date, I bet it's real embarrassing if someone tries to kill you.

42) Whenever I see an old lady slip and fall on a wet sidewalk, my first instinct is to laugh. But then I think, what if I was an ant, and she fell on me. Then it wouldn't seem quite so funny.

43) If you go parachuting, and your parachute doesn't open, and you friends are all watching you fall, I think a funny gag would be to pretend you were swimming.

44) When I was a kid my favorite relative was Uncle Caveman. After school we'd all go play in his cave, and every once in a while he would eat one of us. It wasn't until later that I found out that Uncle Caveman was a bear.

45) I think people tend to forget that trees are living creatures. They're sort of like dogs. Huge, quiet, motionless dogs, with bark instead of fur.

46) Happy Fun Ball
- -only $14.95-

Warning: Pregnant women, the elderly and children under 10 should avoid prolonged exposure to Happy Fun Ball.

Caution: Happy Fun Ball may suddenly accelerate to dangerous speeds.

Happy Fun Ball Contains a liquid core, which, if exposed due to rupture, should not be touched, inhaled, or looked at.

Do not use Happy Fun Ball on concrete.

47) Happy Fun Ball
- -only $14.95-

Discontinue use of Happy Fun Ball if any of the following occurs:
*Itching
*Vertigo
*Dizziness
*Tingling in extremities
*Loss of balance or coordination
*Slurred speech
*Temporary Blindness
*Profuse sweating
*Heart Palpitations

48) Happy Fun Ball
- -only $14.95-

If Happy Fun Ball begins to smoke, get away immediately. Seek shelter and cover head.

Happy Fun Ball may stick to certain types of skin.

When not in use, Happy Fun Ball should be returned to its special container and kept under refrigeration...

Failure to do so relieves the makers of Happy Fun Ball, Wacky Products Incorporated, and its parent company Global Chemical Unlimited, of any and all liability.

Ingredients of Happy Fun Ball include an unknown glowing substance which fell to Earth, presumably from outer space.

49) I'd like to see a nude opera, because when they hit those high notes, I bet you can really see it in those genitals.

50) It's really sad when a family can be torn apart by something as simple as a pack of wild dogs.

51) If they ever come up with a swashbuckling School, I think one of the courses should be Laughing, then Jumping Off Something.

52) When you're riding in a time machine way far into the future, don't stick your elbow out the window, or it'll turn into a fossil.

53) One thing kids like is to be tricked. For instance, I was going to take my little nephew to Disneyland, but instead I drove him to an old burned-out warehouse. "Oh, no," I said. "Disneyland burned down." He cried and cried, but I think that deep down, he thought it was a pretty good joke. I started to drive over to the real Disneyland, but it was getting pretty late.

54) A good way to threaten somebody is to light a stick of dynamite. Then you call the guy and hold the burning fuse up to the phone. "Hear that?" you say. "That's dynamite, baby."

55) Why do people in ship mutinies always ask for "better treatment"? I'd ask for a pinball machine, because with all that rocking back and forth you'd probably be able to get a lot of free games.

56) I'd like to be buried Indian-style, where they put you up on a high rack, above the ground. That way, you could get hit by meteorites and not even feel it.

57) If I lived back in the wild west days, instead of carrying a six-gun in my holster, I'd carry a soldering iron. That way, if some smart-aleck cowboy said something like "Hey, look. He's carrying a soldering iron!" and started laughing, and everybody else started laughing, I could just say, "That's right, it's a soldering iron. The soldering iron of justice." Then everybody would get real quiet and ashamed, because they had made fun of the soldering iron of justice, and I could probably hit them up for a free drink.

58) I bet when the neanderthal kids would make a snowman, someone would always end up saying, "Don't forget the thick, heavy brows." Then they would all get embarrassed because they remembered they had the big hunky brows too, and they'd get mad and eat the snowman.

59) Fear can sometimes be a useful emotion. For instance, let's say you're an astronaut on the moon and you fear that your partner has been turned into Dracula. The next time he goes out for the moon pieces, wham!, you just slam the door behind him and blast off. He might call you on the radio and say he's not Dracula, but you just say, "Think again, bat man."

60) Too bad you can't buy a voodoo globe so that you could make the earth spin real fast and freak everybody out.

61) The people in the village were real poor, so none of the children had any toys. But this one little boy had gotten an old enema bag and filled it with rocks, and he would go around and whap the other children across the face with it. Man, I think my heart almost broke. Later the boy came up and offered to give me the toy. This was too much! I reached out my hand, but then he ran away. I chased him down and took the enema bag. He cried a little, but that's the way of these people.

62) I wish I had a Kryptonite cross, because then you could keep both Dracula AND Superman away.

63) I don't think I'm alone when I say I'd like to see more and more planets fall under the ruthless domination of our solar system.

64) Living on Earth may be expensive, but it includes an annual free trip around the Sun.

65) Marta likes to talk about sensuality, but I don't think she would know sensuality if it bit her on the ass.

66) If you drop your keys into molten lava just let 'em go 'cause, man, they're gone.

67) Contrary to popular belief, the most dangerous animal is not the lion or tiger or even the elephant. The most dangerous animal is a shark riding on an elephant, just trampling and eating everything they see.

68) Once when I was in Hawaii, on the island of Kauai, I met a mysterious old stranger. He said he was about to die and wanted to tell someone about the treasure. I said, "Okay, as long as it's not a long story. Some of us have a plane to catch, you know." He stared telling his story, about the treasure and his life and all, and I thought: "This story isn't too long." But then, he kept going, and I started thinking, "Uh-oh, this story is getting long." But then the story was over, and I said to myself: "You know, that story wasn't too long after all." I forget what the story was about, but there was a good movie on the plane. It was a little long, though.

69) Is there anything more beautiful than a beautiful, beautiful flamingo, flying across in front of a beautiful sunset? And he's carrying a very beautiful rose in his beak, and also he's carrying a very beautiful painting with his feet. And also, you're drunk.

70) In weightlifting, I don't think sudden, uncontrolled urination should automatically disqualify you.



, Mike




Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.