New Year,s Eve News: Unix And Linux Users Beware

Hey there,

As part of a non-existent tradition, composed of this year's "New Year's Eve post" (being written as I type ;) and last year's zany and over-the-top network port querying script post, we're going to make this year's EOY post short, to the point, hopefully fun and somewhat worthwhile, so you can leave work early without getting it ok'ed and begin making bad decisions as soon as possible ;)

In all seriousness, if you go out tonight, have one (or maybe three or four ;) on me, but try to stay safe. According to a recent study by law enforcement and the National Association of Situations You'll Regret Later (NASYRL), lots of people get into, sometimes fatal, accidents and/or end up in jail for driving under the influence during the holidays. Everyone here wishes you a safe, happy and fun New Year's Eve and highly encourage you to partake of your legal drug of choice in quantities that will make you question your prior state of mind tomorrow (we can't officially condone the other stuff, but would recommend - if we were in the business of minding yours ;)- that you do whatever you want to do in moderation. Although certain substances have a higher lethality-rate than others, excess is the enemy. Basic food and water will demonstrate this very well any day of the year ;).

And, if possible, if you must go home at the end of the night, please find someone sober to drive you. If you're that sober person, please do what you can to round up a few rummies that need a lift ;)

And here's a little cautionary tale from 3 years ago, ripped from today's headlines! (wait... no... that's this week's episode of Law & Order ;) This piece is actually from LinuxReviews.org regarding the dangers of using a text web browser, especially during times of crisis and/or excess. Hopefully, it will scare the cr@P out of you and you'll begin doing what you're told. If not, and you either enjoy it, hate it or feel absolutely nothing for it, yet still refuse to walk the line, we salute you!

Have fun this New Year's Eve! Best wishes for your safety and maximum enjoyment from all of us :)

Cheers,

NOTE: We left in all the spelling mistakes from the original articles to maintain the integrity of the sources and, also, to simulate drunken journalism in keeping with the night's theme ;)

---

Solaris-user arrested by British Police for using Lynx

Lynx is a popular text-based browser user by Unix and Linux users. British Telecom, who are the web site administrator for a Tsunami aid site mistook a Lynx web browser user who donated money for a hacker because of the strange user agent the browser leaves in the logs and reported the user to the police.

The police, apparently equally incompetent, raided the private citizen who made the donation swat-style while he was having a quiet lunch.

BBC reported this as Officers from the Metropolitan Police's Computer Crime Unit have begun an inquiry after BT blocked the attempt on New Year's Eve. A 28-year-old man from east London was arrested and released on bail in connection with alleged offences. and this is a good example of how state media reports violent crimes against private citizens when done by the same stats police force.

The persons only crime was to use the non-standard Internet browser Lynx.

From a mailing list:

---

For donating to a Tsunami appeal using Lynx on Solaris 10. BT (British Telecom) who run the donation management system misread an access log and saw hmm thats a non standard browser not identifying it's type and it's doing strange things. Trace that IP. Arrest that hacker.

Armed police, a van, a police cell and national news later the police have gone in SWAT styley and arrested someone having their lunch.

Out on bail till next week and preparing to make a lot of very bad PR for BT and the Police....

So just goes to show if you use anything other than Firefox or IE and you rely on someone else to interogate access logs or IDS logs you too could be sitting in a paper suit in a cell :(

---

What is Lynx?

Lynx, Links, Dillo and other rare browsers are used by thousands browsers are used by thousands of Linux and Unix users world wide, but they only make up a very small percentage of visits to mainstream entertainment sites. This may because many such sites frequently use javascript, flash and other garbage non-standard elements that make these sites impossible to render in standard-compliant browsers.

Average Entertainment Website, Browser market share Desember 2004

202932 visits	Browser
53.54%	Internet Explorer 6.0 (Windows)
27.10%	Gecko 1.7.x (Mozilla/Firefox/Galeon/Epiphany/etc)
4.21%	Internet Explorer 5.0
4.14%	Safari (OSX)
2.54%	Opera

The picture is very different for Linux related sites.

Linux Reviews, Browser market share Desember 2004

56360 visits	Browser
36.63%	Gecko 1.7.x (Mozilla/Firefox/Galeon/Epiphany/etc)
17.58%	Internet Explorer 6.0 (Windows)
10.43%	Internet Explorer 5.0
3.37%	Opera
2.83%	Konqueror

Lynx was not in the entertainment site numbers and had 0.53% share at Linux Reviews.

SourceForge, Slashdot, Freshmeat, Lxer and other sites like these have regular visitors who use rare browsers, but if you are using Lynx then you may want to stay clear of official UK government web sites and perhaps also other government web sites in general.

• Tsunami fund hacking is probed (bbc)
  

---

, Mike

---

Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

---

The Thesaurus Shell Script - New And Improved!

Hey There,

Today's post is a follow up on a post we did just last week on creating an online Thesaurus using shell scripting. In it, we took an already existing Thesaurus script called thes.sh from gentoo.org and spiffed it up a little so that you could use multiple word queries and extract the suggestions the online Thesaurus would give you if it couldn't find a match for your word or phrase. Of course, sometimes it couldn't find a match for you and, also, had no suggestions... You can't win 'em all ;)

Thanks for this post's updated, and more platform-independent, script goes out to Fred Stephens (who's starting his own Linux revolution over at Linux Latitude) and (like so many other folks who, sometimes, prefer to remain nameless) his help in modifying this script so that it would also run correctly on a few more Linux and Unix distro's I didn't have access to, is greatly appreciated!

In an effort to make the script more portable as well, we used wget instead of lynx and took away the dependency on the html2text program by replacing that versatile (but not always available) software with a series of simple sed executions. This, of course, didn't work exactly the same (or "the same enough" ) on as many distro's as possible. Of course, I knew, when I wrote it, that I couldn't possibly test it on every available platform, so the possibility of it behaving differently on someone else's OS was always there (I think it always is and always will be). But, especially as time lumbers on, if we let hurdles like that slow us down, we'll all eventually accomplish less and less in direct opposite proportion to the amount of variety the computing industry offers us (which is growing more and more abundant as I type). Eventually, no one would ever write any sort of program, spark an original thought or attempt to improve and/or modify solutions unless they had a blank check from the government (or a giant conglomerate) to fund their every notion ;)

But that's one of the great things about the internet. Although I'll agree that (to a certain extent) it promotes seclusion and separation of physical entities from the "uncomfortable" prospect of having to physically interact with one another, the flipside of that coin is that the internet provides the world's largest forum for the free interchange and exchange of information and ideas and makes it possible for like-minded individuals to pool their efforts and ingenuity to produce more, and better, solutions to problems at a much snappier pace.

This rewrite addresses an issue with the one part of the script I just "knew" in my gut wouldn't port to some distro somewhere: the sed execution line. As I mentioned above, the free (licensed under the GPL) software html2text is a project that's had a lot of development hours put into it (actual releases and versions, etc ;), so it's obviously much much better at parsing out html and converting it to plain text than any series of sed commands I could ever string together on a given afternoon. Again, we chose to use sed in order to try and make this script accessible to users who couldn't get their hands on html2text (which also requires python), since sed comes standard on every Linux and Unix distro I know of (and has been around for a long time).

In any event, the script's been spruced up a bit and should run cleaner (The change isn't drastic, but it's definitely significant. If you noticed any garbage characters getting returned to you when you ran the original, this rewrite should, hopefully, fix that for you :)

Thanks, again, for your contribution, Fred :)

BTW, if you want to see sample output, there's a picture on the parent post about this Thesaurus shell script. I'm not jamming it in here again to try and keep this blog light (although, I see I've typed another novel already ;)

Cheers,





This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/bash
# Thesaurus - Find something original to say :)
# 2008 - Mike Golvach - eggi@comcast.net (modified slightly by Fred J. Stephens - fjs205@gmail.com)
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#
if [ $# -lt 1 ]
then
 echo "Usage: $0 YourThesaurusTerm"
 echo "May be two or more words separated by spaces"
 echo  "but only one definition per execution."
 echo "Ex: $0 leader"
 echo "Ex: $0 big name"
 exit 1
fi
args="$@"
wget=/usr/bin/wget
if [ $# -gt 1 ]
then
 args=`echo $args|sed 's/ /%20/g'`
fi
echo
$wget -nv -O - http://thesaurus.reference.com/search?q="$args" 2>&1|\
egrep -i 'Synonyms:|Definition:|No results found|Thesaurus suggestions:'|\
sed -e 's/<br \/>/\n/g' -e 's/<[^>]*>//g' -e 's/ / /g' -e 's/\(.\)$/\1\n/' -e 's/Would you.*$//' -e 's/\ / /g'

exit 0

, Mike

Shell Script To Emulate A Thesaurus For Linux or Unix

Hey There,

Today's Linux/Unix bash shell script is for those of us who sometimes get lost for words. This happens to me at least a few times a day as I seem to talk, and type, way too much. Every now and again, I'll find myself facing a sentence that is not only redundant, but also seems to repeat its central message more than once ;) Sometimes redundancy is a good thing, though. If you've ever listened to an instructional or motivational speaker, you've probably noticed that a lot of them like to hit on the "rule of 3's" (sometimes 4's and 5's, with the annoyance factor increasing commensurate to the occurrence of repetition :) I, personally, try not to repeat myself ever (although, in writing about a thesaurus, I'm almost certainly doomed to some sort of meta-paradox).

One of the times a good thesaurus can come in handy is when you're faced with having to use similar words within a restricted amount of space and the resulting text seems stilted because of it. For instance, the sentence:

As good an idea as it may seem, it's generally not good to repeat the same word within a sentence.

With a little thought (or a handy reference) can be made much more palatable, and the redundancy can be made to appear to have disappeared:

As good an idea as it may seem, it's generally not desirable to repeat the same word within a sentence.

Our script today is based on an original script (call "thes") that can be found at Gentoo.org and, like that script, makes use of the Online Thesaurus at reference.com.

The major differences between our script today, and the equally helpful one posted on Gentoo.org are mainly rooted in the method. For instance, their script makes use of lynx and a program you may not have installed by default, called html2text. Ours, while still relying on the online component, uses wget and sed. We went with wget over lynx since it's partial-source dump option is a little more predictable than lynx's. That's not to say that there's anything wrong with lynx, just that it didn't suit our needs for this particular endeavour.

Another major difference between the two is that we decided to go ahead and throw in the "%20" space declaration so that you could submit multi-word queries to the script and get a response that you'd expect. Check out the picture below for a quick example of submitting a bad multi-word query, a good multi-word query, a bad single-word query and a good single-word query. If you can't see the picture, for whatever reason, the output is fairly simple. When you submit a bad query of any type (single or multi-word), you'll get back a "No results found" message and some suggestions. When you submit a query that matches something, you'll receive a varying number of definitions followed by a varying number of synonyms (and, yes, I'm not using the script while I write this ;)

Click the picture below to see it in full size:



As of the writing of this post, I have yet to figure out the "&whatever" suffix to the URL that will make the online Thesaurus return more than 10 results per page, so there's still some work to be done there. If you're so inclined, you can write in a quick check and recheck into the script. The addition to the URL that will start you at definition number 11 (instead of 1; the default) would be "&start=11" - So far, except for with very general words like "good," I've found that this hasn't been necessary, but it would be a cool improvement.

Here's to finding new ways to express yourself (politely ;)

Cheers,





This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/bash

#
# Thesaurus - Find something original to say :)
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

if [ $# -lt 1 ]
then
 echo "Usage: $0 YourThesaurusTerm"
 echo "May be two or more words separated"
 echo "by spaces, but only one definition"
 echo "per execution."
 echo "Ex: $0 goodness"
 echo "Ex: $0 goodness gracious"
 exit 1
fi

args="$@"
wget=/usr/bin/wget

if [ $# -gt 1 ]
then
 args=`echo $args|sed 's/ /%20/g'`
fi

echo
$wget -nv -O - http://thesaurus.reference.com/search?q="$args" 2>&1|egrep -i 'Synonyms:|Definition:|No results found|Thesaurus suggestions:'|sed -e 's/<br \/>/\n/g' -e 's/<[^>]*>//g' -e 's/ / /g' -e 's/\(.\)$/\1\n/' -e 's/Would you.*$//'

exit 0

, Mike

Checking the State of Your DNS Setup Externally!

Every once in a while, it's good practice to take a look at any system you have setup (whether it be a single system, business process, program or computing environment) and make sure it's doing okay. Some sort of regular auditing also helps to find out if there are any areas in which you can make an improvement or fix errors.

One of the areas that seems to require looking over more consistently, is your DNS setup. New versions are released regularly, bugs are found just as regulary, and acceptable syntax can sometime change between releases and/or RFC's.

Note for today's script - It is intended only for your benefit, and I strongly urge you to use any free DNS reporting service you can find to accomplish what we're accomplishing here. I neither work for, nor do I do any afilliate marketing for, www.dnsstuff.com. We only use it where I work, because it's the standard. Probably, most readers already know about it. The site tools require a free registration, if you're not a paying member, but also require a payment for use after 30 DNS Reports. I have placed a very obvious "COMMENT" in the script above the only line you'd need to change to use another service. If you "do" find something equal, or better, I would recommend that you use it (and, maybe, send me an email to let me know the URL so I can check it out and update my scripts, too :) That being said, I'm not trashing them either. If you have to do this sort of thing a lot(for your employer, lets say), the company can probably shell out a few bucks a month for the service. It's worth the price if you need to use it regularly.

Aside from your own internal auditing, it's good to get a fair and objective third-party assessment of the state of your DNS setup. A great site to get this accomplished on the web is located at http://www.dnsstuff.com. They have a tool called "DNS Report" (which used to be its own domain - www.dnsreport.com) which can be used very effectively, even if you choose not to be a paying member of the site. A long long time ago, it was available to everyone for free, but since it's pay-for now, you really have to be careful not to deluge it with requests to check your DNS zones (all 30 of them, if you're doing this for free), or you'll get dumped on their blacklist and barred from using the service at all.

The little Perl script I've written below will help you to automate the usage of that web service for all of your DNS zones. The reports are nice and easy to understand, even for the highest of higher-ups (green = good, red = bad ;) and the service does a very good job of pointing out weaknesses, or areas that don't conform to current RFC expectations, in your DNS zones. This script does require that you have Perl installed (although it's just a script I whipped up so it's submitted under GPL (Gnu Public License), at best, and you can feel free to rewrite it to suite your own needs). It also makes use of curl. You can substitute lynx or wget or any other program that can download and save web pages to your Unix/Linux hard drive. And here it is, below (Please read the comments regarding use of dnsstuff's dnsreport tool):





This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/usr/bin/perl

#
# 2007 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#
# Registration is now required to use this tool - It used to be free.
# When/If you register, be sure to uncheck the two opt-in mail
# checkboxes at the bottom unless you want to receive those emails.
#
# Login to dnsstuff.com before running this script or it will just return
# a page indicating that you need to log in before submitting a full request
# without that request being linked to from one of their pages
#
# Simple Error Checking here, just want to be sure that a file exists at all.
# Disregard everything else on the command line. This script will fail if the
# file named on the command line doesn't include one URL per line, anyway :)

if ( ! -f $ARGV[0] ) {
print "Usage: $0 fileWithUrls\n";
exit(1);
}
$dns_file = $ARGV[0];

open(URL_FILE, "< $dns_file");
@url_file = ;
close(URL_FILE);
$counter = 120;

foreach $url (@url_file) {
print "$url";
$| = 1;
if ($pid = fork) {
if ( $counter = 900 ) {
$counter = 120;
} else {
$counter = $counter+60;
}
print "next\n";
} else {
close (STDOUT);
chomp($url);
#
# COMMENT <--- The Obvious One :)
# CHANGE THE URL ON THIS LINE TO THE URL OF ANY OTHER SITE YOU FIND THAT
# PROVIDES AN EQUAL SERVICE!
#
@output = `curl http://www.dnsstuff.com/tools/dnsreport.ch?domain=$url`;
open(OUTPUT, ">$url.htm");
print OUTPUT @output;
close(OUTPUT);
exit(0);
}
if ( $counter = 900 ) {
$counter = 120;
} else {
$counter = $counter + 60;
}
sleep $counter;
}
exit(0);

And that's that! As I mentioned before, this tool will blacklist you if you hit dnsstuff too hard. My script assumes that you can hit the site at the rate I used last time I used it. Since it's a pay-for service now, I would recommend changing the wait times to at least twice as much. To give you an idea, I ran this when the service was free and ended up getting blacklisted in a under a few hours. Granted, there were certain other factors that contributed to my getting the boot; most prominently that I was checking the DNS for about 300 to 400 zones we were hosting. It could have been left at checking one and making sure all the others were the same, but, as noted above, some bosses like to see reports with lots of colors and lots of pages.

Also, just so you don't feel like I'm leaving you in a lurch, if you do happen to get blacklisted, just go to their member forums (you get access to post to these since you had to do the free registration - you can only browse them if you're not registered) at http://member.dnsstuff.com/forums/ and do a search on "banned," "blacklist" or "black list." Most folks just have to start a thread and request their access back in order to get off the blacklist.

So, be careful out there, and take it slow :)

, Mike

linux unix internet technology