Showing posts with label command. Show all posts
Showing posts with label command. Show all posts

Thursday, April 2, 2009

Command Line Veritas NetBackup Activity Monitor For Linux And Unix

Hey there,

Many of you who use FeedBurner (or whatever smaller, productive, company Google's bought out this week ;) to get blog posts by email have probably already gotten this post even before I finished writing it. I made the cardinal sin of hitting the enter key while I wrote this post online :P I apologize for your (quite understandable) previous confusion ;)

Today's post will be short and sweet, since it's getting late in the day and I'm still working and I don't think I get to sleep in tomorrow either... With that in mind, I tried to make this post heavier on content and have been working on it (on and off) all day. Again, I also apologize for the quality of the video attached to this post, which shows the output of the script. It's very very slow and does not accurately reflect the refresh rate of the PuTTY terminal I was running the program on ;) I had to download 3 or 4 different video processing "portable apps" so that I could try and put something together. I'm not "allowed" to install anything anywhere except for my desktop (and I'm not really sure if that's 100% kosher, which is why those programs are no longer installed) and I had to take what I could get.

Today's Bash script is my first attempt at rewriting one of those oldies-but-goodies that got away. A few years ago, I wrote a TUI Activity Monitor for Veritas NetBackup (In Perl, not like this one in Bash), since that was the only thing I ever used the GUI for. Somehow, over the years, I misplaced the CD (or maybe floppy ;) that I had it saved on and I've finally come to peace with the fact that I'll probably never ever find it again.

This script is much less glitzy than the old one, but it's a good starting point, and I hope a few people (aside from me ;) find it useful. It's a very simple program to run, since it only takes one optional variable (The refresh rate in seconds - defaulted at 10) and, as yet, does not have any special functionality built in. And, yes, you should read that as "The output formatting is God Awful!" :)

Check out the video below to watch a few backup jobs, and a catalog job, run in slooooooooooooooooooo-mo ;) Then grab the script and put it to good use. If you can make it better (I know I can, so there must be thousands of you out there who can, too :), feel free to improve upon it before I get around to that. I try to stay away from exporting Java GUI's over X-Windows as often as I possibly can (especially when working on systems overseas or across-country), and I've been meaning to rewrite this script for a long long time.

If this video makes you feel drowsy, download it and play it at double speed in the player of your choice. For some reason, it recorded at half speed and I ran out of time to try and figure out why ;)



And here's the script. Enjoy it and abuse it in good health :)


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/bash

#
# nbuvtam - v1 of the netbackup vt activity monitor
#
# 2009 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

trap 'exit 0' 2
perl=`which perl`

if [ ! -z $1 ]
then
refresh=$1
else
refresh=10
fi

function local_time()
{
regular_time=$(perl -e "print scalar(localtime($1))")
echo -n "$regular_time"
}


jobtype_array=(backup archive restore verify duplicate import catalogue_backup vault label erase tape_request clean format_tape physical_inventory qualification)
state_array=(queued active wait_for_retry done)
operation_array=(mount position connect write vault_initialize vault_duplication vault_duplication_complete vault_catalog_backup vault_eject report duplicate import verify restore catalog_backup vault label erase)
subtype_array=(immediate scheduled user_initiated quick_erase long_erase)
schedule_type_array=(full incr user_backup user_archive cumulative_incr)

tput clear
OLDIFS=$IFS
IFS=","
while :
do
tput clear
echo "======================================================================="
echo "THE TOTALLY LAME VERITAS NETBACKUP COMMAND LINE ACTIVITY MONITOR"
echo "V0.0000000001-alpha - Ctl-C to Quit"
echo "======================================================================="
echo "jobid type state state_details status policy schedule client media_server start_time elapsed_time end_time storage_unit attempt operation kilobytes files pathname percent_complete job_pid owner master"
echo "======================================================================="
echo "======================================================================="
while read jobid jobtype state status policy schedule client server started elapsed ended storage_unit attempt operation kilobytes number_of_files path_last_written percent jobpid owner subtype classtype scheduled_type priority group master_server retention_unit retention_period compression kb_last_written files_last_written file_list_count files the_rest
do
start_time=$(local_time $started)
end_time=$(local_time $ended)
elapsed=$(echo $elapsed|sed 's/^0*//g')
hours=$((elapsed / 3600))
elapsed=$((elapsed % 3600))
minutes=$((elapsed / 60))
elapsed=$((elapsed % 60))
if [[ $hours -lt 10 ]]
then
hours="0$hours"
fi
if [[ $minutes -lt 10 ]]
then
minutes="0$minutes"
fi
if [[ $elapsed -lt 10 ]]
then
elapsed="0$elapsed"
fi
echo "$jobid ${jobtype_array[${jobtype}]} ${state_array[${state}]} ${operation_array[${operation}]} $policy ${schedule_type_array[${schedule}]} $client $server $start_time $elapsed_time $end_time $storage_unit $attempt $operation $kilobytes $number_of_files $path_last_written ${percent}% $jobpid $owner $master_server"
echo "---------------------------------------------------------------"
done <<< "`/usr/openv/netbackup/bin/admincmd/bpdbjobs -all_columns|head -5`"
sleep $refresh
done
, Mike




Discover the Free Ebook that shows you how to make 100% commissions on ClickBank!



Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Thursday, August 28, 2008

Online Encyclopedia Shell Script For Linux And Unix

Hey There,

Today's Linux/Unix bash shell script is a follow up (and the finishing touch, I hope ;) to a whole slew of scripts we've written to mine the knowledge on tap at reference.com. If you missed any of the others, you can still find them in our older bash script posts to access the online Thesaurus and, of course, the online dictionary. This time we're breaking the chains and losing the albatross by finally attacking (with some level of precision) the Online Encyclopedia and producing a shell script does it as much justice as we could muster.

Note added after initial post: Forgot to include the Online Language Translation Script. It fits in with this bunch fairly well :)

One of the major differences you'll note between this script and the others is that we've included a "pager" variable. If you've ever checked out the reference.com Online Encyclopedia you're aware that the average return on a query is substantially larger than what you get back from a typical dictionary definition or thesaurus synonym query (not to mention the size of your terminal window). The screen shots we have for you, below, were the result of painstaking effort trying to find a word or phrase that didn't have an exceedingly long and thorough write-up. We chose "/usr/bin/more" as the pager for this script, but it can be easily changed to "/usr/bin/less," or whatever your favorite pager program happens to be.

After toying with this project for a while, trying to figure out why some pages worked, some didn't and where in the good Lord's name any consistent marker could be found to create consistent output, I've actually kind of gotten into using it. Now, if I'm at my terminal "working my fingers to the bone ;) " I can just type:

host # ./ency.sh engelbert humperdinck

and try to figure out what my generation "doesn't" see in him ;) ...apologies to die-hard fans.

Like our previous scripts, this script uses wget and sed. This script was also an attempt at "brute force scripting" (just writing it as we think it), but it ended up being so aggravating trying to find the right mix of regular expressions to get back decent results for the broadest amount of queries, that I think the method can, now, only be referred to as "blunt force scripting" (that's our new term for scripting as if you'd just been hit in the head with a brick ;). There are a few places the script could be tightened-up, but, to be perfectly honest with you, we're scared to death of changing anything about it right now. Maybe later, when we've steeled our nerves ;)

Below are a few screen shots of the script's output. The first is for an actual "one-pager" on a query for a pretty decent novel named Glamorama and the other is what you can expect to see when your query returns the equivalent of 15 leatherbound volumes of text on a query for Robert DeNiro. The last, just shows common error messages we generated on purpose.

Possibly interesting fact: If anyone out there is having an 80's moment and vaguely recalls Toni Basil ("oh, Mickey, you're so fine, you're so fine, you blow my mind..." why do I know the lyrics? ;), you'd be surprised to know that even "her" entry spanned multiple pages. I was actually surprised to find that she's also an actress and starred with Jack Nicholson in Five Easy Pieces under the very same name.

Click on the captures below to see them all in IMAX:

Glamorama Encyclopedia Entry

Robert De Niro Encyclopedia Entry

Encyclopedia Shell Script Errors

As one last bit of explanation. There are 2 specific error conditions that we ended up throwing in here. One is pretty obvious, involving No Results being found. The other had me going for a little while, since I was positive that a query on "linux" must have some return, since a query for "unix" did. It turns out that, when the Online Encyclopedia returns way too many results (I believe around 752 for "linux") it will state that it found a whole ton of results and is only going to show you one. The glitch in the program here is that, at some large number, it says that but then doesn't return an actual entry. I ended up noticing this when I finally broke down and parsed the HTML return and saw nothing, followed by a visit to the website via browser where I noticed the condition. You can see what I'm talking about right here, unless they've fixed it already (or it only happens intermittently). In any event, we wrote a check so that, if there are a large number of results, and it does return one, you'll still get it, but you'll get a self-explanatory error message if you hit this bug. Not that it needs to be said, but every query returns more than one result (insofar as our field-testing has gone), and the default behaviour of the site is to only return the top "one" and include links to the others.

Hope you enjoy this script, and can find some use for it. Any suggestions for improvement would be greatly appreciated. Be forewarned; there is the possibility that some of your queries may come back with extra parts added on that we didn't catch (usually tacked on to the end... phew), which could also make the output look horrible. This is the best we've got "for now" :)

I'm just glad somebody already figured out how to order Pizza online so I don't have that hanging over my head ;)

Cheers,


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/bash

#
# ency.sh - More than you ever wanted to know about anything
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

if [ $# -lt 1 ]
then
echo "Usage: $0 Your Encyclopedia Terms"
echo "May be two or more words separated by spaces"
echo "but only one definition per execution."
echo "Ex: $0 money"
echo "Ex: $0 the beatles"
exit 1
fi

args="$@"
wget=/usr/bin/wget
pager=/usr/bin/more

if [ $# -gt 1 ]
then
args=`echo $args|sed 's/ /%20/g'`
fi

echo

$wget -nv -O - http://www.reference.com/search?q="$args" 2>&1|grep -i "No results found" >/dev/null 2>&1

anygood=$?

$wget -nv -O - http://www.reference.com/search?q="$args" 2>&1|grep -i "<cite>" >/dev/null 2>&1

toomuch=$?

if [ $anygood -eq 0 ]
then
args=`echo $args|sed 's/%20/ /g'`
echo "No results found for $args"
exit 2
fi

if [ $toomuch -ne 0 ]
then
args=`echo $args|sed 's/%20/ /g'`
echo "Too many results returned for $args"
echo "Try doing a broader query - For Ex:"
echo "$0 linux = Too many results"
echo "$0 linux os = Information!"
echo "Double check at www.reference.com"
echo "to see the difference"
exit 3
fi

$wget -nv -O - http://www.reference.com/search?q="$args" 2>&1|sed '1,/Cite This Source/d'|sed '/External links\|See also\|More from Wikipedia/,$d'|sed -e :a -e 's/<[^>]*>/ /g;/</N;//ba' -e 's/$/\n/'|$pager

exit 0


, Mike




Douglas Taylor submitted these enhancements to make the output look much nicer. Thanks, Douglas!


1) Run the output through fmt before outputting. It will wrap the paragraph text fairly nicely, although it kind of destroys tabular data, and the indenting can get weird, or

2) Use lynx with the -dump option instead of wget. I replaced the final wget command with

lynx -dump http://www.reference.com/search?q="$arg" |sed '1,/Cite This Source/d'|sed '/External links\|See also\|More from Wikipedia/,$d'|sed 's/\[[0-9]*\]//g'|$pager

and the result looks fairly nice.


Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Sunday, July 13, 2008

More Unix And Linux Humor

Hey there,

For this "Lazy Sunday" post, I'm putting some more funny stuff I've found while crusing the web, regarding our two favourite flavours of OS. Of course, all the content has been sanitized, so I don't lose my "non-adult" status on this blog. If you want to check out the original sources, please stop by jeffcovey.net and jokes2go.com to have a look around.

Hope you find some of this stuff funny. Hilarious works too ;) Everyone needs a little time off, which is why I'm posting some goofy stuff over the weekend (Really, on the weekends readership goes way down, anyway, since I still haven't figured out how to get people interested in reading about work on their days off ;) Please note that the "command line" humor in the last section was done in a few different shells (a lot of it in csh, which won't reproduce in any other shell except tcsh, possibly)

Since there's so much space being used in this post, here's a little roadmap. Jokes 1 and 2 are song parodies, joke 3 is a clever paragraph and everything after that (All considered Joke 4 ;) are command line funnies.

Tomorrow, we'll get back to the "business" of Unix and Linux.

Cheers, and enjoy!

1. 50 ways to leave your editor

From: tbpic1@aurora.cc.monash.edu.au (Tim Pickett)
Subject: Re: Simon and Garfunkel (was Re: This is a test

umisef@yoyo.cc.monash.edu.au (Bernd Meyer) writes:
>So let's go for "there must be 50 ways to leave your editor" :-)

All right, you've got a deal. Memories of trying to quit an editor in
VMS that I didn't understand . . . (A bad song this for parodying. It
makes it sound like I can't make words scan, even though I'm referring
to the sheet music in front of me.)

The problem is all inside your ed(1), she sed(1) to me,
But the answer is emacs if you like C-x C-c,
I'm here to help you if you're struggling to use free(),
There must be fifty ways to leave your editor.

She said, `I hope your print job doesn't get dequeued,
I hope that you've stocked up lots of vending machine food.
But I'll repeat myself - have you gone and colon-q'd?
There must be fifty ways to leave your editor.
Fifty ways to leave your editor.'

Just press control-K, Ray,
Try double-Z, Fred,
You're still in insert, Bert,
So hit control-C.
Press meta-x, Lex,
Then type, `exit-quit-leave'
Or else just suspend, friend,
and kill(1) it with glee.

She said, `It grieves me now to stty(1) sane,
I wish there was somthing I could do to see your prompt again,'
I said, "I appreciate that,' and started to complain
About the fifty ways.

She said, `Why don't we both just sleep(1) on it tonight?
I'm sure in the morning you'll be sick of reading Byte.'
And then she warned me not to be a power-cycling neophyte,
There must be fifty ways to leave your editor.
Fifty ways to leave your editor.

(Repeat chorus)

2. Addicted to vi

Title : Addicted To Vi
Original : Addicted To Love
Group : Robert Palmer
Author : Chuck Musciano <chuck@trantor.harris-atd.com>
Intro : After thinking about that poor wretch who has become
addicted to vi, I was inspired to compose the following
ditty, sung to the tune of "Addicted To Love" by Robert
Palmer. As you sing this, it may help the effect to imagine
a dozen women, all of whom resemble Bill Joy, dressed in
black and dancing sinuously.
Song :


Addicted To Vi
(with apologies to Robert Palmer)

You press the keys with no effect,
Your mode is not correct.
The screen blurs, your fingers shake;
You forgot to press escape.
Can't insert, can't delete,
Cursor keys won't repeat.
You try to quit, but can't leave,
An extra "bang" is all you need.

You think it's neat to type an "a" or an "i"--
Oh yeah?
You won't look at emacs, no you'd just rather die
You know you're gonna have to face it;
You're addicted to vi!

You edit files one at a time;
That doesn't seem too out of line?
You don't think of keys to bind--
A meta key would blow your mind.
H, J, K, L? You're not annoyed?
Expressions must be a Joy!
Just press "f", or is it "t"?
Maybe "n", or just "g"?

Oh--You think it's neat to type an "a" or an "i"--
Oh yeah?
You won't look at emacs, no you'd just rather die
You know you're gonna have to face it;
You're addicted to vi!

Might as well face it,
You're addicted to vi!
You press the keys without effect,
Your life is now a wreck.
What a waste! Such a shame!
And all you have is vi to blame.

Oh--You think it's neat to type an "a" or an "i"--
Oh yeah?
You won't look at emacs, no you'd just rather die
You know you're gonna have to face it;
You're addicted to vi!

Might as well face it,
You're addicted to vi!

3. Santa Claus Learns Unix

better !pout !cry
better watchout
lpr why
santa claus <north pole >town
cat /etc/passwd >list
ncheck list
ncheck list
cat list | grep naughty >nogiftlist
cat list | grep nice >giftlist
santa claus <north pole > town

who | grep sleeping
who | grep awake
who | grep bad || good
for (goodness sake) {be good}

echo "Oh,"

better !pout !cry
better watchout
lpr why
santa claus <north pole >town

4. Command Line Jokes.

1. % make love
Make: Don't know how to make love. Stop.

2. % got a light?
No match.

3. % sleep with me
bad character

4. % man: Why did you get a divorce?
man:: Too many arguments.

5. % rm God
rm: God nonexistent

6. % make 'heads or tails of all this'
Make: Don't know how to make heads or tails of all this. Stop.

7. % make sense
Make: Don't know how to make sense. Stop.

8. % make mistake
Make: Don't know how to make mistake. Stop.

9. % make bottle.open
Make: Don't know how to make bottle.open. Stop.

10. % \(-
(-: Command not found.

11. % rm -i God
rm: remove God? y
% ls God
God not found

% make light
Make: Don't know how to make light. Stop.

12. % date me
You are not superuser: date not set
Thu Aug 25 15:52:30 PDT 1988

13. % man rear
No manual entry for rear.

14. % If I had a ) for every dollar Reagan spent, what would I have?
Too many )'s.

15. % * How would you describe Bill Clinton
*: Ambiguous.

16. % %Vice-President
%Vice-President: No such job.

17. % ls Meese-Ethics
Meese-Ethics not found

18. % "How would you rate Reagan's senility?
Unmatched ".

19. % [Where is Jimmy Hoffa?
Missing ].

20. % ^How did the^sex change operation go?
Modifier failed.

21. % cp /dev/null sex;chmod 000 sex
% more sex
sex: Permission denied

% strip show
strip: show: Permission denied

22. % who is my match?
No match.

23. % set i="DemocraticPlatform";mkdir $i;chmod 000 $i;ls $i
DemocraticPlatform unreadable

24. % awk "Polly, the ship is sinking"
awk: syntax error near line 1
awk: bailing out near line

25. % %blow
%blow: No such job.

26. % 'thou shalt not commit adultery'
thou shalt not commit adultery: Command not found.

27. $ test my argument
test: too many arguments

28. $ "Amelia Earhart"
Amelia Earhart: not found

29. $ PATH=pretending! /usr/ucb/which sense
no sense in pretending!

30. $ man -kisses dog
dog: nothing appropriate

, Mike

Saturday, June 28, 2008

Fight Phishing From The Unix Or Linux Command Line

Hey There,

This weekend, we're going to look at a little something (actually a HUGE something) that we all seem to have to deal with now (through email, IM, etc). It's called phishing and, for lack of a better explanation, it's a fraudulent way for a malicious individual, or group, to get username and password information from you using deceptive practices (was that redundant? ;) It's most commonly used to get information regarding a person's online bank accounts, credit card/mortgage management information, etc. In other words, stuff that could really screw up their lives if someone else had it.

First off, I'd like to be clear that I am "not against SPAM." As annoying as it may be, I don't begrudge anyone their right to send me a sales pitch, even if that product is a complete scam and I didn't ask to be pitched. I've been throwing stuff out that I get in my regular postal mail since I've been able to throw stuff out and I'm not sure if I've ever double-opted-in (the ethical way) or just had my address sold to some random company when it comes to internet SPAM.

Phishing, on the other hand, upsets me to no end. I do begrudge everyone their right to try and, literally, steal from others, using the internet. I feel the same way about the folks who used to steal carbons from credit cards (when that trick used to work... sadly, to a degree, it still does) and that other, more honest, faction of individuals who walk right up to the counter, point a gun in a bank teller's face and demand everyone else's hard earned cash. I'm not that this sort of activity should be encouraged, or glorified, in any way. I'm just saying it, at least, shows some character. If you're going to steal, and that's your business, why not come right out and say it? Only cowards pretend they're not pointing that figurative gun in your face and giving you a good mugging.

I can, literally, feel my email box filling with requests for password-resets at my bank of choice. Funny thing is, a lot of times, these clowns don't even bother to verify your personal information before they try and scam you. IMPORTANT NOTICE TO PHISHER'S WORLDWIDE: I do not have an account at the Federated Bank Of Hindustan. And, even if I did, it's been so long since I've attempted to speak Urdu that I can't possibly remember my clever username ;)

If you have the time and inclination to hurt these phishers, check out sites like The Anti-Phishing Working Group and PhishTank, and read up on how this whole operation works at sites like Security Focus. PhishTank actually even has a plug in for FireFox named Site Checker. Here's another site that provides a great list of possible phishing programs that may be running on your machine. The advantages that these organizations bring is that they don't merely do "anti-spam" work (which is, by all means, necessary and laudable). They actually collect data on phishing schemes, get to these people backward through their own chains of deceit and, if possible, have them put in cell block A (or wherever it is that they can legally squirrel away thieves and murderers - preferably together).

Unfortunately, though the calls-for-action are many, you should not expect any automated anti-phishing toolbars, etc, for your web browser. There's a very good reason why they're slow in coming out. Phishing is a lot like "social engineering," whereby a stranger attempts to extract personal information from you under a false pretense. Using this as an example, a stranger may also innocently request information you're not comfortable with giving under a legitimate pretense (?) or it all may be a big misunderstanding. This is why most phishing, SPAM and "social engineering" confidence violations are handled by manual submission or, at least, with some degree of user interaction. If you set Internet Explorer on auto-pilot to trash every site it thought was a phishing site, I'd bet about 50% of legitimately required bank (or other e-commerce) authorization, or confirmation, emails would end up getting skunked and ruin your web experience (and a few reputations in the process) very very quickly.

To wrap this all up (although it's certainly not the only solution), I prefer to use PhishTank since they have a very easy email submission process (You must register with them to send phishing complaints, and I encourage you to investigate their legitimacy on your own; but I like 'em a lot) which works whether or not you're sending them phishing information from your PC email client or from your Linux or Unix box (or any box that you may not have actually "received" the message on -- more below).

This is where the Linux and Unix command line comes into play. While you can easily forward an email from your mail client of choice to phish@phishtank.com (Not a mailto link on purpose), you will almost always lose some vital information in the forwarding. Instead I prefer to do the following:

1. From my email software of choice, view the entire email, including headers <-- Aside from the phishing URL, this is the most important part!

2. Log on to my Linux box (like I'm not already ;) and use any editor, like vi to paste the "entire contents" of the above output into a file.

3. Then, I send email to PhishTank using the customized email address that they've provided me. Their customized email address, for a user to send reports in from anywhere, is of the following format: phish-YOURACCOUNTNAME.YOUR_RANDOMLY_GENERATED_PHISHTANK_KEY@phishtank.com. Note that your randomly generated key will remain the same unless you want to change it. You can use any email client you like to send them the email, then. Assuming you put in it a text file called WTF, you might use mailx to send it, like so (Note that no Subject line is actually necessary):

host # mailx -s "Phishing Abuse Report" phish-YOURACCOUNTNAME.YOUR_RANDOMLY_GENERATED_PHISHTANK_KEY@phishtank.com < WTF

And, just like that, you've launched an investigation and, hopefully, put some purse-snatcher out of business.

And, lest it be said that I never said it, always double check any emails you receive from PhishTank, any bank, any service, even from me :) It never hurts to take a second and make sure you're not being scammed. The alternative can be devastating!

Best wishes,

, Mike

Friday, June 27, 2008

SunCluster Quick Command Reference

Hey There,

Well, sitting back and reflecting today, I realized that we've done our fair share of posts on cheat-sheets and such (Our LVM command reference, for example). However, when it comes to clustering systems, we've probably been unfair in our bias toward Veritas Clustering as opposed to SunCluster, for which we've only put up, I believe, one post with our Simple SunCluster Monitoring Script (Thankfully linked back to from The SunCluster Wiki). What's a poor blog to do? ;)

With that in mind, today we're going to lay down a quick command reference for SunCluster. Hopefully you'll find it useful and enjoy it :) Of course, it assumes that you know, mostly, how to use SunCluster (and are familiar with basic clustering concepts), but it can be used as an attractive wallpaper, if nothing else ;)

All of this information, and much more, can be found at Sun's Online Documentation Center for SunCluster.

Enjoy,

Quorum Commands:

host # clquorum add device
- Add a SCSI Quorum Device
host # clquorum add -t netapp_nas -p filer =nasdevicename,lun_id =IDnumdevice Nasdevice - Add a NAS Quorum Device
host # clquorum add -t quorumserver -p qshost =IPaddress, port =portnumber quorumservername - Add a Quorum Server
host # clquorum remove device - Remove a Quorum Device

Resource Type Commands:

host # clresourcetype register type
- Register a Resource Type
host # clresourcetype unregister type - Remove a Resource Type

Resource Group Commands:

host # clresourcegroup create group
- Create a Failover Resource Group
host # clresourcegroup create -S group - Create a Scalable Resource Group
host # clresourcegroup online + - Bring Online All Resource Groups
host # clresourcegroup delete group - Delete a Resource Group
host # clresourcegroup delete -F group - Delete a Resource Group and All of Its Resources
host # clresourcegroup switch -n nodename group - Switch the Current Primary Node of a Resource Group
host # clresourcegroup unmanage group - Move a Resource Group Into the UNMANAGED State
host # clresourcegroup suspend group - Suspend Automatic Recovery of a Resource Group
host # clresourcegroup resume group - Resume Automatic Recovery of a Resource Group
host # clresourcegroup set -p Failback=true + name=value - Change a Resource Group Property
host # clresourcegroup add-node -n nodename group - Add a Node To a Resource Group
host # clresourcegroup remove-node -n nodename group - Remove a Node From a Resource Group

Resource Administration Commands:

host # clreslogicalhostname create -g group lh-resource
- Create a Logical Hostname Resource
host # clressharedaddress create -g group sa-resource - Create a Shared Address Resource
host # clresource create -g group -t type resource - Create a Resource
host # clresource delete resource - Remove a Resource
host # clresource disable resource - Disable a Resource
host # clresource set -t type -p name=value + - Change a Single-Value Resource Property
host # clresource set -p name+=value resource - Add a Value to a List of Property Values; Existing values in the list are unchanged.
host # clresource create -t HAStoragePlus -g group -p FileSystemMountPoints=mount-point-list -p Affinityon=true rs-hasp - Create an HAStorage Plus Resource
host # clresource clear -f STOP_FAILED resource - Clear the STOP_FAILED Error Flag on a Resource

Device Administration Commands:

host # cldevicegroup create -t vxvm -n node-list -p failback=true vxdevgrp
- Add a VxVM Device Group
host # cldevicegroup delete devgrp - Remove a Device Group
host # cldevicegroup switch -n nodename devgrp - Switch a Device Group to a New Node
host # cldevicegroup offline devgrp - Bring Offline a Device Group
host # cldevice refresh diskname - Update Device IDs for the Cluster

Additional Administration and Monitoring Commands:

To add a Node to Cluster:


From the node to be added, which has access:

host # clnode add -c clustername -n nodename -e endpoint1, endpoint2 - Use this only if the node has access to the cluster

To remove a Node From the Cluster:

From the node to be removed, which is in noncluster
mode and has access:

host # clnode remove - Use this only if the node has access to the cluster

host # clnode evacuate nodename - Switch All Resource Groups and Device Groups Off of a Node
host # clinterconnect disable nodename:endpoint - Manage the Interconnect Interfaces
host # clinterconnect enable nodename:endpoint - To disable a cable so that maintenance can be performed, then enable the same cable afterward.
host # cluster status - Display the Status of All Cluster Components
host # command status - Display the Status of One Type of Cluster Component
host # cluster show - Display the Complete Cluster Configuration
host # command show Component - Display the Configuration of One Type of Cluster
host # command list - List One Type of Cluster Component
host # clnode show-rev -v - Display Sun Cluster Release and Version Information

To list the software versionson the current node.

host # clnode show | grep nodename
- Map Node ID to Node Name
host # cltelemetryattribute enable -t disk rbyte.rate wbyte.rate read.rate write.rate - Enable Disk Attribute Monitoring on All Cluster Disks
host # cltelemetryattribute disable -t disk rbyte.rate wbyte.rate read.rate write.rate - Disable Disk Attribute Monitoring on All Cluster Disks

SHUTTING DOWN AND BOOTING A CLUSTER

To shut Down the Entire Cluster:

host # cluster shutdown

From one node:

host # clnode evacuate
host # shutdown - Shut Down a Single Node


To boot a Single Node from the OBP:

ok> boot

To reboot a Node Into Noncluster Mode from the OBP:

ok> boot -x

, Mike

Tuesday, May 27, 2008

Using Perl On Linux To Do Mass Synchronization Of File Time Stamps

Hey There,

Today we're going to take a look at another quick and simple Perl command line execution statement that you can use to save yourself lots of time ( If it really does equal money, this post is going to be a lot more valuable than I originally thought ;)

In a previous post, we looked at using Perl to figure out how old all our files really are. This post is a little twist on that; and it's a lot less complicated. It will run on whatever version of Perl you have (unless it's ancient ;) and produced equal results on all tested Linux and Unix flavours I could get my hands on.

Today we're going to be using Perl's stat and utime functions (similar to the way we did in our previous Perl post), but instead of using them do determine the ages of all of our files, we're going to use them make all of our files conform to the specific time and date of one representative file. This trick can come in really handy if you wanted to make a system backup consisting only of files that, let's say, are a certain number of days old, and only some of the files that you know you need to backup are slightly older or newer.

The trick is very simple. All you need to do is find one specific file that has a time stamp you want to mimic, and then apply its time stamp attributes to the group of files that you want to modify the time stamp on. So (for a simple example) if you had a directory consisting of 5 files, like this:

host # ls
file1 file2 file3 file4 file5


and you needed them all to share the same time stamp as the file "file1," I'd first suggest that you take some form of online-backup, using tar or cpio (assuming that you may need to reverse the process later):

host # tar cpf /tmp/backout.tar file1 file2 file3 file4 file5

or

host # tar cpf /tmp/backout.tar f* <--- I rarely glob this globally, but, for our example, all the files we want to back up start with "f" and there aren't any others. Worst case, on a tar backup, you can just write over it if you accidentally tar-copy something you didn't want to.

Now, let's take one more look at those files. We'll use "ls -l" to show the time stamp at this time:

host # ls -l
total 4
-rw-r--r-- 1 eggi newpeople 90 May 13 22:27 file1
-rw-r--r-- 1 eggi newpeople 0 May 25 14:25 file2
-rw-r--r-- 1 eggi newpeople 0 May 25 14:25 file3
-rw-r--r-- 1 eggi newpeople 0 May 25 14:25 file4
-rw-r--r-- 1 eggi newpeople 0 May 25 14:25 file5


Now, well make them all have the same time stamp.

We need files 2 through 5 to match the time stamp of file1, as this is how the system is going to determine what to back up later in the day (fill in your own hypothetical situation here ;)

And here's all we have to do to make all the files have the same time stamp as file1 :)

host # perl -e '$x=utime ((stat($ARGV[0]))[8,9], @ARGV);print $x' file1 file[2345]

And we should be all set!

host # ls -l
total 4
-rw-r--r-- 1 eggi newpeople 90 May 13 22:27 file1
-rw-r--r-- 1 eggi newpeople 0 May 13 22:27 file2
-rw-r--r-- 1 eggi newpeople 0 May 13 22:27 file3
-rw-r--r-- 1 eggi newpeople 0 May 13 22:27 file4
-rw-r--r-- 1 eggi newpeople 0 May 13 22:27 file5


Now all the files have the same time stamp! You'll note, in the command above, that we only modifed the atime and mtime of the files. This is sufficient because Perl's utime function automatically changes the ctime (or inode change time) to the current date and time. You can include the utime in this Perl command line execution statement (no error will be generated), but it's not necessary.

If you ever do need to copy the ctime, just add the 10th index of the array returned by stat to your command line, like this:

host # perl -e '$x=utime ((stat($ARGV[0]))[8,9,10], @ARGV);print $x' file1 file[2345]

And, of course, you can use this same process on directories as well, without a direct correlation (e.g. copying file timestamps to directories, vice versa and any combination between file types that support time stamps will work :)

Here's hoping this helps you out (at least, a little ;) This simple command can be a very convenient way for you to make massive changes by applying the time stamp of any one file to as many other files as you like. And, no matter how many files you need to do this for, as long as you can wait a few moments for ingenuity to strike, the command line shouldn't get too much longer :)

Cheers,

, Mike

Thursday, November 22, 2007

A Quick Guide To Setting Up SSH Keys Network Wide

Today, in honor of the spirit of the day, I thought I'd write a little something for which you might, eventually, give many thanks :)

As we mentioned in yesterday's post, once you have SSH keys set up network-wide, so that you can passwordlessly login as yourself on all the machines you administrate, our "scmd" script can be a blessing; especially if you're ever stuck with the seemingly impossible task of retrieving some obscure bit of information off of every single box!

I won't lie to you; this is the worst part of the process. In order to get your keys initially set up, you'll have to put forth a good deal of effort (Your misery will be commensurate with the size of your network). But, once you're done, you'll be on easy street.

For our purposes, we're going to assume that you're using Solaris' (now) standard implementation of SSH (Basically OpenSSH) and craft our instructions using those assumptions. Depending on what kind of SSH client/server you're using, there may be differences in the names of certain files (e.g. authorized_keys2 might be authorized_keys -- The man pages are your friends :) We're also assuming that you actually already have user accounts on all the machines you're required to do work on.

So, to get the worst part over with, simply do the following (cut it up into chunks if it becomes frustrating. Depending on the size of your network, this is quite possible):

1. Create a file with the names or IP addresses of all of the machines you need to work on. For our purposes, format that file with one hostname/IP per line. We'll assume it's named "HostFile"

2. If you haven't done so already, generate your personal ssh-keys on the one machine that you're going to use as your central hub for administrating all the others, like so:

ssh-keygen -t dsa
(Answer yes to the default file locations -- /your/home/directory/.ssh/id_dsa and /your/home/directory/.ssh/id_dsa.pub)
(Also, just hit return both times you're prompted to enter a password. While this is just slightly less secure than entering the double-confirm password, it will completely defeat the purpose of setting this easy-administration up)

2. Run the following from your home directory (This is the first really long and trying part, as you'll have to interact manually):

for x in `cat HostFile`;do ssh -n $x "mkdir -m 700 .ssh";done
(Actuall SSH command may vary depending on your distro. I'm using the "-n" flag to avoid having SSH break out of the "for loop")

Now, if we're starting from scratch, you'll need to answer "yes" for each machine's initial prompt (it will ask you if you trust the host and want to accept and save the key) and enter your password.

3. When that's finally over, cd into your home directory's .ssh directory and run the following on the command line ("HostFile" is still in your home directory, so we're using a relative path to it here):

for x in `cat ../HostFile`;do scp id_dsa.pub $x:/your/home/dir/.ssh/authorized_keys2
(Again, you'll need to enter your password for every machine. Note that if you run a variety of servers, or build standards have changed over time so that your home directory isn't always in the exact same place on every box, you can use "~.ssh/authorized_keys2" as a substitute for "/your/home/dir/.ssh/authorized_keys2)"

4. And, now, you're all done, except to make sure that it actually worked! My favorite way to do this is to just run the same command again. Overwriting your authorized_keys2 file on all the remote hosts won't hurt, and - this time - you should be just kicking back, watching scp's progress meters fly by as you try not to fall asleep ;) Again the "~.ssh/authorized_keys2" substitution is perfectly valid and useful:

for x in `cat ../HostFile`;do scp id_dsa.pub $x:/your/home/dir/.ssh/authorized_keys2

For all the machines that still ask you for your password, take note of those hosts and troubleshoot as necessary. For the most part, you shouldn't have to worry about running into too many of those.

And from here on out, you can use the "scmd" command we put in our pre-Thanksgiving post, or your own custom scripts, to effortlessly run any command (as yourself, of course) on all your machines, by typing a single line on your hub computer.

Happy Thanksgiving :)

, Mike





Wednesday, November 21, 2007

SSH Command Runner To Help With Those Big Tiresome Tasks!

You may recall that we looked at one of the core components of this script in an earlier post, located here.

Today, in preparation for Thanksgiving, I'm putting up a little script that can help you run almost any command line you can dream up, and save the output for you in a relatively nicely formatted report. I'm calling it "scmd" (short for SSH command) but you can call it whatever you want :)

Now, of course, this script comes with a pre-requisite. For instance, it won't really help you save time if you don't have ssh-key-based passwordless logins set up for yourself on all the servers you manage. For Thanksgiving day, we'll go over how to easily set yourself up with those.

For today, assuming you've got your passwordless logins all set, feel free to modify this script as you wish, and use it to kick back and relax when the boss asks you to verify the version of Veritas Foundation Suite on all 300 servers in your farm. Otherwise, until tomorrow, enjoy only having to type in your password over and over and over again (at least you won't have to keep re-typing the command ;)

The script is posted below, for your enjoyment. Note that the "hostfile" is set to be of the format: hostname colon ip_address (e.g. host.xyz.com : 192.168.0.1). The hostname is the only necessary component of each line and all lines that begin with pound symbols (#'s) will be skipped. One host per line, please.

Again, tomorrow, we'll go over how to easily set up ssh keys for yourself network wide, and feel free to modify this little script to make your worklife as effortless as possible:


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/ksh

#
# 2007 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

trap 'rm tmpfile.$pid;echo "Caught Signal. Cleaning Up And Quitting...";exit 3"' 1 2 3 9 15

pid=$$

if [ $# -ne 1 ]
then
echo "Usage: $0 \"quoted command\""
exit 1
fi

if [ -f hostfile ]
then
hostfile="hostfile"
elif [ -f /export/home/bob/data/hostfile ]
then
hostfile="/export/home/bob/data/hostfile"
else
echo "Can't find hostfile. No hosts to ping. Out..."
exit 2
fi

command=$1
squashed_command=`echo $command|/bin/sed -e 's/ * *//g' -e 's/|//' -e 's/\///g'`
buffered_command="${command};echo";

print "" >>tmpfile.$pid
print "Report Output for \"${command}\"" >>tmpfile.$pid
print "______________________________" >>tmpfile.$pid
print "" >>tmpfile.$pid

cat $hostfile|while read name colon address
do
if [ $name == "#" ]
then
:
else
print "\nRunning \"${command}\" on \c" >>tmpfile.$pid
print "$name... " >>tmpfile.$pid
print "$name... "
/usr/bin/ssh -n $address $buffered_command 2>/dev/null |tee -a tmpfile.$pid
fi
done

mv tmpfile.$pid OUTPUT.${squashed_command}.$pid


After a successful run of, for instance - scmd "/usr/sbin/pkginfo VRTSvcs" - you'll end up with a nicely formatted report with the output from all of the servers in your "hostfile," named OUTPUT.usrsbinpkginfoVRTSvcs.14756; the only real variable in the output report name will be the process id tacked on the end so you can run the same script multiple times and not overwrite your old data.

And, oh yes, don't forget to backslash all your special characters :)

, Mike