Duplicating And Modifying NetBackup Policies, Schedules And Client Via Command Line On Linux And Unix

Hey There,

SPOILER ALERT: THE BULLET POINTS IN THIS POST ARE IN ALL CAPS. I'M SORRY FOR SHOUTING. I HAVE NO SENSE OF HOW OBNOXIOUSLY LOUD I AM!

Actually tomorrow's post has turned into today's and I'm going to do today's tomorrow. I don't know why I felt compelled to write that. I guess I just don't want you to lose track of time like I do ;)

Today we're going to check out a way to copy (and modify) an existing host's NetBackup policies, clients and schedules to a new host, and then modify those policies, clients and schedules (just slightly) to suit the new host (you can't backup to a hostname that isn't the name of the host you want to back up. ...if you want to backup your the new host, that is ;)

The beauty of it is that it can all be done from the command line. No more UI GUI for you :)

And now, in one of my driest pieces to date, a step by step guide on how to perform the process I just described above (If you're interested in staying away from the GUI, check out our old post on our NetBackup command line activity monitor)

1. GET YOUR POLICIES AND SCHEDULES FROM THE KNOWN-GOOD HOST YOU WANT TO EMULATE/REPLICATE (orig_host will be the known-good host and new_host will be the host you want to be just about the same. Also, we're assuming you've installed the base product. This walk-through is focused only on policies, policy schedules and policy clients).

orig_host # ls /usr/openv/netbackup/db/class (you may have more than this - generally there's one directory per policy. Each directory includes the schedules for the policy the directory name indicates):

HOST_BACKUP           HOST_OS_BACKUP      HOST_CATALOG

You may want to copy off the old policy and schedule output, just so you can compare it with your new host setup later and make sure you didn't forget to change any instances of an old hostname to your new hostname

orig_host # /usr/openv/netbackup/bin/admincmd/bppllist -allpolicies -U >/var/tmp/policies.txt

Then just tar that all up:

orig_host # tar cpf /var/tmp/policies_and_schedules.tar /usr/openv/netbackup/db/class /var/tmp/policies.txt

2. VERIFY THAT EITHER NO "class" DIRECTORY EXISTS ON YOUR NEW HOST, THE EXISTING "class" DIRECTORY IS EMPTY OR THAT IT JUST CONTAINS THE STANDARD TEST GARBAGE YOUR SETUP MAY COME WITH - DELETE IT OR UNTAR OVER IT IF YOU WANT TO (USER DISCRETION ADVISED!) - THIS MAY NOT BE A BIG DEAL SINCE YOU PROBABLY HAVE NO SCHEDULES OR POLICIES ON YOUR NEW HOST ANYWAY.

new_host # ls /usr/openv/netbackup/db
/usr/openv/netbackup/db: No such file or directory
new_host # ls /usr/openv/netbackp/db/class
/usr/openv/netbackp/db/class: No such file or directory
new_host # ls /usr/openv/netbackp/db
/usr/openv/netbackp/db: No such file or directory

3. AFTER SCP'ING/COPYING THE TAR FILE FROM YOUR KNOWN GOOD HOST, UNTAR IT ON THE NEW HOST AND LIST ALL POLICIES TO DOUBLE VERIFY - THE HOST NAMES WILL BE WRONG (FOR YOUR NEW HOST) SO THE DIFF WON'T BE TOO DIFFERENT!

new_host # tar xpf /var/tmp/policies_and_schedules.tar
new_host # /usr/openv/netbackup/bin/admincmd/bppllist -allpolicies -U >/var/tmp/policies_new.txt
diff /var/tmp/policies.txt /var/tmp/policies_new.txt

4. CHECK THAT ALL THE POLICIES ARE THERE IN YOUR policies_new.txt FILE (YOU CAN LOOK IN BPADM, TOO). ALL HOSTNAMES AND TAPE RESIDENCE NAMES WILL STILL BE WRONG (ANYTHING THAT CONTAINS A HOSTNAME, ANYWAY)

new_host # /usr/openv/netbackup/bin/admincmd/bppllist -M new_host
HOST_BACKUP
HOST_OS_BACKUP
HOST_CATALOG

5. DETERMINE YOUR TAPE RESIDENCE (TLD NAME) ON THE NEW HOST (FIRST ENTRY IN OUTPUT BELOW)

new_host # /usr/openv/netbackup/bin/admincmd/bpstulist
new_host-hcart3-robot-tld-o 2 new_host 8 0 20 2 0 "*NULL*" 0 1 1048576 *NULL* 0 1 0 0 0 0 *NULL* new_host

"new_host-hcart3-robot-tld-o" is the residence name

6. ASSUMING ALL POLICIES LISTED ABOVE ARE GOOD, YOU WON'T HAVE TO MODIFY THIS LINE TO REMOVE AND/OR UPDATE THE RESIDENCE FOR ALL POLICIES (OF COURSE, THEY PROBABLY AREN'T SINCE THIS POST WAS WRITTEN FOR THE GENERAL PUBLIC ;)

new_host # while read line;do /usr/openv/netbackup/bin/admincmd/bpplinfo $line -modify -residence new_host-hcart3-robot-tld-o;done <<< "`/usr/openv/netbackup/bin/admincmd/bppllist -M new_host`"

7. LIST OUT ALL THE SCHEDULES NOW - NOTE THAT YOU MAY NOT WANT TO MODIFY YOUR CATALOG POLICY AND THAT SOME POLICIES HAVE MORE THAN ONE CLIENT (This may vary from setup to setup, so it's important to check!)

NOTE ALSO THAT, SINCE YOU'RE NOT CHANGING YOUR SCHEDULES, ALL YOU NEED TO DO TO MODIFY THEM IS TO CHANGE THE CLIENTS ASSOCIATED WITH THEM, WHICH GETS DONE IN THE FOLLOWING STEPS, AS WELL :)

new_host # while read line;do echo "POLICY $line";/usr/openv/netbackup/bin/admincmd/bpplclients $line;done <<< "`/usr/openv/netbackup/bin/admincmd/bppllist -M new_host`"
POLICY HOST_BACKUP
Hardware         OS               Client
---------------  ---------------  --------------
Solaris          Solaris10        orig_host
POLICY HOST_OS_BACKUP
Hardware         OS               Client
---------------  ---------------  --------------
Solaris          Solaris10        orig_host
Solaris          Solaris10        orig_host2
POLICY HOST_CATALOG
Hardware         OS               Client
---------------  ---------------  --------------
Solaris          Solaris8         orig_host

8. WITH THE SAME ASSUMPTIONS AS ABOVE, USE THIS TO CHANGE CLIENTS FOR YOUR POLICIES - CHOPPING OFF LAST POLICY AND MODIFIYING CLIENTS BY TAKING INPUT FROM LIST, MODIFYING AND THEN RUNNING "ADD NEW" AND "DELETE OLD" IN A LOOP (The simple sed expression to change the host name can be as specific as you like)

A. DO A "PROOF OF CONCEPT" RUN FIRST

while read line;do count=3;for x in $(/usr/openv/netbackup/bin/admincmd/bpplclients $line|awk '{print $3}'|grep db|/usr/bin/sed 's/orig_/new_/g'|xargs echo);do oldhwos=$(/usr/openv/netbackup/bin/admincmd/bpplclients $line|sed -n ${count}p|awk '{print $1,$2}');let count=$count+1;echo "POC TEST $line /usr/openv/netbackup/bin/admincmd/bpplclients $line -add $x $oldhwos";done;done <<< "`/usr/openv/netbackup/bin/admincmd/bppllist -M new_host|egrep -iv 'vault|catalog'`"
POC TEST HOST_BACKUP /usr/openv/netbackup/bin/admincmd/bpplclients HOST_BACKUP -add new_host Solaris Solaris10
POC TEST HOST_OS_BACKUP /usr/openv/netbackup/bin/admincmd/bpplclients HOST_OS_BACKUP -add new_host Solaris Solaris10
POC TEST HOST_OS_BACKUP /usr/openv/netbackup/bin/admincmd/bpplclients HOST_OS_BACKUP -add new_host2 Solaris Solaris10

B. THEN - IF ALL IS WELL - DO IT FOR REAL

while read line;do count=3;for x in $(/usr/openv/netbackup/bin/admincmd/bpplclients $line|awk '{print $3}'|grep db|/usr/bin/sed 's/orig_/new_/g'|xargs echo);do oldhwos=$(/usr/openv/netbackup/bin/admincmd/bpplclients $line|sed -n ${count}p|awk '{print $1,$2}');let count=$count+1;echo "Running /usr/openv/netbackup/bin/admincmd/bpplclients $line -add $x $oldhwos";/usr/openv/netbackup/bin/admincmd/bpplclients $line -add $x $oldhwos;done;done <<< "`/usr/openv/netbackup/bin/admincmd/bppllist -M new_host|egrep -iv 'vault|catalog'`"
Running /usr/openv/netbackup/bin/admincmd/bpplclients HOST_BACKUP -add new_host Solaris Solaris10
Running /usr/openv/netbackup/bin/admincmd/bpplclients HOST_OS_BACKUP -add new_host Solaris Solaris10
Running /usr/openv/netbackup/bin/admincmd/bpplclients HOST_OS_BACKUP -add new_host Solaris Solaris10

C. CHECK TO MAKE SURE THE NEW CLIENTS ARE ADDED

while read line;do /usr/openv/netbackup/bin/admincmd/bpplclients $line ;done <<< "`/usr/openv/netbackup/bin/admincmd/bppllist -M new_host`"         
Hardware         OS               Client
---------------  ---------------  --------------
Solaris          Solaris10        orig_host
Solaris          Solaris10        new_host
Hardware         OS               Client
---------------  ---------------  --------------
Solaris          Solaris10        orig_host
Solaris          Solaris10        orig_host1
Solaris          Solaris10        new_host
Solaris          Solaris10        new_host1
Hardware         OS               Client
---------------  ---------------  --------------
Solaris          Solaris8         orig_host
Solaris          Solaris8         new_host

D. NOW SOME POC TO MAKE SURE THE DELETE OF THE OLD HOSTS WORKS OKAY

while read line;do for x in $(/usr/openv/netbackup/bin/admincmd/bpplclients $line|awk '{print $3}'|grep orig_|xargs echo);do echo "POC TEST $line /usr/openv/netbackup/bin/admincmd/bpplclients $line -delete $x";done;done <<< "`/usr/openv/netbackup/bin/admincmd/bppllist -M new_host|egrep -iv 'vault|catalog'`"
POC TEST HOST_BACKUP /usr/openv/netbackup/bin/admincmd/bpplclients HOST_BACKUP -delete orig_host
POC TEST HOST_OS_BACKUP /usr/openv/netbackup/bin/admincmd/bpplclients HOST_OS_BACKUP -delete orig_host
POC TEST HOST_OS_BACKUP /usr/openv/netbackup/bin/admincmd/bpplclients HOST_OS_BACKUP -delete orig_host1

E. THEN REMOVE THEM FOR REAL

while read line;do for x in $(/usr/openv/netbackup/bin/admincmd/bpplclients $line|awk '{print $3}'|grep orig_|xargs echo);do echo "Running /usr/openv/netbackup/bin/admincmd/bpplclients $line -delete $x";/usr/openv/netbackup/bin/admincmd/bpplclients $line -delete $x;done;done <<< "`/usr/openv/netbackup/bin/admincmd/bppllist -M new_host|egrep -iv 'vault|catalog'`"
Running /usr/openv/netbackup/bin/admincmd/bpplclients HOST_BACKUP -delete orig_host
Running /usr/openv/netbackup/bin/admincmd/bpplclients HOST_OS_BACKUP -delete orig_host
Running /usr/openv/netbackup/bin/admincmd/bpplclients HOST_OS_BACKUP -delete orig_host1

F. THEN CHECK AGAIN

while read line;do /usr/openv/netbackup/bin/admincmd/bpplclients $line ;done <<< "`/usr/openv/netbackup/bin/admincmd/bppllist -M new_host`"         
Hardware         OS               Client
---------------  ---------------  --------------
Solaris          Solaris10        new_host
Hardware         OS               Client
---------------  ---------------  --------------
Solaris          Solaris10        new_host
Solaris          Solaris10        new_host1
Hardware         OS               Client
---------------  ---------------  --------------
Solaris          Solaris8         orig_host

9. NOW THAT YOU'VE GOT THE POLICIES, SCHEDULES AND CLIENTS MOVED OVER, MAKE SURE THAT YOUR CATALOG JOB EXISTS (IF YOU USE ONE)

new_host # /usr/openv/netbackup/bin/admincmd/bppllist HOST_CATALOG
CLASS HOST_CATALOG *NULL* 0 650000 0 *NULL*
NAMES
INFO 35 1 0 0 *NULL* 0 0 1 1 0 1 2 0 0 0 0 0 0 1236208342 EA91EDF01DD111B2AB1900144FEAC8FE 1 0 0 0 0 0 0 0 2 0 0 0 0 0 6
KEY *NULL*
BCMD *NULL*
RCMD *NULL*
RES new_host-hcart3-robot-tld-o *NULL* *NULL* *NULL* *NULL* *NULL* *NULL* *NULL* *NULL* *NULL*
POOL CATALOGPOOL NetBackup NetBackup NetBackup NetBackup NetBackup NetBackup NetBackup NetBackup NetBackup
FOE 0 0 0 0 0 0 0 0 0 0
SHAREGROUP *ANY*
DATACLASSIFICATION *NULL*
CLIENT new_host Solaris Solaris8 0 0 0 0 *NULL*
INCLUDE CATALOG_DRIVEN_BACKUP

10. NOW JUST DEACTIVE ANY POLICIES THAT ARE ACTIVE AND SHOULDN'T BE AND ACTIVATE ANY POLICIES THAT SHOULD BE ACTIVE BUT AREN'T!

All set!

CHEERS! ;)

, Mike

---

Discover the Free Ebook that shows you how to make 100% commissions on ClickBank!

---

Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

---

Simple SCP Utility For Network Wide File Distribution

Hey again,

This little shell script is a somewhat follow-up to an older post we did on creating an SSH command line runner. It's basically the same concept, except this time we're looking specifically at copying files all over the network, using SCP, rather than executing commands all over the network using SSH (One might argue that, under the covers, they're the same thing. But one might also argue that each has an advantage over the other, depending on what it's being used for and there's no argument that neither takes the exact same command line options :)

One other thing to note, before you begin using this script, is that it assumes that you have passwordless key-based SSH authentication set up on all the hosts you're going to be copying to. If you haven't got that set up already, check out our post on setting up your SSH keys network wide quickly and easily.

This script is fairly simple to run and should work on most flavors of Linux and Unix. If any modification is necessary it should be minor, and center around the line where we actually invoke SCP. It takes two arguments (where we're copying from and where we're copying to) and can be run simply, from the command line, like so:

host # ./scopy /users/bob/file /users/bob/file

Of course, you may note that the above example is incredibly specific. You don't need to include the directory if you don't want to (I just usually do so to be sure things end up where they're supposed to). You could easily do this, as well (assuming that your home directory isn't in the same location on every box on your network):

host # ./scopy /users/bob/file ~/bob/file

or any number of variations. Have fun with it :) The only thing it looks for outside of itself, is a host file, which I've cleverly named "hostfile" ;) This is completely arbitrary and can be changed, as well. The format I've chosen is "name : IP Address," like this:

host1 : 192.168.0.10

with one entry per line. Again, with some simple modification, this can be changed easily to suit your taste.

Hope you enjoy it and it helps you out some :)

Cheers,





This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/ksh

#
# scopy - scp files from one location to 
#   all of your network hosts
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

trap 'rm tmpfile.$pid;echo "Caught Signal.  Cleaning Up And Quitting...";exit 3"' 1 2 3 9 15

pid=$$

if [ $# -ne 2 ] 
then
 echo "Usage: $0 fromDirFile ToDirFile"
 exit 1
fi

if [ -f hostfile ]
then
 hostfile="hostfile"
elif [ -f /users/bob/data/hostfile ]
then
 hostfile="/users/bob/data/hostfile"
else
 echo "Can't find hostfile.  No hosts to copy to.  Out..."
 exit 2
fi

from=$1
to=$2
squashed_command=`echo $1 $2|/bin/sed -e 's/ * *//g' -e 's/|//' -e 's/\///g'`

print "" >>tmpfile.$pid
print "Report Output for \"scp $1 $2\"" >>tmpfile.$pid
print "______________________________" >>tmpfile.$pid
print "" >>tmpfile.$pid

cat $hostfile|while read name colon address
do
 if [ $name == "#" ]
 then
  :
 else
  print "\nCopying \"$1 to $2\" on \c" >>tmpfile.$pid
  print "$name... " >>tmpfile.$pid
  print "$name... \"scp $1 $2\"... "
  /usr/bin/scp -r $1 ${address}:$2 2>/dev/null|tee -a tmpfile.$pid
 fi
done 

mv tmpfile.$pid OUTPUT.${squashed_command}.$pid

, Mike

linux unix internet technology

Sifting Through the PROM

Every once in a while, like most admins (I hope;), I'll find myself in a situation where I'm working on a downed Solaris box and stuck at the PROM level. For the purposes of this post, we'll assume that diagnostics over and above the PROM level are impossible (Which is sometimes true).

The issue here isn't that I'm stuck at the PROM. That's no big deal and happens often. The problem is, every so often, I'll find myself stuck at the PROM on a Sun system type I've never worked on before. Depending upon how varied the PROM interface and commands are for that system, digging into my old bag of tricks my have me drawing up blanks left and right. The PROM "is" always very courteous about letting you know you got the command you typed in wrong and doesn't offer much in the way of help. For instance:

ok> show-me-the-money
show-me-the-money ?
ok>

The output from the PROM, letting you know it doesn't know what you're talking about, can vary also.

One of the most useful commands I've ever run across, at the PROM level, is called "sifting." Since you really have no help immediately available, and searching for particular commands in manuals or on the net can suck up huge amounts of time, this command is a life saver. Some folks refer to it as a "sifting dump" (the technical name), but the end result and usage are the same.

Best of all, the sifting command is incredibly easy to use (and the "one" command you should remember if you forget everything else ;). The command itself is incredibly simplistic. All it does is find all the OBP/PROM commands that contain the string you've specified as the only argument, and prints them out to the screen. That's it!

So, even if you've got your PROM chops down, when life throws you that curveball, you can still find the solution easily. Consider that you're working on a 4500 series server that hasn't been rebooted in 3 years. On boot up, everything seems to be going fine and, suddenly, you're presented with a poorly worded error about clock's being out of sync. What??? Assuming, still, that you have no access to any reference material of any kind on any medium, you can still figure this one out using sifting.

To continue, once that error pops up, you get dropped back to the PROM. Subsequent attempts to just boot again and hope the problem goes away have equal results. At this point, you know two things from reading the error itself: One clock on the system is out of sync with another clock, and/or the opposite. But how to synchronize them, since that would seem to be all the boot process wants you to do? This is where sifting becomes invaluable. Just like grep, the smaller the search string you provide, the more results you'll get. In this instance, we could do:

sifting clock

and we'd get back, among a few other things that can easily be dismissed:

copy-clock-tod-to-io-boards
copy-io-board-tod-to-clock-tod

I usually opt to copy the system clock to the I/O boards (copy-clock-tod-to-io-boards). My idea of best practices dictates that I'd go about this like so:

setenv auto-boot? false
reset-all
copy-clock-tod-to-io-boards
setenv auto-boot? true
boot

And, magically, the "clocks out of sync" error is gone!

The little example I showed you above is just the beginning. You can use sifting to find any command you need to know the exact name of (the PROM is so unforgiving ;). Try "sifting probe" if you can't remember what that probe command was that you needed to run, or "sifting net" if you've forgotten the exact name of "show-nets" or "test-net" (notice the seemingly arbitrary difference; one is plural and the other is singular). You can run sifting against a single character if you want to. Just get ready to have plenty of options ;)

Enjoy your exploration of the PROM commands available on your system of choice, and remember: sifting through the PROM will eventually lead you to the answer :)

, Mike

linux unix internet technology