The Linux and Unix Menagerie: Getting Faster Support For Your VCS-Clustered NetBackup Servers

Hey There,

Today's post is a little trick that anyone running Veritas/Symantec NetBackup (Linux or Unix) on VCS - Veritas Cluster Server - should know. As the title suggests, doing this one little thing will almost guarantee you more responsive support from Symantec (given the highly specific situation outlined in the first sentence, of course ;). The funny thing, though, is that many people have this problem already; they just may not have had to deal with Symantec, with regards to it, yet.

The problem you have (or will have) to face is that Symantec "does not" fully support VCS Clustered NetBackup installations if that same cluster has other Service Groups and/or major resources active on it. I'd almost say that they don't support it at all, but that's not entirely true (just keep complaining. You'll see ;). However, Symantec's stated position is that they really "don't" support it at all (and this is only hearsay, of course. Something I've heard while on the phone with another person who answered the phone at the number listed on our contract for Symantec NetBackup Support. So, of course, I may not be entirely correct, here ...).

Basically, what this means is that Symantec has VCS Cluster support for NetBackup (They own both NetBackup and Veritas Cluster Server), have VCS modules (types files, pkg's, etc) for NetBackup and even have full documentation online to help you setup your NetBackup Server(s) in VCS. Even given that fact, they do not support NetBackup in a VCS cluster that also runs, say, NFS or Oracle or even a shared mountpoint that isn't NB-related. NOTE: They especially don't support IPmultiNIC_B (I'm not sure why, since NIC auto-failback seems like a great idea to me)!! Their contention is that NetBackup is meant to function on VCS as the "only service or resource." This is fine, if you can spare 2 or 3 servers to just run NetBackup (larger companies, and data security/storage/backup companies, probably have no issue with this). However, most businesses can't afford to spare the extra servers "just" for NetBackup and will often have NetBackup in the same VCS cluster that runs an NFS group/resource, other shared mountpoints, web servers, databases, etc. Hopefully not "all" of those on one machine, but times are tight; you never know ;)

Now, here's the trick (it's not even a trick, really, just some good advice to help you out) to getting speedy support from Symantec when you call them about an issue with NetBackup on VCS: You're going to have to do a little bit of prep work and (even if it hurts) lie a little bit. If you're not comfortable "bending the truth," just remember that dishonesty is the second-best policy. Things could be worse ;)

1. Before you call, have a backup main.cf ready to go. And, this is very important if you're working on a cluster that's been around for a while, make sure that you do one of two things when Symantec Support asks for anything like an "ls" of your /etc/VRTSvcs/conf/config directory:

a. Do a very specific ls:

host # ls /etc/VRTSvcs/conf/config/main.cf <-- They don't need to see the other 500 hundred backups that are three times the size of your faked-up file ;)

b. Create a staging directory, copy only the minimal files you want into it, and then give them an "ls" of that:

host # cd /etc/VRTSvcs/conf/config/stage <-- Don't show them this!
host # ls -l *

2. As noted above, you need a clean main.cf for NetBackup Support, or you're going to end up getting bounced around from department to department, at best. If you're running anything aside from NetBackup in one cluster, you'll need to remove it (not literally, of course). Here's a quick example (This setup is going to be very basic, just to keep it short - which means the commented dependency trees that VCS puts into the conf file are removed, for this example, as well) - BTW, feel free to use this exact main.cf (just change a few names and numbers) if it works for you. It's been verified on the system I'm goofing with, but it probably won't work out-of-the-box on your system :)

Since this example is so long, I'll bid you farewell for the day, now. Hope this helps you if you ever get in a "We don't support such-and-such" or "You really shouldn't be doing such-and-such" or "It's a VCS Problem <--> It's a NetBackup Problem" sort-of-situation. And, remember, even if you goof up once and tell them the truth (who amongst us isn't guilty of being upfront every once in a while ;), you can always ask them to cancel your request ("Oh wait, I see what the problem is... Sorry, you can close the ticket" etc) and then call back later and go through normal channels so you'll hopefully get a different person (and, when you do, be sure to intimate that the problem you're having now is totally unrelated to the previous ticket, if they're efficient and happen to look for, and find, it. It's worth a shot ;)

Cheers,

--- Stripped-Down main.cf ---

include "types.cf"
include "/usr/openv/netbackup/bin/cluster/vcs/NetBackupTypes.cf"

cluster VCScluster1 (
        UserNames = { admin = EncryptedPassword }
        ClusterAddress = "10.10.10.199"
        Administrators = { admin }
        )

system VCShost1 (
        )

system VCShost2 (
        )

group ClusterService (
        SystemList = { VCShost1 = 0, VCShost2 = 1 }
        AutoStartList = { VCShost1 }
        )

        IP webip (
                Device = bge0
                Address = "10.10.10.199"
                NetMask = "255.255.255.0"
        )

        NIC webnic (
                Device = bge0
        )

        webip requires webnic

group SG_VCSNB1 (
        SystemList = { VCShost1 = 0, VCShost2 = 1 }
        AutoStartList = { VCShost1 }
        )

        IPMultiNIC nbuIP (
                Address = "10.10.10.199"
                MultiNICResName = mnic
        )

        MultiNICA mnic (
                Device @VCShost1 = { bge1 = "10.10.10.197", bge2 = "10.10.10.198" }
                Device @VCShost2 = { bge2 = "10.10.10.198", bge1 = "10.10.10.197" }
                ArpDelay = 5
                IfconfigTwice = 1
                PingOptimize = 0
        )

        DiskGroup VCShostDG (
                DiskGroup = VCShostDG
                )

        Mount MNT_NBmount_VCSNB1 (
                MountPoint = "/opt/VRTSnbu"
                BlockDevice = "/dev/vx/dsk/VCShostDG/NBmount"
                FSType = vxfs
                MountOpt = largefiles
                FsckOpt = "-y"
                )

        NetBackup APP_nbu_VCSNB1 (
                Critical = 0
                ServerName = NBU_Server
                ServerType = NBUMaster
                )

        Volume VOL_NBmount_VCSNB1 (
                Volume = NBmount
                DiskGroup = VCShostDG
                )

        APP_nbu_VCSNB1 requires nbuIP
        APP_nbu_VCSNB1 requires MNT_NBmount_VCSNB1
        MNT_NBmount_VCSNB1 requires VOL_NBmount_VCSNB1
        VOL_NBmount_VCSNB1 requires VCShostDG
        nbuIP requires mnic