Adding Storage using Volume Manager in a Veritas Cluster

Today, we're going to take a look at adding extra disk to a Veritas Cluster using Veritas Volume Manager. We'll assume for the purposes of this post, that you've been asked to add 35gb of space to a Veritas volume (filesystem: /veritas/a - volume name: veritas_a). Now we'll walk, step by step, through determining whether we have the disk available to add, and adding it if we do.

The first thing you'll want to do is determine what disk group(s) the filesystem/volume belongs to. Generally, you'll be told this when you're asked to add disk to an existing system, but we'll just assume the very worst ;) In any event, even if you are told, it's good practice to verify that the filesystem/volume does, in fact, belong to the disk group(s) the request is asking to have you augment.

Now, since we're dealing with a Veritas Cluster, if the disk group is shared between more than one server node, you can run:

vxdctl -c mode

and that will show you what the master node is. This server is where you'll want to execute all of your Veritas Volume Manager commands. If the cluster is inactive, vxdctl will tell you that, and you'll need to execute your commands on the server you're already on.

Now, either on the Cluster's master node, or the machine you're on if the cluster is inactive, get a list of all the disk groups, like so:

vxdg list
NAME STATE ID
rootdg enabled 1199919191.1025.host1
mydg enabled 8373777737.1314.host2

Since the user requested more filespace under /veritas/a, we'll need to see what disk group this filesystem/volume belongs to:

df -kl | grep veritas
/dev/vx/dsk/mydg/veritas_a 34598912 24632328 9903320 72% /veritas/a
/dev/vx/dsk/mydg/veritas_b 34598912 649112 33687984 2% /veritas/b
/dev/vx/dsk/mydg/veritas_c 35350080 17696 35056360 1% /veritas/c

We can see that, under /dev/vx/dsk, mydg is showing as the disk group for our /veritas/a filesystem, and veritas_a is showing as the volume. In cases like this, it's a time saver if you just set the default group once on the command line, so you don't have to use the "-g" option for every command you run:

vxdctl defaultdg mydg

Now you'll run some standard commands to make sure that your system can see all the disk available to it and make sure that the data is fresh:

devfsadm -Cv (note that the cluster will only update the volume manager configuration for cluster nodes that use the mydg diskgroup -- we don't have to specify that here, because we defined it as the default above)
vxdctl enable (this is basically the second part of the same update. The devfsadm command is Solaris specific while vxdctl is Veritas specific)

Now we need to check for free space, to see if we can accommodate the request with what we have available to us at present:

vxdg -g mydg list <-- Again "-g mydg" is optional because it's the default.
DISK DEVICE TAG OFFSET LENGTH FLAGS
DISK1 c5t27d27s2 c5t27d27 69198720 1501440 n
DISK2 c5t27d30s2 c5t27d30 69198720 1501440 n
DISK3 c5t27d28s2 c5t27d28 69198720 1501440 n

This output is, admittedly, a pain to read, because it expects you to be able to calculate disk offsets in your head. For this output, if the offset were zero and length was 70700160 and flags were '-' then that DISK would have 35 GB of free space available to us. Here all the listed storage has been partially used up. Hence, there is not enough storage space available!. We can also verify this by running

vxdg free (optionally: vxdg -g mydg free) to see if there is any free space left in the disk group. Either way, we're out of luck, it would seem.

So, now we're going to have to look into Veritas' disk configuration and see if there isn't a spare disk (hope) that we can pick up and import into the mydg diskgroup so that we can add the space (One thing to note in the output below is that all online disk shows its disk group membership even though it's imported locally):

vxdisk -o alldgs list <--- (abbreviated to keep this post bearable ;)
DEVICE TYPE DISK GROUP STATUS
c0t0d0s2 sliced - - error
c0t2d0s2 sliced disk02 rootdg online
c1t2d0s2 sliced disk01 rootdg online
c5t27d40s2 sliced - (mydg200) online
c5t27d41s2 sliced - (mydg2) online
c5t27d300s2 sliced - (mydg300) online
c5t20d99s2 sliced - - error

Luckily, it appears that disk c5t20d99s2 is available and doesn't belong to any existing disk groups! We'll use the command line to initialize this free disk and add it to our disk group (mydg):

vxdiskadd c5t20d99s2

You will be asked if you want to initialize the disk (or re-initialize it, if it's already initialized). Answer yes to either option, to keep things simple. Then it will ask you what disk group you would like to add the disk to. Answer: mydg. It will ask you if you want to use a default disk name. Depending on your situation, this may be okay, but we'll say no and add it as "DISK4" to keep with our naming standard. Then you will be prompted if you want to add the disk as a spare for the mydg disk group. Answer no to this, or you won't be able to use it for the additional storage you want! Answer no again, when it asks you if you want to encapsulate the disk, and then answer yes when it asks you if you want to initalize it. You can then choose to perform a surface analysis before adding it (It says it's highly recommended but my experience is it's generally not necessary and takes a very very very long time). Phew... that wasn't too much to info to have to feed to a single command ;)

Now, we'll want to grow the mydg disk group's volume and filesystem size. We've added the new disk, so we have more space, but the user can't see it or use it at this point. Now, common wisdom is to use the vxassist command. It's common, because it can be a great command to use (very simple), but I prefer to use "vxresize" because, unless your filesystem is vxfs, running vxassist to grow the volume (/veritas_a) can cause serious issues when growing the filesystem (if, for instance, you're using the Veritas Volume Manager but have your volume filesystem type set to Solaris "ufs")!

Calculate the maximum length of your new volume using vxassist (this part is okay - forget about what I just said above - that only applies to using vxassist to do the actually growing):

vxassist -g mydg (optional, if default is already set) maxsize <--- This will be the NEW_LENGTH argument for the next command.

vxresize -F (vxfs or ufs,usually) -g mydg (optional, still) -x (to make sure that the operation fails if the volume doesn't increase in length) veritas_a NEW_LENGTH <-- The last time I checked, - and another reason I prefer this method - this command grows your filesystem with the volume so you don't need to run any more steps after this.

Of course, you can keep tabs on all this, after you're done typing, using the vxtask command. At no point during this entire operation will you need to unmount your filesystem. I believe it's "recommended" (because something could always go wrong), but I've never found it to be necessary (technically, it's not supposed to fail, but Veritas has to cover itself, just in case)

Once the task is finished, your "df -k /veritas/a" output should show your new disk available and ready for use!

Cheers,

, Mike

linux unix internet technology