Volume Groups Setup With Veritas Volume Manager

Hey There,

Today we're going to set up volume groups using the disks, and disk groups, that we setup yesterday in our post on creating disk groups with Veritas Volume Manager.

Our assumptions for the disks, and disk groups, we've setup are also spelled out in that same post about initializing disks and creating those disk groups. If you have any questions regarding the setup we're working from, they should be answered there :)

Now, we're ready to take the disk groups we created yesterday (again, not recommended to have all 3 of those exist at one time in reality. But highly recommended for the purposes of explaining the basic commands and concepts in today's post ;)

We're going to create the 4 most basic types of volumes today. These are referred to by their RAID (Redundant Array of Indepedent Disks) numbers, which translate as follows:

RAID 0 = Striped/Concatenated disk <--- Use this if you just want to combine any number of disks into one giant volume. Stripes of data are written across multiple disks, but this only enhances usability and doesn't provide any protection from disaster if a disk in the volume group fails.

RAID 1 = Mirrored disk <--- Use this if you want to mirror your disk, so that you have a simple mirror to fail over to if your primary disk fails.

RAID 5 = Striped disk with parity <--- Use this if you want to make your volume group highly available and recoverable. You'll need at least 4 disks in your RAID 5 group for this setup to be worthwhile, but it can make uptime and recovery a lot less of a hassle if you've got the disk to throw at your storage needs.

RAID 1+0 = Concatenated Mirrored disk <--- (as opposed to Mirrored Concatenated disk). The distinction to be made here is that RAID 1+0 (Striped Mirrors) consists of several disk mirrors, each reflection of which is concatenated/striped to create a larger volume. In RAID 0+1 (Mirrored Stripes), you have a set of disks Concatenated/Striped first and then Mirrored back to an equal amount of concatenated disk. Hopefully, I'm explaining this well enough, and not over-explaining it ;) Basically, with RAID 1+0, you can afford to lose a lot more disk and remain "up" than you can with RAID 0+1 (In RAID 0+1, loss of any disk A on one side of the mirror and disk C on the other side will bring you down, since each loss of a single disk, on either side, takes down that whole Concatenated miror. RAID 0+1 is like creating a much larger simple Mirrored disk than you could create with RAID 1).

For the following examples, we're going to use the command "vxassist." While you can use vxmake, etc, to perform these same functions (and, eventually, may have to if you need to do things more precisely), using vxassist is optimal when you're working with fresh disk and want to get up and running quickly :)

1. Creating a RAID 0 Volume Group named volume_group1 (From disk_group2, which contains hdc, hdd, hde and hdf):

host # /usr/sbin/vxassist -g disk_group2 -b make volume_group1 768000s layout=striped stripeunit=128 ncolumn=4

2. Creating a RAID 1 Volume Group name volume_group2 (from disk_group1, which contains hda and hdb):

host # /usr/sbin/vxassist -g disk_group1 -b make volume_group2 512000s layout=nostripe,log nmirror=2

3. Creating a RAID 5 Volume Group name volume_group3 (from disk_group3, which contains all 6 disks):

host # /usr/sbin/vxassist -g disk_group3 -b make volume_group3 1280000 layout=raid5,log stripeunit=32 ncolumn=6

4. Creating a RAID 1+0 Volume Group name volume_group4 (from disk_group3, which contains all 6 disks):

host # /usr/sbin/vxassist -g disk_group3 -b make volume_group4 768000s layout=stripe-mirror,log nmirror=2 stripeunit=128 ncolumn=2

As a bit of explanation, for each instance of vxassist that we invoke, you'll notice some common elements:

The "-g" flag simply asks us to name the disk group that we'll be using to create the volume group.

The volume group name (after "-b make") is simply what we want to call the particular volume group we're creating.

The following field is the size of the volume in sectors.

The field after that is the "layout" that we're looking to create. For RAID 0, we chose "striped." For RAID 1, "nostripe, log." For RAID 5, "raid5, log." And, for RAID 1+0, "stripe-mirror,log."

The "stripeunit" is only required for striped volumes and you can leave this out if you want to accept your VVM version's default.

The final column is also variable. For RAID 0, 5 and 1+0 we have the "ncolumn" attribute which basically translates to the number of disks in our volume group. For RAID 1 and 1+0 we have the "nmirror" attribute which translates to how many mirror devices we want to have. Two mirrors is actually the minimum you can have and really means "one disk, mirrored to one other disk." I initially found that confusing. Assuming I'm not the only guy who wondered if I was triple-mirroring by specifying two mirrors, I thought I'd just add that on there ;)

Hopefully this has been helpful to you. I'll be writing more on this, Solaris Volume Manager and LVM for Linux in posts to come :)

Best Wishes,

, Mike

linux unix internet technology

Adding Storage using Volume Manager in a Veritas Cluster

Today, we're going to take a look at adding extra disk to a Veritas Cluster using Veritas Volume Manager. We'll assume for the purposes of this post, that you've been asked to add 35gb of space to a Veritas volume (filesystem: /veritas/a - volume name: veritas_a). Now we'll walk, step by step, through determining whether we have the disk available to add, and adding it if we do.

The first thing you'll want to do is determine what disk group(s) the filesystem/volume belongs to. Generally, you'll be told this when you're asked to add disk to an existing system, but we'll just assume the very worst ;) In any event, even if you are told, it's good practice to verify that the filesystem/volume does, in fact, belong to the disk group(s) the request is asking to have you augment.

Now, since we're dealing with a Veritas Cluster, if the disk group is shared between more than one server node, you can run:

vxdctl -c mode

and that will show you what the master node is. This server is where you'll want to execute all of your Veritas Volume Manager commands. If the cluster is inactive, vxdctl will tell you that, and you'll need to execute your commands on the server you're already on.

Now, either on the Cluster's master node, or the machine you're on if the cluster is inactive, get a list of all the disk groups, like so:

vxdg list
NAME STATE ID
rootdg enabled 1199919191.1025.host1
mydg enabled 8373777737.1314.host2

Since the user requested more filespace under /veritas/a, we'll need to see what disk group this filesystem/volume belongs to:

df -kl | grep veritas
/dev/vx/dsk/mydg/veritas_a 34598912 24632328 9903320 72% /veritas/a
/dev/vx/dsk/mydg/veritas_b 34598912 649112 33687984 2% /veritas/b
/dev/vx/dsk/mydg/veritas_c 35350080 17696 35056360 1% /veritas/c

We can see that, under /dev/vx/dsk, mydg is showing as the disk group for our /veritas/a filesystem, and veritas_a is showing as the volume. In cases like this, it's a time saver if you just set the default group once on the command line, so you don't have to use the "-g" option for every command you run:

vxdctl defaultdg mydg

Now you'll run some standard commands to make sure that your system can see all the disk available to it and make sure that the data is fresh:

devfsadm -Cv (note that the cluster will only update the volume manager configuration for cluster nodes that use the mydg diskgroup -- we don't have to specify that here, because we defined it as the default above)
vxdctl enable (this is basically the second part of the same update. The devfsadm command is Solaris specific while vxdctl is Veritas specific)

Now we need to check for free space, to see if we can accommodate the request with what we have available to us at present:

vxdg -g mydg list <-- Again "-g mydg" is optional because it's the default.
DISK DEVICE TAG OFFSET LENGTH FLAGS
DISK1 c5t27d27s2 c5t27d27 69198720 1501440 n
DISK2 c5t27d30s2 c5t27d30 69198720 1501440 n
DISK3 c5t27d28s2 c5t27d28 69198720 1501440 n

This output is, admittedly, a pain to read, because it expects you to be able to calculate disk offsets in your head. For this output, if the offset were zero and length was 70700160 and flags were '-' then that DISK would have 35 GB of free space available to us. Here all the listed storage has been partially used up. Hence, there is not enough storage space available!. We can also verify this by running

vxdg free (optionally: vxdg -g mydg free) to see if there is any free space left in the disk group. Either way, we're out of luck, it would seem.

So, now we're going to have to look into Veritas' disk configuration and see if there isn't a spare disk (hope) that we can pick up and import into the mydg diskgroup so that we can add the space (One thing to note in the output below is that all online disk shows its disk group membership even though it's imported locally):

vxdisk -o alldgs list <--- (abbreviated to keep this post bearable ;)
DEVICE TYPE DISK GROUP STATUS
c0t0d0s2 sliced - - error
c0t2d0s2 sliced disk02 rootdg online
c1t2d0s2 sliced disk01 rootdg online
c5t27d40s2 sliced - (mydg200) online
c5t27d41s2 sliced - (mydg2) online
c5t27d300s2 sliced - (mydg300) online
c5t20d99s2 sliced - - error

Luckily, it appears that disk c5t20d99s2 is available and doesn't belong to any existing disk groups! We'll use the command line to initialize this free disk and add it to our disk group (mydg):

vxdiskadd c5t20d99s2

You will be asked if you want to initialize the disk (or re-initialize it, if it's already initialized). Answer yes to either option, to keep things simple. Then it will ask you what disk group you would like to add the disk to. Answer: mydg. It will ask you if you want to use a default disk name. Depending on your situation, this may be okay, but we'll say no and add it as "DISK4" to keep with our naming standard. Then you will be prompted if you want to add the disk as a spare for the mydg disk group. Answer no to this, or you won't be able to use it for the additional storage you want! Answer no again, when it asks you if you want to encapsulate the disk, and then answer yes when it asks you if you want to initalize it. You can then choose to perform a surface analysis before adding it (It says it's highly recommended but my experience is it's generally not necessary and takes a very very very long time). Phew... that wasn't too much to info to have to feed to a single command ;)

Now, we'll want to grow the mydg disk group's volume and filesystem size. We've added the new disk, so we have more space, but the user can't see it or use it at this point. Now, common wisdom is to use the vxassist command. It's common, because it can be a great command to use (very simple), but I prefer to use "vxresize" because, unless your filesystem is vxfs, running vxassist to grow the volume (/veritas_a) can cause serious issues when growing the filesystem (if, for instance, you're using the Veritas Volume Manager but have your volume filesystem type set to Solaris "ufs")!

Calculate the maximum length of your new volume using vxassist (this part is okay - forget about what I just said above - that only applies to using vxassist to do the actually growing):

vxassist -g mydg (optional, if default is already set) maxsize <--- This will be the NEW_LENGTH argument for the next command.

vxresize -F (vxfs or ufs,usually) -g mydg (optional, still) -x (to make sure that the operation fails if the volume doesn't increase in length) veritas_a NEW_LENGTH <-- The last time I checked, - and another reason I prefer this method - this command grows your filesystem with the volume so you don't need to run any more steps after this.

Of course, you can keep tabs on all this, after you're done typing, using the vxtask command. At no point during this entire operation will you need to unmount your filesystem. I believe it's "recommended" (because something could always go wrong), but I've never found it to be necessary (technically, it's not supposed to fail, but Veritas has to cover itself, just in case)

Once the task is finished, your "df -k /veritas/a" output should show your new disk available and ready for use!

Cheers,

, Mike

linux unix internet technology