How To Get Started With Logical Volume Management In Linux

Hey again,

Today, since past posts regarding disk management software have mostly been limited to Veritas Volume Manager, I thought we'd look at LVM on Linux (In a future post, we'll check out the differences between LVM 1 and 2, but for now I figured we could keep it simple and broad).

If you use any of the "major player" Linux flavours out there, you've probably noticed (over the last year or so) that standard installs show all your disks listed out in df, like:

/dev/mapper/vg01-lvol01 246M   75.2M    158M 32%    /dir

rather than the old-style:

/dev/hdd1      246M   75.2M    158M    32%    /dir

that you've (if you're anything like me and only have two hard drives on the home PC you're running Linux on ;) grown to love.

And, while I think it's fantastic that Linux has grown into a mostly-user-friendly operating system with GUI's to make all of the extended partitioning and volume management more accessible, I also think there's something to be said for knowing how to be able to set all that stuff up on your own. You never know when your main display (or X-windows session) will die on you and, at some point, you may end up having to do some disk repair on the lonely old command line.

With that in mind, we'll step through the most basic LVM setup: Getting from /dev/hdd to /dev/mapper/vg01-lvol01 from the command prompt.

Our basic (and huge assumption) here is that we've got a primary drive to work from (although we could do this just as easily from a rescue, or live, CD-ROM if we needed to perform this work on our primary disk during the Linux installation process or a freakish-nightmare of a fix)

First, we'll want to use fdisk to create a "new" "primary" partition that consists of the entire disk (512Mb in size). Barring a primer on fdisk, we'll want to set up our disk to have one primary partition that consists of all cylinders (first and last should be the default start and end values) and is set to the "Linux LVM" partition type.

Once we're done with the stuff we'd "normally" do (for the most part) to set up any other type of partition table on our /dev/hdd disk, we can begin to play around with the LVM commands. Note that if you chose any partition type other than LVM when setting up your partition, using the following commands will, at best, result in error messages and accomplish nothing :)

If you've ever used HP-UX before, a lot of these commands will seem familiar. There's a reason, but the legalities of discussing it (and possibly getting bits of it wrong) fall outside the scope of my pocketbook ;)

The first thing we'll do is create a "physical volume," like so:

host # pvcreate /dev/hdd1

Yes, it was a physical volume already. The logical-physical relationship sometimes doesn't hold up to scrutiny outside of the "rules" ;)

Next, well create a "volume group." This is necessary, because "logical volumes" run in packs... pardon my lack of propriety ;) We could use the "-s" option here to define the "physical extent" size, but we'll leave that alone. To say the least, specifying this requires some research and planning. If you get that number wrong, things can go South later, and you'll have to do some ugly backtracking. We just want to define a "volume group" so we have something to work with. We'll call it vg01, for lack of a more original name.

host # vgcreate vg01 /dev/hdd1

Next, we'll create a "logical volume" (we'll name this one "lvol01" to keep things consistent) of 246 Megabytes size. That should fit nicely inside vg01 (which we're basically using to represent the entire partition /dev/hdd1). We could make our "logical volume" as large as the entire partition and/or "volume group", but that would defeat the purpose of this whole exercise. The aim, of course, is to end up with flexible volumes so we can resize them later if we need to.

host # lvcreate -L 246M -n lvol01 vg01

This is where the /dev/mapper stuff comes into play (it never made sense to me the first time I saw it... really. It's because of my past experience with HP-UX ;) The "lvcreate" command creates a symlink between /dev/vg01/lvol01 and the character device /dev/mapper/vg01-lvol01.

And, next, all we have to do is format that bad boy:

host # mkfs -t ext3 /dev/vg01/lvol01

Note that, above, you can set a reserved block count (based on percentage) using the -m flag. I would usually set this to "-m 5" so that 5% of the allocated space would be reserved for root (although you can set it as low as 1). This way, if someone jams your filesystem up to 100% and no one can get any work done, you can still log in as root and fix the issue. -v is another useful flag if you want to see verbose output. You can insert these two flags anywhere in the above command line except at the very end (after the logical device name) or in between the -t flag and its argument. Common sense, I know, but I feel compelled to over-explain ;)

Now, we'll make a mount point, or, really, just make a new directory off of the root filesystem (although you can do overlays if you wanna get creative ;)

host # mkdir /dir

and then we'll mount our logical volume up and check it out with df:

host # mount -t ext3 /dev/vg01/lvol01 /dir

host # df -h /dir
...
/dev/mapper/vg0-lvol0 246M   75.2M    158M 32%    /dir

Tomorrow, or maybe the day after that, we'll take a look at some more ways to manipulate your existing volumes, add more volumes, do mirroring and lots of other fun stuff that LVM allows you to do.

For now, we'll just thank our deity-of-choice that this all worked out ;)

Cheers,

, Mike

Creating RAID disk sets on Linux

Today's post is a somewhat-continuation of yesterday's post on RAID disk monitoring. We looked at monitoring then, and (the reason I call this a somewhat-continuation) now we're going to take a step back and look at setting up those RAID disk sets (a simple mirror set in this example)in the first place ;) Note that all the examples I'm using today are taken from RedHat Linux AS and can be run in bash or any native shell. Scripting some of this out is near impossible, but someday soon I may give it a go.

In any event, once we've covered all this ground, we'll definitely look at how a handy shell script or two can save a whole lot of time on these processes :) So, off we go:

The first thing you'll want to do is to create your partitions on each physical disk, using the fdisk utility. I personally prefer it from the command line, so if you use Disk Druid or any other GUI interface, it should be easy enough to translate this to that so I don't have to over-explain.

The first thing we'll do is to create the disk partitions and assign them to an appropriate RAID device. The steps should roughly be the same as these:

1. Invoke fdisk at that command line.
2. Select "n" to add a new partition (anything from a slice of disk for mirroring two disks, to an entire disk for higher level RAID and mega-storage).
3. Once added, select "t" to change the partition flag and select "Linux Raid Autodetect" (selection "fd" on my console, possibly different on yours).
4. Select "a" to add a bootable flag to the partition, if necessary.
5. Save the new disk layout, and quit, by selecting "w"
6. Repeat as necessary

It's important, since we're doing this from the command line down-and-dirty, that we make sure that all partitions on both disks are "exactly" the same size. Unless your secondary mirror disk is larger, in which case you just need to make sure that you only use partitions larger than the originals to mirror to (It only makes sense. You can't copy a larger filesystem to a smaller one without losing data - The RAID utilities will discourage you from doing so, anyway).

Now, you'll need to put filesystems on all of those partitions. Generally, you should be able to take care of this using mkfs:

host # mkfs -t ext3 /dev/hda1

In the bizarre event that this fails, or you're using an older or different OS than I am, this should also work:

host # mkfs -t ext2 -j /dev/hda1

Now we'll use set up our /etc/raidtab file, with the following (assuming the same for all slices):

host # raiddev /dev/md0
        raid-level      linear
        nr-raid-disks   2
        nr-spare-disks  0
        chunk-size      64
        persistent-superblock 1
        device          /dev/hda1
        raid-disk       0
        device          /dev/hdc1
        raid-disk       1
...

Then we'll initialize the RAID device, like so:

host # mkraid /dev/md0

Once again, per yesterday's post on RAID disk monitoring, we can check on the RAID device's progress by simply doing the following:

host # cat /proc/mdstat

and we'll know instantly if the RAID device is in a good or bad state, and how far along it is if it's still synching. You can use the above command to check status whenever you make a change to your RAID configuration for any valid RAID md devices.

If you ever want, or need, to remove any RAID partitions, you can do that like this (In this example, the hda1 slice is having issues on the md0 device):

host # raidsetfaulty /dev/md0 /dev/hda1
host # raidhotremove /dev/md0 /dev/hda1

Sometimes, as a failsafe (this depends on your situation and what device you're removing) you may want to re-install grub on your disk device, like so:

host # grub-install /dev/hda

And, finally for today, if you do end up having to remove a RAID mirror partition and it can be replaced inline (A whole disk in itself, for instance - possibly just a partition on a disk if bizarre filesystem corruption was the only issue and you've recreated the filesystem), do the following:

1. Add the new RAID devices as before (create, set filesystem type, assign same partition names if possible, etc) and check the status of the RAID groups. [2/2] means that the RAID group is ok, [2/1] means that the RAID group needs to be resynced (you'll note from yesterday that the RAID group we're working on is in a [2/1] state, with only one active mirror. If you noted that this could be, and should be, fixed - as well as the fact that this failure would drop that RAID partition because of this condition - you're sprinting ahead of my tutorial skills ;)

host # cat /proc/mdstat 
Personalities : [raid1] 
read_ahead 1024 sectors 
md0 : active raid1 hda1[0] 
      30716160 blocks [2/1] [U_] 
...
unused devices: <none>

2. Determine your RAID group and mirrors from /etc/raidtab:

host # cat /etc/raidtab 
...
raiddev             /dev/md0 
raid-level                  1 
nr-raid-disks               2 
chunk-size                  64k 
persistent-superblock       1 
nr-spare-disks              0 
    device          /dev/hda1 
    raid-disk     0 
    device          /dev/hdc1 
    raid-disk     1 
...

3. Now, you'll want to resync the RAID mirror sets that got goofed up when we had to replace the RAID parititions:

host # raidhotadd /dev/md0 /dev/hda1

4. Assuming you've lost more than 1 mirror, just repeat the procedure above as many times as necessary.

5. Lastly, assuming all went well, kick back and enjoy the view ;) - Note that Linux will normally queue resyncs and do them one at a time, so don't panic if you only see one out of 5 disk groups being worked on when you check this out!

host # cat /proc/mdstat  
...     
md3 : active raid1 hda1[2] hdc1[1] 
      30716160 blocks [2/1] [_U] 
      [===========>..........]  recovery = 45.9% (34790000/61432320) finish=98.7min speed=8452K/sec 
...      
unused devices: <none>

And that's it for today. In another near future post (probably tomorrow) we'll look at the fun involved with RAID disk failure ;)

Best Wishes,