Cheers,
Today's post is the third volume in our series on covert file storage. If you're interested in reading up on the 57th installment of this series, please feel free to click the link that's way behind the anchor text right here ;) Today, we're going to be looking at the tip discussed in volume 57, but with respect, specifically, to the ZFS filesystem. The meaning of those initials is included in the hyperlink, again, preceding the sentence that should land you at your anchor text. Of course, one connecting theme of these three posts is that everything is out of order; hence the signature sign-off in place of the usual greeting. We're trying really hard to be clever and, if we're lucky, we'll unintentionally create a perpetually-looping self-parody that unexpectedly creates infinite depths of recursion as we explore the meaning of the meaning of the meaning of... In the meantime, we'll plow straight ahead and bring you Volume 3 (which, of course, is the second entry) in our series of posts on hiding things in and under your Unix or Linux partitions and overlay-mounts (one's the same as the other but the other isn't necessarily the same as the one ;)
This post's twist is that we'll be specifically approaching hiding data in ZFS mountpoints. For the purposes of our demonstration and experimentation, we'll assume that the version of ZFS we're using is the one that roughly corresponds with the date of this post. If any feature mentioned didn't work a year ago, or you're reading this in the future and find yourself disgusted with the lack of an "obvious" and simple solution that didn't exist yet, we beg your pardon. This blog is written in the present (over and over and over again), edited in the very near relative-future and published before the aforementioned activities become distant recollections from the past ;)
Again, you can refer to our previous post on finding space hogs on multiple-overlay mountpoints if you want to look into that aspect a little bit deeper. Also, we'll be using the /usr/local overlay mount, mounted on top of /usr, for our models.
ZFS is actually quite a robust files system and is improving with each release. One really nice thing about it is that it's available for both commercial Solaris and OpenSolaris, so we don't have to exclude the open source community in this post like we have to with most of our Solaris posts. You're always welcome to read of course, but we prefer to only advertise or offer-up relevant content to our readers (Meaning that, since most of our traffic has been (and still is) the result of shameless self-promotion -- but, thank you Google for indexing us! -- we try not to forget to fail to "drop bombs" on purely Open Source forums when we write about proprietary, and unbelievably costly, Operating Systems).
Today's experiment will have us butting into a few problems along the way. But we'll still be able to get away with hiding our stuff. Like Rocky Balboa, we're about to be repeatedly punched in the face (and if you've ever seen that movie, you know that he gets hit around 70 or 80 times at full-force in the last fight alone) and yet still manage to lose the fight... okay; that was a bad allegory, not to mention a bad analogue. Think "Rocky II" instead. He get's his face smashed in some more, but we're pretty sure he wins at the end of that movie ;)
Step 1: Take stock of your surroundings. You're going to, first and foremost, ensure that the information you're hiding won't be compromised by the little things (like putting big things in small spaces or using big spaces with little room to store any things ;) Just like in the 57th method, we've got it pretty good, with regards to the amount of space available on /usr. The space on /usr is, ultimately, more important than the space available on /usr/local, since that's were our hidden files will actually reside and any changes in size might get noticed by tools like "du,""df" or the people who get paid to keep an eye on you (Sorry - that one was just way too easy to let it go ;)
Since our example output (aside from the ZFS-specific errors) will be the same, we're going to assume the example output from our post on the 57th method, which you can refer back to, before this post gets even longer than way-too-long!
Step 2: Pry that floorboard loose. This is where you still have to be nimble, and is one part of this operation that you'll want to execute as quickly as possible (putting everything back to normal would be the other thing you want to do quickly. Restoration of seeming-normality in as little time as possible is key).
First, we'll unmount the /usr/local filesystem, which leaves us with "df -k" (or "df -whatever" - however you like it) showing that /usr/local is now simply a directory on the /usr mountpoint. It's no longer a mountpoint, which is good. If you haven't been storing stuff here already, the directory should be empty.
Step 3: Throw all that stuff you're not supposed to have into the /usr/local directory and then nail the floorboards back down by remounting /usr/local.
NOTE: This is where ZFS bites you in the arse, if you've set it up using its defaults. Now you're in it deep because you're half-invested in a conspiracy to cover something up (we won't judge) and you have to decide if you want to put off until the future what you could figure out today, or just figure everything out right now.
An important distinction to be made at this point is that ZFS doesn't use the standard "mount" and "umount" commands that, say, UFS, does. It uses the very-similar "zfs mount" and "zfs umount" commands, instead. Assuming that your ZFS filesystem is a default ZFS filesystem and you've run the following commands (so far), you'll get the error at the end of the output below:
host # zfs umount rootdg/usr/local <-- NOTE HERE: The ZFS filesystems are part of a data pool or dataset, so the output looks slightly different in df, mount, du, etc.
host # mv /tmp/coworkers_salaries.db /usr/local/.
host # zfs mount rootdg/usr/local
cannot mount '/usr/local': directory is not empty <-- Also notice that the error message will usually use the "mountpoint" name as opposed to the "dataset" name. In this instance, the mountpoint /usr/local is actually mounted on rootdg/usr/local.
OUCH! Now the important thing to do is not to be like Rocky! Completely ignore Mickey while he calls you a tomato and goes on to explain some insanity about how he's not running some goddamn soup kitchen ;) You can escape from this situation - on the fly - quite easily if you know your command line options (or look them up really quickly):
host # zfs mount -O rootdg/usr/local
host #
Phew...
Step 4. Ensure everything looks good, and, if you have the time, consider one other option that might make your life easier later on (although it may draw unwanted attention. Each situation is unique. If your boss and/or co-workers are paranoid, they're probably out to get you and are all in on it ;) If you're reasonably confident you have a good excuse prepared and can set this up (if you'll permit us one more chug of spite from the "Rocky" well), Mickey also told you to keep hitting that bastard in the ribs, ya see? "Don't let him breathe." Your ZFS filesystem can be modified to keep your future covert operations more efficient. We'll look at that before we bid you adieu, since statistics show that the average human's attention span is ...something less than optimal. The specifics are fuzzy, but it's definitely not "above average" ;) That sentence couldn't have made more sense, by adhering to its own logic, if we paid it to...
The way to do this is, of course, to make your ZFS filesystems into legacy filesystems, rather than remembering to use "-O" at the command line every time (which won't work on a reboot or automated remount), and keep them that way! This other method doesn't rob you of all of the benefits of ZFS, but does make it so that commands like "zfs mount -a" won't auto-mount your ZFS mountpoint. In fact, "zfs mount" won't work for that mountpoint anymore, and you'll need to include it in /etc/vfstab, just like the UFS and VXFS, etc, filesystems if you want it to mount at boot, etc. So, for our rootdg/usr/local filesystem/mountpoint/dataset, we'll need to do the following (and you'll see how incredibly obvious this move can be ;)
host # zfs set mountpoint=legacy rootdg/usr/local
host # mount -F zfs rootdg/usr/local /usr/local
Actually, that isn't "so" bad, but, as we mentioned, if you want to automate this, you'll need to add a line to /etc/vfstab that looks odd and out of place:rootdg/usr/local - /usr/local zfs - yes -
This is particularly easy to spot because the logical device has a different format than most (/dev/dsk/c0t0d0s0, for example) and the "device to fsck" and "fsck pass" columns are both not used and need to be set to "-" rather than the standard "/dev/rdsk/c0t0d0s0" and "1" that might usually go there. Also, and this is the worst part, you have to put the name of the filesystem in there, so "zfs" is spelled out right in the middle of the entry. Nasty...
Step 5: Make sure things look relatively the same, so you don't get in trouble, and walk away. Don't look back. People who regret past decisions often look over their shoulders compulsively. Most people know this even if they don't "know" that they know this. Kind of in the same way some folks might make you feel uneasy or suspicious, although you couldn't say for exactly what reason.
And bang, zoom; you're all set, yet again. Unless you're incredibly unlucky, or tragically misinformed, your stuff should be there waiting for you when you go to get it back.
Well be back with our final self-referencing 437th Issue in this three part series. As it began, so shall it start. As above, so it goes. When God opens one door, He closes another :)
Howdy,
, Mike
Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.
