Getting Started With CFEngine's cfagent.conf On Linux And Unix

Hey there,

It seems like a year since primary on-call duty came by and knocked me off my high horse. But, as they say, there's a reason we all fall down. It's so that we can learn how to deal with public humiliation, come to know the true meaning of pain and realize just how alone we are in this great big universe ;) It's either that or so we can learn to get back up again. The latter seems more like common sense, though. Does anyone reading this blog need to learn this lesson (after the first time)? Unless you prefer sitting, or spending your days sprawled-out face-down in the dirt, the odds are you've gotten back up before and you'll get back up again. Class dismissed ;)

Today, we're going to take a look at cfengine. It's a great (free) program that can be used for a wide variety of things (think Tripwire, Nagios, Jumpstart/Kickstart pre/post installation helper, etc). It's a decent product, that can be configured to run as simple or as complicated a setup as you prefer. So far, in my usage, I've come nowhere near the boundaries that cfengine can reach. I don't even know, for sure, what those boundaries are, since (as a side effect of not using it to its fullest potential) I've never fully tested its limitations.

That being what it is (and that is what it's being ;), today we're going to look at a skeleton of a configuration file for the cfagent program (part of cfengine, along with cfenvd, cfrun, etc). The cfagent program will allow you to hit the ground running and begin to see what cfengine can do for you. We're assuming that you've already installed cfengine, which you can download from cfengine.org or, if you're running a Linux distro, odds are there are packages available for your OS. That means that installation could just be a command away ("rpm -i cfengineXXXX.i386.rpm" or something like that). If you do need to build it from source, we'll be sure to address that in a future post on this blog. For now, we'll just assume it's already been installed, you can install it or you can find someone who'll install it for you. Whatever works for you. We'll wait ;)

For today we're going to go with a setup so basic, it may be of no use whatsoever. But, then, that's not the point :) The file we'll be using today (to control the actions of cfagent) is called cfagent.conf (Don't be fooled by the clever name ;) If we make a very simple configuration file, we could get away with this (running on a single server, just to keep things simple):

control:
        any::
                actionsequence = (
                        files
                )

files:
        any::
                /bin/ls=0555 owner=root group=bin action=fixall

And that's it. We're not going to get into specifics (like we'll only run this command on this set of servers, etc). The basic structure of our config file is pretty much this:

CONTROL:
  CLASSES::
       VARIABLE/FUNCTION = (
         LIST OF FUNCTION OR FACILITY VALUES
       )
       
FACILITY:
  CLASSES::
       OBJECT RULES

So, today, we'll just examine what those CAPITALIZED references mean. Obviously, they correspond to the contents of the cfagent.conf above :) It should be noted that all of the names we used in our sample configuration file are special to cfengine and don't require any specific definition by you. The only line where we really take any license is with the OBJECT RULES. More on that below!

CONTROL: This is a fundamental section in any cfagent.conf. Without this section, your cfagent run won't do anything! The colon at the end of the name "control:" indicates that the name is complete. This form can also be used to assign a value to a variable under a different context using a slightly different form (not to confuse... sorry; getting off track)

CLASSES:: This is sometimes known as "GROUPS::" although both will work at version 1.4 and above if I'm anywhere near correct. Hopefully you won't have to go back that far to get your setup working (They're on 3.x right now). The CLASSES:: variable ends with a double colon. Here we're not being as specific as we could be. We're setting the class "any" so that any defined class will match and cause cfagent to proceed deeper within the nest. "any" is pre-defined by cfagent and matches everything.

VARIABLE/FUNCTION In the CONTROL section, this generally defines the main flow of execution for the remainder of the configuration file. In this instance our variable is named "actionsequence." The "actionsequence" FUNCTION simply contains a list of the FACILITY's (pardon the improper spelling :) that we want to run through and in what order. If we had more than one, they would generally be listed one per line and executed from top to bottom. The FACILITY we chose to use is called "files"

FACILITY: As we noted just above, the FACILITY: "files" is the first FACILITY: executed by our CONTROL: section above. The "files" keyword (as also noted higher above) is special to cfagent and denotes files ;) That is to say that there are a lot of basic options you can choose from when you define your files and built-in actions that you can use simply by calling them within the configuration. The FACILITY: definition ends with a colon.

CLASSES:: This works the same as above. "any"-thing will match this!

OBJECT RULES This is where we set up what we want to do with our "files." We've only selected one rule, and it breaks down like this. In standard format (and this can get very complicated if you like) your rule definition would begin with the "name" of the file you wanted to act upon, followed by specific "attributes" of that file (if necessary/required) and the "action(s)" you intend to take upon it. So our RULE

/bin/ls=0555 owner=root group=bin action=fixall

could be picked apart like so:

file name = /bin/ls
file permissions = 0555 (Note that we took the liberty of jamming the file "name" and an "attribute" together here: /bin/ls=0555.  This could also be written as: /bin/ls mode=0555)
file "attribute" owner = root
file "attribute" group = bin
file "action" = fixall <-- Again, this action is built-in and instructs cfagent to fix any problems it finds with /bin/ls.  Those problems would be defined as "attributes" of the actual file that differ from the "attributes" set for it in our cfagent.conf rule.

So if we did a:

host # ls -l /bin/ls
-rwxr-xr-x  1 root  root  174432 Nov  8 14:21 /bin/ls

and then we ran:

host # cfagent -qv ( the -q option turns of host sleeping - also referred to as "splaying" (???) and the -v option is the expected "verbose")

it would find /bin/ls, note that the file mode was incorrect (0755 instead of 0555 and with the group root instead of bin) and fix that for us, so that our next ls would show the "corrected" file, like so:

host # ls -l /bin/ls
-r-xr-xr-x  1 root  bin  174432 Nov  8 14:21 /bin/ls

And I think that's enough for today. You never realize how much there is to explain about something until you try to explain it. But, as they say, that's why we learn to fall down... Or am I mixing my metaphors again? ;)

Cheers,

, Mike

---

Discover the Free Ebook that shows you how to make 100% commissions on ClickBank!

---

Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

---

Volume 3 Of The Linux/Unix SysAdmin Covert File Storage Method

Cheers,

Today's post is the third volume in our series on covert file storage. If you're interested in reading up on the 57th installment of this series, please feel free to click the link that's way behind the anchor text right here ;) Today, we're going to be looking at the tip discussed in volume 57, but with respect, specifically, to the ZFS filesystem. The meaning of those initials is included in the hyperlink, again, preceding the sentence that should land you at your anchor text. Of course, one connecting theme of these three posts is that everything is out of order; hence the signature sign-off in place of the usual greeting. We're trying really hard to be clever and, if we're lucky, we'll unintentionally create a perpetually-looping self-parody that unexpectedly creates infinite depths of recursion as we explore the meaning of the meaning of the meaning of... In the meantime, we'll plow straight ahead and bring you Volume 3 (which, of course, is the second entry) in our series of posts on hiding things in and under your Unix or Linux partitions and overlay-mounts (one's the same as the other but the other isn't necessarily the same as the one ;)

This post's twist is that we'll be specifically approaching hiding data in ZFS mountpoints. For the purposes of our demonstration and experimentation, we'll assume that the version of ZFS we're using is the one that roughly corresponds with the date of this post. If any feature mentioned didn't work a year ago, or you're reading this in the future and find yourself disgusted with the lack of an "obvious" and simple solution that didn't exist yet, we beg your pardon. This blog is written in the present (over and over and over again), edited in the very near relative-future and published before the aforementioned activities become distant recollections from the past ;)

Again, you can refer to our previous post on finding space hogs on multiple-overlay mountpoints if you want to look into that aspect a little bit deeper. Also, we'll be using the /usr/local overlay mount, mounted on top of /usr, for our models.

ZFS is actually quite a robust files system and is improving with each release. One really nice thing about it is that it's available for both commercial Solaris and OpenSolaris, so we don't have to exclude the open source community in this post like we have to with most of our Solaris posts. You're always welcome to read of course, but we prefer to only advertise or offer-up relevant content to our readers (Meaning that, since most of our traffic has been (and still is) the result of shameless self-promotion -- but, thank you Google for indexing us! -- we try not to forget to fail to "drop bombs" on purely Open Source forums when we write about proprietary, and unbelievably costly, Operating Systems).

Today's experiment will have us butting into a few problems along the way. But we'll still be able to get away with hiding our stuff. Like Rocky Balboa, we're about to be repeatedly punched in the face (and if you've ever seen that movie, you know that he gets hit around 70 or 80 times at full-force in the last fight alone) and yet still manage to lose the fight... okay; that was a bad allegory, not to mention a bad analogue. Think "Rocky II" instead. He get's his face smashed in some more, but we're pretty sure he wins at the end of that movie ;)

Step 1: Take stock of your surroundings. You're going to, first and foremost, ensure that the information you're hiding won't be compromised by the little things (like putting big things in small spaces or using big spaces with little room to store any things ;) Just like in the 57th method, we've got it pretty good, with regards to the amount of space available on /usr. The space on /usr is, ultimately, more important than the space available on /usr/local, since that's were our hidden files will actually reside and any changes in size might get noticed by tools like "du,""df" or the people who get paid to keep an eye on you (Sorry - that one was just way too easy to let it go ;)

Since our example output (aside from the ZFS-specific errors) will be the same, we're going to assume the example output from our post on the 57th method, which you can refer back to, before this post gets even longer than way-too-long!

Step 2: Pry that floorboard loose. This is where you still have to be nimble, and is one part of this operation that you'll want to execute as quickly as possible (putting everything back to normal would be the other thing you want to do quickly. Restoration of seeming-normality in as little time as possible is key).

First, we'll unmount the /usr/local filesystem, which leaves us with "df -k" (or "df -whatever" - however you like it) showing that /usr/local is now simply a directory on the /usr mountpoint. It's no longer a mountpoint, which is good. If you haven't been storing stuff here already, the directory should be empty.

Step 3: Throw all that stuff you're not supposed to have into the /usr/local directory and then nail the floorboards back down by remounting /usr/local.

NOTE: This is where ZFS bites you in the arse, if you've set it up using its defaults. Now you're in it deep because you're half-invested in a conspiracy to cover something up (we won't judge) and you have to decide if you want to put off until the future what you could figure out today, or just figure everything out right now.

An important distinction to be made at this point is that ZFS doesn't use the standard "mount" and "umount" commands that, say, UFS, does. It uses the very-similar "zfs mount" and "zfs umount" commands, instead. Assuming that your ZFS filesystem is a default ZFS filesystem and you've run the following commands (so far), you'll get the error at the end of the output below:

host # zfs umount rootdg/usr/local <-- NOTE HERE: The ZFS filesystems are part of a data pool or dataset, so the output looks slightly different in df, mount, du, etc.
host # mv /tmp/coworkers_salaries.db /usr/local/.
host # zfs mount rootdg/usr/local
cannot mount '/usr/local': directory is not empty <-- Also notice that the error message will usually use the "mountpoint" name as opposed to the "dataset" name. In this instance, the mountpoint /usr/local is actually mounted on rootdg/usr/local.

OUCH! Now the important thing to do is not to be like Rocky! Completely ignore Mickey while he calls you a tomato and goes on to explain some insanity about how he's not running some goddamn soup kitchen ;) You can escape from this situation - on the fly - quite easily if you know your command line options (or look them up really quickly):

host # zfs mount -O rootdg/usr/local
host #

Phew...

Step 4. Ensure everything looks good, and, if you have the time, consider one other option that might make your life easier later on (although it may draw unwanted attention. Each situation is unique. If your boss and/or co-workers are paranoid, they're probably out to get you and are all in on it ;) If you're reasonably confident you have a good excuse prepared and can set this up (if you'll permit us one more chug of spite from the "Rocky" well), Mickey also told you to keep hitting that bastard in the ribs, ya see? "Don't let him breathe." Your ZFS filesystem can be modified to keep your future covert operations more efficient. We'll look at that before we bid you adieu, since statistics show that the average human's attention span is ...something less than optimal. The specifics are fuzzy, but it's definitely not "above average" ;) That sentence couldn't have made more sense, by adhering to its own logic, if we paid it to...

The way to do this is, of course, to make your ZFS filesystems into legacy filesystems, rather than remembering to use "-O" at the command line every time (which won't work on a reboot or automated remount), and keep them that way! This other method doesn't rob you of all of the benefits of ZFS, but does make it so that commands like "zfs mount -a" won't auto-mount your ZFS mountpoint. In fact, "zfs mount" won't work for that mountpoint anymore, and you'll need to include it in /etc/vfstab, just like the UFS and VXFS, etc, filesystems if you want it to mount at boot, etc. So, for our rootdg/usr/local filesystem/mountpoint/dataset, we'll need to do the following (and you'll see how incredibly obvious this move can be ;)

host # zfs set mountpoint=legacy rootdg/usr/local
host # mount -F zfs rootdg/usr/local /usr/local

Actually, that isn't "so" bad, but, as we mentioned, if you want to automate this, you'll need to add a line to /etc/vfstab that looks odd and out of place:

rootdg/usr/local -  /usr/local     zfs  -  yes  -

This is particularly easy to spot because the logical device has a different format than most (/dev/dsk/c0t0d0s0, for example) and the "device to fsck" and "fsck pass" columns are both not used and need to be set to "-" rather than the standard "/dev/rdsk/c0t0d0s0" and "1" that might usually go there. Also, and this is the worst part, you have to put the name of the filesystem in there, so "zfs" is spelled out right in the middle of the entry. Nasty...

Step 5: Make sure things look relatively the same, so you don't get in trouble, and walk away. Don't look back. People who regret past decisions often look over their shoulders compulsively. Most people know this even if they don't "know" that they know this. Kind of in the same way some folks might make you feel uneasy or suspicious, although you couldn't say for exactly what reason.

And bang, zoom; you're all set, yet again. Unless you're incredibly unlucky, or tragically misinformed, your stuff should be there waiting for you when you go to get it back.

Well be back with our final self-referencing 437th Issue in this three part series. As it began, so shall it start. As above, so it goes. When God opens one door, He closes another :)

Howdy,

, Mike

---

Discover the Free Ebook that shows you how to make 100% commissions on ClickBank!

---

Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

---

Doing Search And Replace In Multiple Files With Unix and Linux Perl - Easy Or Hard?

Hey There,

For this weeks "Lazy Sunday" post we're going to take a look at the versatility of Perl on Linux or Unix. While I'm fairly certain there's little argument that it's the best tool for most "extraction" and "reporting" functions when used in complicated situations, it's always been more interesting to me because of the wide range of ways you can complete any sort of task.

Today we're going to take a look at two different ways to do search and replace in multiple files using strictly Perl. The first way will be obnoxiously long and the second way will be almost invisible ;)

For both situations, we'll assume that we have 15 files all in the same directory. We'll also assume that we're logged into our favorite flavour of Linux or Unix OS and, coincidentally, in the same directory as those files. All the files are text files and are humungous. And, finally, all of the files are stories where the main character's name is Waldo, they've never been published and the writer's had a change of heart and decided to name his main character Humphrey. It could happen ;)

1. The hard way (or, if you prefer, the long way):

We'll write a script to read in each file and scour it, line by line. For lines on which the name Waldo appears, we'll replace that with Humphrey. We're taking into account, also, that Waldo may be named more than once on any particular line and that the name Waldo may have accidentally been mistyped with a leading lowercase "w," which needs to be corrected. That script would look something like this:

#!/usr/bin/perl

#
# replace_waldo.pl - change Waldo to Humphrey in all files.
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#
 
@all_files = `ls -1d *`;
$search_term = "Waldo";
$replace_term = "Humphrey";

foreach $file (@all_files) {
 $got_one = 0;
 chomp($file);
 open(FILE, "<$file");
 @file=<FILE>;
 close(FILE);
 foreach $line (@file) {
  if ( $line =~ /Waldo/i ) {
   $line =~ s/Waldo/Humphrey/gi;
   $got_one = 1
  }
 }
 if ( $got_one ) {
  open(NEWFILE, ">$file.new");
  print NEWFILE @file;
  close(NEWFILE);
  rename("$file.new", "$file");
 }
}

2. The easy way (or, again, the short way):

Assuming the exact same convoluted situation, here's another way to do it (which we've covered in a bit more detail in this older post on using Perl like Sed or Awk):

From the command line we'll type:

host # perl -p -i -e 's/Waldo/Humphrey/gi' *

And we're done :)

Of course, the longer method is better suited for situations in which there are other extenuating circumstances. Or, perhaps, even more work to do. For the sort of limited situation we've laid out today, I will almost always go with the second method (Who wants pie? :)... Unless I have lots of time on my hands ;)

Cheers,

, Mike

Generating Only Unique Content From Two Somewhat Similar Files

Hey There,

I see this, in one variation or another, on the message boards from time to time. So, touching back on our earlier post regarding making our own diff to deal with directory permissions, today we're going to look at a script that performs another function not built in to the standard "diff" command on Linux or Unix, and fairly simply implemented in ksh or bash.

Our script today, takes the input of two files (generally text files) of different sizes; however, they can be of equal size. It doesn't seem to make a difference ;) The only restriction on the usage of the script we're presenting today is that, assuming both files are of unequal size, the smaller of the two files should be listed as the primary argument to the script (we'll call it "rdiff") and the larger file should be the secondary argument, like so:

host # ./rdiff smallfile largefile

and, of course, if they're of equal size:

host # ./rdiff file file

The output of the script will be a file named "Unique.out.smallfile.largefile" with "smallfile" and "largefile" being the values of the file names passed to the script on the command line.

Basically, our script uses sed and grep to determine if lines in the smaller file are duplicated in the larger file. If those lines are duplicated, they are then removed from the final output, so that your "unique" output file only includes lines that existed exclusively in the small and large file. All duplicate information is removed.

Enjoy and cheers,





This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#/bin/ksh

#
# rdiff - find unique content in two files
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

if [ $# -ne 2 ]
then
 echo "Usage: $0 SmallFile LargeFile"
 exit
fi

FileA=$1
FileB=$2

cp $FileB ${FileB}.old

for x in `<$FileA`
do
 grep ^${x}$ $FileB >/dev/null 2>&1
 if [ $? -eq 0 ]
 then
  echo $x >>tmpfile
 fi
done

for x in `<tmpfile`
do
 sed "/^$x$/d" $FileB >>newtmpfile
 mv newtmpfile Unique.out.${FileB}.${FileA}
done

rm tmpfile

, Mike

linux unix internet technology

Finalizing The Spec File And Building Your Own Linux RPM

Hey There,

We're finally ready to create our very own Linux RPM package! Following up on yesterday's post regarding beginning to write your spec file and the preceding initial software build, we're now going to look at the "build" section of the RPM spec file and compile our new RPM.

The "build" section of the spec file follows the "header section" we went over previously, and contains all that the "rpmbuild" command needs to know in order to create your RPM for you. I should note here that, on some systems, this command may be called "rpm-build" and, on yet others, the rpmbuild flags may simply be options to the rpm command itself (as with SUSE 8.x)

The declarations required in this section are:

%clean <--- Well, not really required, but keeps things nice and tidy ;)
%prep
%setup
%files
%build
%install

These sections are defined as follows (Note that for these declarations, the values are listed beginning on the line below, rather than on the same line, as we did in the "header section":

%clean - this deletes everything in the RPM_BUILD_ROOT, as defined by either the BuildRoot declaration in the "header section" (Which we didn't use for this build) or by the system default. In our case, this should be /usr/src/packages/BUILD. Cautious users may want to add a line of code in front of this, assuring that the RPM_BUILD_ROOT variable isn't set to /. This method is not absolutely necessary, either, so the incredibly cautious can just not call it and remove old garbage files, whenever they want to, manually. It can also cause issues when running concurrent builds if using the same RPM_BUILD_ROOT! Some folks prefer to run this method at the end of the spec file, or script out what it does automatically. It's a matter of preference :)

%prep - This simply indicates that we're moving on to the next section of our spec file. It is necessary, as everything beyond it will be considered part of the build process, rather than part of the RPM definition process, listed above it.

%setup - This part of the spec file indicates that "rpmbuild" should do whatever is necessary to prepare the source for building/compilation. Of course, we've done this already (to make sure we could build from source properly), but this declaration is necessary so that rpmbuild sets everything up for the following sections. Note that without any arguments, in our setup, it's just going to unzip and untar our source package in /usr/src/packages/SOURCES to the /usr/src/packages/BUILD directory.

%files - This is where we have to list all of the files that are going to be in our RPM, for reference when its installed and/or removed using the "rpm" command. In our previous post on building the software from source, we created a file called FILELIST (from the output of the find command). On the line following the %files declaritive, you can just read that file list into your spec file. If you elect to type each file name in, that's your decision, but you should include only one file per line (which can make your spec file pretty long depending on what you're installing. Given how many files most software packages contain, you'll grow sick and tired of this method soon enough ;)

%build - Here you'll provide all the information required to build the software. This can be culled from the notes you took while compiling the program initially.

%install - And finally, here you'll provide all the information required to install the program (Basically, the "make install" command and all arguments from the original compile of the software that you did earlier.

Our spec file will look something like this:

%clean
%prep
%setup
%files
/usr/local/PACKAGE-3.2-1/bin/list
/usr/local/PACKAGE-3.2.1/lib/liblist.so
...
%build
./configure --prefix=%{prefix}
make
make check
%install
make install
/bin/chown -Rv root:root %{prefix}

Before we actually build our RPM, we'll want to change the names of all the files listed in the %files section, that we got from our initial build. Assuming you wanted the RPM to install in /usr/local, you would want to use your favorite editor to replace all instances of "/usr/local/PACKAGE-3.2-1" in this section with "/usr/local" - this would change our %files section from:

%files
/usr/local/PACKAGE-3.2-1/bin/list
/usr/local/PACKAGE-3.2.1/lib/liblist.so
...

to

%files
/usr/local/bin/list
/usr/local/lib/liblist.so
...

Now our spec file is complete and we can create our RPM using this one simple command: rpmbuild. For our purposes, we'll run:

host # rpmbuild -ba PACKAGE.spec

then we'll kick back and watch the magic happen :) Note that I used the "-ba" option to build both source and binary RPM packages. This created the files /usr/src/packages/SRPMS/PACKAGE-3.2-1.src.rpm and /usr/src/packages/RPMS/x86_64/PACKAGE-3.2-1.x86_64.rpm <--- Note that this directory may be slightly different depending on what OS version of Linux you're running.

Now, you're all set to run "rpm -ivh PACKAGE-3.2-1.x86_64.rpm" and your software package will be installed, and can be manipulated by all the standard "rpm" facilities.

Of course there are tons of other options and additions to the spec file (%post, etc) and rpmbuild command, but this should be enough to get you started. For more specific information, check out the online man pages and chat boards - You'd be surprised at what you can do with this software if you use your imagination :)

Best wishes,

, Mike

linux unix internet technology

Finding User Accounts To Disable - Follow-Up Post

Hey there,

This is a follow up to a previous post where we're walking through the process of developing a decent script, from scratch; examining how to figure out what user accounts you could disable or delete (times vary, depending on your company's policy) on any particular box you have to administrate.

Today I've taken it a step farther. At this point the below script will check the modification time of all users listed in /etc/passwd and compare it against the current Unix time. If the mtime on the home directory is over 45 days and less than 90, we warn to disable (actual code not included) and if it's over 90 days, we warn to delete (same "no code" policy for that ;). If the time on the home directory is under 45 days, the account is consider active and OK.

Note that I found a few incorrect things and changed them, as I augmented this script; so this isn't just a bigger version of the original. First of all, it seems that the user's home directory is just as reliable as the existence of a history file. Also, it's much more likely to exist ;) Also, by verifying, I noticed that the mtime (Last Modification Time) was more reliable than the atime (Last Access Time) that we used before. That seems a bit counter-intuitive, but so far, it's proving out.

We've also added some extra functionality to remove users that we don't want to check. The first group are in a file we've called locked.txt, the second group are users without a valid shell and the final group are users who are in a particular group (like administrators who need accounts on all boxes, but might not login to any particular one for a long time).

Hope this script helps, at least, to point a few people in the right direction. This is a work in progress since it still contains several flaws, including no cross-referencing of possibly-available data that the system can provide with "last" and other such commands, as well as still having to deal with "touch."

Note that, on the lines ---> @locked_text = ###LOCKEDTEXT###;
and
open(LOCKEDTEXT, "###locked.txt");

the ### space-holders should be greater than and equal signs <>, in the first instance, and a < sign in the second, but the blog software interprets them as a unclosed and unallowed tags!

Have fun :)





This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/perl

#
# 2007 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#
# switching to mtime 9 - from atime - 8
#
# We don't need to check for errors on the command line since
# we won't be accepting any arguments
#

if ( -f "moreinfo") {
print "Removing old temporary file \"moreinfo\"\n";
unlink("moreinfo");
}

$today=time();

@no_shells=`egrep -i 'nologin|false' /etc/passwd|awk -F":" '{print \$1}`;
@in_unixtech=`grep 1621 /etc/passwd|awk -F":" '{print \$1}`;
@user_names=`awk -F":" '{print \$1}' /etc/passwd`;
open(LOCKEDTEXT, "###locked.txt");

@locked_text = ###LOCKEDTEXT###;
close(LOCKEDTEST);
print "\nRemoving LOCKED ACCOUNTS ";
foreach $locked_user (@user_names) {
print ".";
chomp($locked_user);
if ( grep /$locked_user/,@locked_text ) {
push(@tmp_user_names,$locked_user);
print "$locked_user ";
}
}
print "\n";
print "\nRemoving NO_SHELL ACCOUNTS ";
foreach $locked_user (@user_names) {
print ".";
chomp($locked_user);
if ( grep /$locked_user/,@no_shells ) {
push(@tmp_user_names,$locked_user);
print "$locked_user ";
}
}
print "\n";
print "\nRemoving UNIXTECH ACCOUNTS ";
foreach $locked_user (@user_names) {
print ".";
chomp($locked_user);
if ( grep /$locked_user/,@in_unixtech ) {
push(@tmp_user_names,$locked_user);
print "$locked_user ";
}
}
print "\n\n";
@total = @both = @unique_names = ();
%count = ();
foreach $possible_dupe (@user_names, @tmp_user_names) {
$count{$possible_dupe}++
}
foreach $possible_dupe (keys %count) {
push @total, $possible_dupe;
push @{ $count{$possible_dupe} > 1 ? \@both : \@unique_names }, $possible_dupe;
}
@user_names = @unique_names;
foreach $diruser (@user_names) {
$filer = `grep $diruser /etc/passwd|awk -F":" '{print \$6}'`;
push(@user_dirs,$filer);
}
print "FIRST RUN - Checking Home Directories...\n\n";
foreach $history_file (@user_dirs) {
chomp($history_file);
chomp($user_name = $user_names[$counter]);
print "${user_name}: ";
$counter++;
if ( -d "$history_file" ) {
@file_dates=stat("$history_file");
$file_date=$file_dates[9];
} else {
$file_date=0;
}
$difference=$today - $file_date;
if ( $difference >= 3888000 && $difference < 7776000 ) {
print "User needs to be disabled - ";
print "45 days old or older, but under 90\n";
# YOUR CUSTOM CODE HERE - "passwd -l" would probably suffice"
} elsif ( $difference >= 7776000 ) {
print "User needs to be deleted - ";
print "90 days old or older\n";
# YOUR CUSTOM CODE HERE -
} else {
print "OK\n";
}
}


, Mike

linux unix internet technology

Disabling and Removing Old User Accounts

Depending upon the size of your network, keeping track of user accounts on all of your Unix and Linux boxes can range from fairly simple to unbelievably hard to keep track of. I've noticed on the internet boards that lots of people are looking for the "one" way to semi-automate determining if user accounts need to be disabled, or if they're so old they need to be deleted.

I'm not a believer in the "one" solution theory. One of the things I love about Linux and Unix is, depending on how creative you're willing to be, there are probably 50 or more ways to do anything.

In any event I started looking at this problem, and noticed that almost everyone was using the "finger," "logins" or "last" commands to figure out how long it had been since someone logged in. In some extreme cases, I've even found examples where folks "pack" and "unpack" /var/adm/wtmpx with Perl to get the information. So, I naturally thought: Is it possible to do this without using the system's login-logging facilities?

The more I thought about this, the more I came to understand that both methods had their virtues and their faults. For instance, if /var/adm/wtmpx is missing, a lot of regular system accounting programs won't work. If you corrupt that file and remove /var/adm/lastlog (I'm using Solaris as my model - substitute file locations and names as necessary), "logins," "finger" and "last" don't report correctly unless they're reporting an error. The alternative approach also has its flaws. If, for instance, a user utilizes SSH to run a single command from a remote host using key-based passworless login, that shell won't be interactive and it won't update any shell history files!

Tackling this problem will take more than one post, so we'll start out with the core; treat this as a sort of tutorial on the phases that you go through from scratch to a decent script. Our assumption here will be that we have no access to any of the regular utilities like "last," "finger" and "logins." We'll use Perl to check out all users' .sh_history or .bash_history files (just covering sh, ksh and bash for now) and use the "stat" and "time" functions to determine how long it's been since the user last logged in.

Here's the core of the code, to get started. I've written it so it can be run on its own for now:





This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/perl

#
# 2007 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#
#
# We don't need to check for errors on the command line since
# we won't be accepting any arguments
#

$today=time();

@user_dirs=`awk -F":" '{print \$6}' /etc/passwd`;
foreach $history_file (@user_dirs) {
chomp($history_file);
print "${history_file}: ";
if ( -f "$history_file/.sh_history" ) {
@file_dates=stat("$history_file/.sh_history");
$sh_file_date=$file_dates[8];
} else {
$sh_file_date=0;
}
if ( -f "$history_file/.bash_history" ) {
@file_dates=stat("$history_file/.sh_history");
$bash_file_date=$file_dates[8];
} else {
$bash_file_date=0;
}
if ( $sh_file_date > $bash_file_date ) {
$file_date = $sh_file_date;
} elsif ( $bash_file_date > $sh_file_date ) {
$file_date = $bash_file_date;
} else {
print "Cannot find .bash_history or .sh_history for $history_file. Moving on!\n";
next;
}
$difference=$today - $file_date;
if ( $difference >= 3888000 && $difference < 7776000 ) {
print "User needs to be disabled- ";
print "45 days old or older, but under 90\n";
# YOUR CUSTOM CODE HERE - "passwd -l" would probably suffice"
} elsif ( $difference >= 7776000 ) {
print "User needs to be deleted - ";
print "90 days old or older\n";
# YOUR CUSTOM CODE HERE -
} else {
print "OK\n";
}
}

Within the next post or two, we'll follow up on this, so that we can add functionality to check system files, using system utilities, to verify that our initial information is correct. It's possible that someone just "touch"ed a user's history file and that access time information isn't necessarily correct. Also, as we noted above, folks can be using the host remotely, in a non-interactive shell, and never alter the access time of their history file.

For the time being, this code should be a good start, and something to think about and improve upon. Until next time :)

, Mike

linux unix internet technology

Using find and xargs to locate Windows Files

A lot of times, when you're asked to find something on a machine, and you only have a moderate idea of what you're specifically looking for, you'll use the obvious command: find. find is a great command to use because you can use wildcards in your expression argument. So, if you know that you're looking for something like "theWordiestScriptEver," and you have no idea where it's located on your box, you could find it by typing just this:

find / -iname "*word*" -print

This will find every file on the system (even on non-local mounts if you have them set up) and only print the results for files with the word "word" in the name. Note that the "-iname" option matches without regards to case, so h and H both match. This option isn't available in all versions of find. If you don't have this option available to you, you'll get an error when you run the above line (just use "-name" instead). The standard Solaris find does not do "case insensitive" pattern matching, so your best bet is to find the smallest substring that you're sure of the case on, or use another attribute to search for the file (like -user for the userid or -atime for the last access time). Alternatively, you could spend hours stringing together a bunch of "or" conditions for every conceivable combination of upper and lower case letters in your expression.

Now suppose you needed to perform an action on a file you found. You could use find,s built-in exec function, like so:

find / -iname "*word*" -print -exec grep spider {} \;

This will perform the command "grep spider" on all files that match the expression. Which brings us around to the next predicament. What do you do if you have to try and find something simple, have no idea where it is on your box "and" that box hosts file systems that Windows users are allowed to write files to. The above example should work just fine on those. My own advice is, if you can get away with just using find, do so, since it handles all of the rogue characters, tabs and spaces in Windows files on its own.

Now, if you have to do something much more complicated (or convoluted), you'll want to pipe to a program like xargs, which is where all those funny Windows file names and characters (some of which are special to your shell) start to cause issues. Again, this would return ok:

# find . -name "*word*" -print
./word - file's
./word file
./word & file's
./word file's

But this will become an issue if you pipe it to xargs, as shown below:

# find . -name "*word*" -print|xargs ls
xargs: Missing quote: files

Ouch! xargs doesn't deal with those spaces, tabs and special characters very well. You can fix the space/tab problem very simply by using xarg's "named variable" option. Normally, xargs acts on the input it receives (thus: "xargs ls," above, is processing ls on each file name find sends it), but you can alter how it deals with that data in a simple way (at least as far as the spacing issue is concerned). Example below:

# find . -name "*file" -print|xargs -ivar ls "var"
./word file
# find . -name "*word*" -print|xargs -ivar ls "var"
xargs: Missing quote: ./word - files

But, in the second invocation above, you see that it still can't handle the "shell special" characters, like "'" or """ <--- Double quote - so it's time to step it up. I prefer to just sanitize everything that's not kosher, even though I know I don't technically have to avoid the ---> \ / : * ? "< > <--- characters, since Windows won't allow them as parts of file names. It seems easier just to react on anything that isn't a letter or number and pass it along with enough escapes (back slashes) so that xargs can parse it correctly, and get you back good information. Here's how to do that; using sed (and a little grep, to keep it neat), also:

# find . -name "*word*" 2>&1|grep -iv denied|sed "s/\([^A-Za-z0-9]\)/\\\\\1/g"|xargs -ivar ls "var"
./word - file's
./word file
./word & file's
./word file's

And now you can use find, combined with xargs, on all the files you have permission to see, no matter what goofy characters are in them :)

, Mike

linux unix internet technology