Showing posts with label time. Show all posts
Showing posts with label time. Show all posts

Tuesday, March 3, 2009

Simple Code To Implement C's stat() function On Linux And Unix

Hey There,

Today's post loosely ties back to our post last week on Perl stat() basics for Linux and Unix, although this post is not a continuation of that one. Just trying to keep it confusing ;)

I've often been bashed for my C programming skills (the "bash" in that sentence wasn't meant to be a pun), and, although I've come to expect the vicious backlash, I still like to come back to C every once in a while to look at some of the things you can do with it and to keep me awake ;) That being said, any competent C programmer out there who finds fault with this post's attached C-code is, most probably, correct. This could be written better, have more robust error-checking, implement measures to prevent buffer over-runs, under-runs, and a thousand other exploits. That being said; it works, which is miracle enough for me :) Feel free to modify it in any way, shape or form you prefer. Everything I put on this blog is considered at-your-own-risk material. There's not all that much to worry about, though, if you just use this code for yourself to test and play with. Just secure it with simple ownership and file permission settings and delete the binary when you're done.

It's been a long time since I've had to write this, but typing it will be faster than trying to find another post with C code in it ;) If you want to compile the attached code, you can do the following (should work with most any C compiler, I'm just using gcc as an example since that's what I have installed on my machine):

host # gcc -o filestat filestat.c

and the program is ready to run. That part's pretty simple, too. You just execute it and follow it with one or more file names (they can be "any" type of file: file, directory, link, socket, etc). If you want to see the usage information, just run it without any arguments, like so:

host # ./filestat
Usage: ./filestat file1 file2 fileN...


The most interesting part for me was figuring out how to decipher the "file mode" information. If you check your system's "stat" manpage, you'll see that there are actually a good deal of macros already written-in that make printing out plain English file types and modes much simpler, but I decided to keep it cryptic. I take my modes like I take my permissions; in octal ;)

Here's a sample run of the program - just using one file to make it simple (it should be noted, also, that you can only run this command against file types that you have permission to read. If you run this as your regular user on a file owned by root with 700 permissions, you'll get the "File or Directory Not Found!" error, which is, technically, incorrect. My God, am I a lazy C programmer. No wonder I get beat up so bad every time I goof around with it on here ;)

NOTE: This program interacts with the shell you're using, so if, for instance, you can access a file named "myfile" by typing "my*", the same globbing rules will be taken care of, applied to the filename and expanded by the shell before this program processes it!

host # ./filestat tct-1.18
****************************************
Checking tct-1.18
****************************************
tct-1.18 stat values are:
****************************************
File Mode: 40700
Inode Number: 17540
Device ID: 35651599
Raw Device ID: 0
Number Of Links: 12
User ID: 0
Group ID: 0
Bytes Size: 1024
Access Time: Mon, March 02, 2009 - 14:15:31
Modification Time: Mon, April 09, 2007 - 08:36:09
Status Change Time: Mon, March 02, 2009 - 15:47:38
Block Size: 8192
Number Of Blocks: 2
File System Type: ufs
****************************************


Most of the above is self-explanatory. The few things to note (if you just "really" don't want to read the manpage ;) are that the "Raw Device ID" is almost always zero unless you're dealing with a disk, or other file type, that supports raw access. From my own testing, you can definitely get back a "Raw Device ID" from any disk (/dev/rdsk/c0t0d0s0, /dev/dsk/c0t0d0s0, /dev/hda1, etc). The "Bytes Size" is simply the Size in Bytes. I left it the way it is because it reminds me that I need to eat less ;)

Lastly, you may have noted that the "File Mode" (listed first in the output, since I just went through the "stat" struct and used each value in order) seems a bit off. The last 3 digits are probably familiar to most Linux and Unix users, as they represent the file permissions. By the same virtue, the first 4 (including those original 3), probably seem familiar as well, since that 4th-left bit is used in octal permissions settings to represent setuid, setgid, and sticky-bit, etc, special permissions. So, for the above "File Mode" of 40700, you know that the file permissions are "0700" (no setuid, setgid or sticky-bit - read/write/execute for the user and no permissions for the group and world).

The only question, once you've got the right side of the "File Mode" down is, what do the variable number of numerals to the left of those stand for? The answer is pretty simple and (as I noted above) there are standard macros that you can use in your C code to test the mode and print it out so that it's more human-readable. Basically, they determine the type of file you're interrogating. Since our one number to the left of the four permission bits is "4," we know that the file we're accessing is a directory. You can see this in the table below (shamelessly lifted from the system header file - /usr/include/sys/stat.h ;) <-- This is just a subset of all available macros. There's a lot more information in that header file than I want to jam into this post... It's getting long enough now. Especially bad if you're an "F" reader ;)

As one last interesting side-note, the "File Mode" for the gzipped tarball, that was extracted to create this directory, is "100700" which gives us the same permissions, but a "10" on the left-hand "file type" side. You can also see this showing up below as a "regular" file.

#define S_IFMT  0170000  /* type of file mask */
#define S_IFIFO 0010000 /* named pipe (fifo) */
#define S_IFCHR 0020000 /* character special */
#define S_IFDIR 0040000 /* directory */
#define S_IFBLK 0060000 /* block special */
#define S_IFREG 0100000 /* regular */
#define S_IFDB 0110000 /* record access file */
#define S_IFLNK 0120000 /* symbolic link */
#define S_IFSOCK 0140000 /* socket */
#define S_IFWHT 0160000 /* whiteout */
#define S_ISVTX 0001000 /* save swapped text even after use */
#define S_IRWXU 00700 /* read, write, execute: owner */
#define S_IRUSR 00400 /* read permission: owner */
#define S_IWUSR 00200 /* write permission: owner */
#define S_IXUSR 00100 /* execute permission: owner */
#define S_IRWXG 00070 /* read, write, execute: group */
#define S_IRGRP 00040 /* read permission: group */
#define S_IWGRP 00020 /* write permission: group */
#define S_IXGRP 00010 /* execute permission: group */
#define S_IRWXO 00007 /* read, write, execute: other */
#define S_IROTH 00004 /* read permission: other */
#define S_IWOTH 00002 /* write permission: other */
#define S_IXOTH 00001 /* execute permission: other */


Hope you enjoy the attached C code and get some use out of it (or your own Frankenstein'ed version of it ;)

Cheers,

Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <dirent.h>
#include <pwd.h>
#include <grp.h>
#include <time.h>
#include <locale.h>
#include <langinfo.h>
#include <fcntl.h>

/*********************************************
filestat.c - Would you like fries with that?
2009 - Mike Golvach - eggi@comcast.net
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
*********************************************/
int main (int argc, char *argv[])
{
if (argc <= 1) {
printf("Usage: %s file1 file2 fileN...\n", argv[0]);
exit(1);
}
struct stat dirfilebuf;
int arguments, dirfilestate;
const char *dirfilename;
char *nicetime, *errormessage, *divider;

dirfilename=(char*)(malloc(sizeof(char)));
nicetime=(char*)(malloc(sizeof(char)));
errormessage=(char*)(malloc(sizeof(char)));
divider=(char*)(malloc(sizeof(char)));

errormessage="File or Directory Not Found!";
divider="****************************************";

for (arguments = 1; arguments < argc; arguments++) {
dirfilename = argv[arguments];
printf("%s\nChecking %s\n%s\n", divider,dirfilename,divider);
dirfilestate = stat(dirfilename, &dirfilebuf);
if ( dirfilestate == -1 ) {
printf("%s - Quitting!\n", errormessage);
exit(1);
} else{
printf("%s stat values are:\n%s\n", dirfilename,divider);
printf("File Mode: %lo\n", dirfilebuf.st_mode);
printf("Inode Number: %-10d\n", dirfilebuf.st_ino);
printf("Device ID: %d\n", dirfilebuf.st_dev);
printf("Raw Device ID: %d\n", dirfilebuf.st_rdev);
printf("Number Of Links: %d\n", dirfilebuf.st_nlink);
printf("User ID: %-8d\n", dirfilebuf.st_uid);
printf("Group ID: %-8d\n", dirfilebuf.st_gid);
printf("Bytes Size: %jd\n", (intmax_t)dirfilebuf.st_size);
strftime(nicetime,34,"%a, %B %d, %Y - %H:%M:%S", localtime(&dirfilebuf.st_atime));
printf("Access Time: %s\n", nicetime);
strftime(nicetime,34,"%a, %B %d, %Y - %H:%M:%S", localtime(&dirfilebuf.st_mtime));
printf("Modification Time: %s\n", nicetime);
strftime(nicetime,34,"%a, %B %d, %Y - %H:%M:%S", localtime(&dirfilebuf.st_ctime));
printf("Status Change Time: %s\n", nicetime);
printf("Block Size: %d\n", dirfilebuf.st_blksize);
printf("Number Of Blocks: %d\n", dirfilebuf.st_blocks);
printf("File System Type: %s\n", dirfilebuf.st_fstype);
printf("%s\n", divider);
}
}
}


, Mike




Discover the Free Ebook that shows you how to make 100% commissions on ClickBank!



Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

Tuesday, May 27, 2008

Using Perl On Linux To Do Mass Synchronization Of File Time Stamps

Hey There,

Today we're going to take a look at another quick and simple Perl command line execution statement that you can use to save yourself lots of time ( If it really does equal money, this post is going to be a lot more valuable than I originally thought ;)

In a previous post, we looked at using Perl to figure out how old all our files really are. This post is a little twist on that; and it's a lot less complicated. It will run on whatever version of Perl you have (unless it's ancient ;) and produced equal results on all tested Linux and Unix flavours I could get my hands on.

Today we're going to be using Perl's stat and utime functions (similar to the way we did in our previous Perl post), but instead of using them do determine the ages of all of our files, we're going to use them make all of our files conform to the specific time and date of one representative file. This trick can come in really handy if you wanted to make a system backup consisting only of files that, let's say, are a certain number of days old, and only some of the files that you know you need to backup are slightly older or newer.

The trick is very simple. All you need to do is find one specific file that has a time stamp you want to mimic, and then apply its time stamp attributes to the group of files that you want to modify the time stamp on. So (for a simple example) if you had a directory consisting of 5 files, like this:

host # ls
file1 file2 file3 file4 file5


and you needed them all to share the same time stamp as the file "file1," I'd first suggest that you take some form of online-backup, using tar or cpio (assuming that you may need to reverse the process later):

host # tar cpf /tmp/backout.tar file1 file2 file3 file4 file5

or

host # tar cpf /tmp/backout.tar f* <--- I rarely glob this globally, but, for our example, all the files we want to back up start with "f" and there aren't any others. Worst case, on a tar backup, you can just write over it if you accidentally tar-copy something you didn't want to.

Now, let's take one more look at those files. We'll use "ls -l" to show the time stamp at this time:

host # ls -l
total 4
-rw-r--r-- 1 eggi newpeople 90 May 13 22:27 file1
-rw-r--r-- 1 eggi newpeople 0 May 25 14:25 file2
-rw-r--r-- 1 eggi newpeople 0 May 25 14:25 file3
-rw-r--r-- 1 eggi newpeople 0 May 25 14:25 file4
-rw-r--r-- 1 eggi newpeople 0 May 25 14:25 file5


Now, well make them all have the same time stamp.

We need files 2 through 5 to match the time stamp of file1, as this is how the system is going to determine what to back up later in the day (fill in your own hypothetical situation here ;)

And here's all we have to do to make all the files have the same time stamp as file1 :)

host # perl -e '$x=utime ((stat($ARGV[0]))[8,9], @ARGV);print $x' file1 file[2345]

And we should be all set!

host # ls -l
total 4
-rw-r--r-- 1 eggi newpeople 90 May 13 22:27 file1
-rw-r--r-- 1 eggi newpeople 0 May 13 22:27 file2
-rw-r--r-- 1 eggi newpeople 0 May 13 22:27 file3
-rw-r--r-- 1 eggi newpeople 0 May 13 22:27 file4
-rw-r--r-- 1 eggi newpeople 0 May 13 22:27 file5


Now all the files have the same time stamp! You'll note, in the command above, that we only modifed the atime and mtime of the files. This is sufficient because Perl's utime function automatically changes the ctime (or inode change time) to the current date and time. You can include the utime in this Perl command line execution statement (no error will be generated), but it's not necessary.

If you ever do need to copy the ctime, just add the 10th index of the array returned by stat to your command line, like this:

host # perl -e '$x=utime ((stat($ARGV[0]))[8,9,10], @ARGV);print $x' file1 file[2345]

And, of course, you can use this same process on directories as well, without a direct correlation (e.g. copying file timestamps to directories, vice versa and any combination between file types that support time stamps will work :)

Here's hoping this helps you out (at least, a little ;) This simple command can be a very convenient way for you to make massive changes by applying the time stamp of any one file to as many other files as you like. And, no matter how many files you need to do this for, as long as you can wait a few moments for ingenuity to strike, the command line shouldn't get too much longer :)

Cheers,

, Mike

Thursday, November 22, 2007

A Quick Guide To Setting Up SSH Keys Network Wide

Today, in honor of the spirit of the day, I thought I'd write a little something for which you might, eventually, give many thanks :)

As we mentioned in yesterday's post, once you have SSH keys set up network-wide, so that you can passwordlessly login as yourself on all the machines you administrate, our "scmd" script can be a blessing; especially if you're ever stuck with the seemingly impossible task of retrieving some obscure bit of information off of every single box!

I won't lie to you; this is the worst part of the process. In order to get your keys initially set up, you'll have to put forth a good deal of effort (Your misery will be commensurate with the size of your network). But, once you're done, you'll be on easy street.

For our purposes, we're going to assume that you're using Solaris' (now) standard implementation of SSH (Basically OpenSSH) and craft our instructions using those assumptions. Depending on what kind of SSH client/server you're using, there may be differences in the names of certain files (e.g. authorized_keys2 might be authorized_keys -- The man pages are your friends :) We're also assuming that you actually already have user accounts on all the machines you're required to do work on.

So, to get the worst part over with, simply do the following (cut it up into chunks if it becomes frustrating. Depending on the size of your network, this is quite possible):

1. Create a file with the names or IP addresses of all of the machines you need to work on. For our purposes, format that file with one hostname/IP per line. We'll assume it's named "HostFile"

2. If you haven't done so already, generate your personal ssh-keys on the one machine that you're going to use as your central hub for administrating all the others, like so:

ssh-keygen -t dsa
(Answer yes to the default file locations -- /your/home/directory/.ssh/id_dsa and /your/home/directory/.ssh/id_dsa.pub)
(Also, just hit return both times you're prompted to enter a password. While this is just slightly less secure than entering the double-confirm password, it will completely defeat the purpose of setting this easy-administration up)

2. Run the following from your home directory (This is the first really long and trying part, as you'll have to interact manually):

for x in `cat HostFile`;do ssh -n $x "mkdir -m 700 .ssh";done
(Actuall SSH command may vary depending on your distro. I'm using the "-n" flag to avoid having SSH break out of the "for loop")

Now, if we're starting from scratch, you'll need to answer "yes" for each machine's initial prompt (it will ask you if you trust the host and want to accept and save the key) and enter your password.

3. When that's finally over, cd into your home directory's .ssh directory and run the following on the command line ("HostFile" is still in your home directory, so we're using a relative path to it here):

for x in `cat ../HostFile`;do scp id_dsa.pub $x:/your/home/dir/.ssh/authorized_keys2
(Again, you'll need to enter your password for every machine. Note that if you run a variety of servers, or build standards have changed over time so that your home directory isn't always in the exact same place on every box, you can use "~.ssh/authorized_keys2" as a substitute for "/your/home/dir/.ssh/authorized_keys2)"

4. And, now, you're all done, except to make sure that it actually worked! My favorite way to do this is to just run the same command again. Overwriting your authorized_keys2 file on all the remote hosts won't hurt, and - this time - you should be just kicking back, watching scp's progress meters fly by as you try not to fall asleep ;) Again the "~.ssh/authorized_keys2" substitution is perfectly valid and useful:

for x in `cat ../HostFile`;do scp id_dsa.pub $x:/your/home/dir/.ssh/authorized_keys2

For all the machines that still ask you for your password, take note of those hosts and troubleshoot as necessary. For the most part, you shouldn't have to worry about running into too many of those.

And from here on out, you can use the "scmd" command we put in our pre-Thanksgiving post, or your own custom scripts, to effortlessly run any command (as yourself, of course) on all your machines, by typing a single line on your hub computer.

Happy Thanksgiving :)

, Mike





Wednesday, November 21, 2007

SSH Command Runner To Help With Those Big Tiresome Tasks!

You may recall that we looked at one of the core components of this script in an earlier post, located here.

Today, in preparation for Thanksgiving, I'm putting up a little script that can help you run almost any command line you can dream up, and save the output for you in a relatively nicely formatted report. I'm calling it "scmd" (short for SSH command) but you can call it whatever you want :)

Now, of course, this script comes with a pre-requisite. For instance, it won't really help you save time if you don't have ssh-key-based passwordless logins set up for yourself on all the servers you manage. For Thanksgiving day, we'll go over how to easily set yourself up with those.

For today, assuming you've got your passwordless logins all set, feel free to modify this script as you wish, and use it to kick back and relax when the boss asks you to verify the version of Veritas Foundation Suite on all 300 servers in your farm. Otherwise, until tomorrow, enjoy only having to type in your password over and over and over again (at least you won't have to keep re-typing the command ;)

The script is posted below, for your enjoyment. Note that the "hostfile" is set to be of the format: hostname colon ip_address (e.g. host.xyz.com : 192.168.0.1). The hostname is the only necessary component of each line and all lines that begin with pound symbols (#'s) will be skipped. One host per line, please.

Again, tomorrow, we'll go over how to easily set up ssh keys for yourself network wide, and feel free to modify this little script to make your worklife as effortless as possible:


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/ksh

#
# 2007 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

trap 'rm tmpfile.$pid;echo "Caught Signal. Cleaning Up And Quitting...";exit 3"' 1 2 3 9 15

pid=$$

if [ $# -ne 1 ]
then
echo "Usage: $0 \"quoted command\""
exit 1
fi

if [ -f hostfile ]
then
hostfile="hostfile"
elif [ -f /export/home/bob/data/hostfile ]
then
hostfile="/export/home/bob/data/hostfile"
else
echo "Can't find hostfile. No hosts to ping. Out..."
exit 2
fi

command=$1
squashed_command=`echo $command|/bin/sed -e 's/ * *//g' -e 's/|//' -e 's/\///g'`
buffered_command="${command};echo";

print "" >>tmpfile.$pid
print "Report Output for \"${command}\"" >>tmpfile.$pid
print "______________________________" >>tmpfile.$pid
print "" >>tmpfile.$pid

cat $hostfile|while read name colon address
do
if [ $name == "#" ]
then
:
else
print "\nRunning \"${command}\" on \c" >>tmpfile.$pid
print "$name... " >>tmpfile.$pid
print "$name... "
/usr/bin/ssh -n $address $buffered_command 2>/dev/null |tee -a tmpfile.$pid
fi
done

mv tmpfile.$pid OUTPUT.${squashed_command}.$pid


After a successful run of, for instance - scmd "/usr/sbin/pkginfo VRTSvcs" - you'll end up with a nicely formatted report with the output from all of the servers in your "hostfile," named OUTPUT.usrsbinpkginfoVRTSvcs.14756; the only real variable in the output report name will be the process id tacked on the end so you can run the same script multiple times and not overwrite your old data.

And, oh yes, don't forget to backslash all your special characters :)

, Mike