Hey there,
Today we've got an update that's been a while in the coming (mostly because no one ever requested it ;). Back in December of 2007, we put out a Perl script to report on NetBackup backup activity and, today, it's finally being updated.
But first, a few notes:
NOTE: If you don't care about what got updated or seeing any usage statistics and information, the script is tacked on to the end of this post and you can jump to it by clicking here.
VERY SPECIAL NOTE: This script's update has only been tested on Solaris (7,8,9,10) with the regular NetBackup backup and Oracle modules. If anyone runs this against another module (for Windows backups or any other module NetBackup has available) and it doesn't work, please send me (via the email link at the upper right of the page) your /usr/openv/netbackup/bin/admincmd/bpdbjobs -all_columns output (The location of your bpdbjobs binary may differ) for a successful job and a failed one. I'll be happy to update this script to include support for them. Of course, if you do send me that output, please alter any information that may be proprietary and get you in trouble! I promise not to do anything with it, but it's best to play it safe.
Some of the changes incorporated in this update include a check for "Catalog Backup" and "Backup Image Cleanup" jobs as well as checks to make the output more easily and quickly viewable. For instance, in the original script, no status was actually put in the Subject line. In this version, you'll get a status summary ("NO JOBS," "ALL OK" or "ERRORS") as well as the name of the server the report was generated on. Then you can just delete all the email reports you get that report that everything's fine (according to the reporter - Now, I've officially used that word enough to match my keywords. Let's just pray I don't have to use it again ;). And (since it was simple to do and not all of you may need to keep evidence that all was well at one point ;) we added a small - commented out for now - outer loop so you can remove those comments and have the script "only" mail you if there are errors to report(argh!).
Click on the picture below and maybe you'll be able to read those poorly doctored Subject lines ;)
The script is, again, very simple to use. All you need to do is run it from the command line and let it know how far back you want to go for your reporting (Generally, putting this in cron to run once a day and setting the command line offset to 86400 is good enough). Of course, the script can only report (I've stopped counting, now ;) as far back as "bpdbjobs" is set to remember! If you forget how to use it, and don't want to read the first 10 or 15 lines of it, just run it without any arguments and you'll get a "Usage" message.
host # ./bpreport.pl
Usage: ./bpreport.pl TimeInSecondsToReportBack
The output generally looks like the output below and, under moderate load on a system that only does its own backups, it runs in under a second for a day's worth of information (again, this timing would be affected by how much activity is logged on your server in the time span you specify on the command line)
Click the picture below and maybe something really cool will happen... probably not. Most likely the picture will just get larger again ;)
Below are some sample report outputs from the three types of reports shown in the "Subjects" picture from way up above, and the command line output directly above. After that, it's the script and we're out, baby ;)
Cheers,
P.S. Please forgive the exclamation marks. Typing in all caps is just so very exciting ;)
Click here to get directly to the script!
SAMPLE EMAIL REPORT WITH ERRORED JOBS - 24 HOUR SPAN!
NetBackup Status Report
-----------------------------------------------
Report Span = 06/06/09 ¦ 13:46:15 through 06/07/09 ¦ 13:46:15
-----------------------------------------------
Jobs With Errors To Report!
***************************************************
Server ¦ Policy ¦ Kb ¦ Min Elapsed ¦ Start Date ¦ Time ¦End Date ¦ Time - Error - File List
***************************************************
host123 ¦ warm_backup ¦ 0 kb ¦ 13.90 min ¦ 06/06/09 ¦ 23:24:02 ¦ 06/06/09 ¦ 23:37:56
ERROR: the backup failed to back up the requested files!!!
/usr/local/bin/mybackup.sh
***************************************************
-----------------------------------------------
Jobs Completed Successfully!
***************************************************
Server ¦ Policy ¦ Kb ¦ Min Elapsed ¦ Start Date ¦ Time ¦End Date ¦ Time - File List
***************************************************
host123 ¦ Catalog Backup ¦ 209056 kb ¦ 0.25 min ¦ 06/07/09 ¦ 02:04:35 ¦ 06/07/09 ¦ 02:04:50
***************************************************
host123 ¦ Catalog Backup ¦ 211872 kb ¦ 0.23 min ¦ 06/06/09 ¦ 23:37:52 ¦ 06/06/09 ¦ 23:38:06
***************************************************
host123 ¦ Default-Backup-Policy ¦ 12589911 kb ¦ 10.77 min ¦ 06/06/09 ¦ 23:00:20 ¦ 06/06/09 ¦ 23:11:06
/abcdef_1
***************************************************
host123 ¦ Default-Backup-Policy ¦ 12589911 kb ¦ 11.45 min ¦ 06/06/09 ¦ 23:24:09 ¦ 06/06/09 ¦ 23:35:36
/ghijkl_1
***************************************************
host123 ¦ Default-Backup-Policy ¦ 240761 kb ¦ 0.92 min ¦ 06/06/09 ¦ 23:12:31 ¦ 06/06/09 ¦ 23:13:26
/mnopqr_1
***************************************************
host123 ¦ Default-Backup-Policy ¦ 2759769 kb ¦ 3.77 min ¦ 06/06/09 ¦ 23:00:20 ¦ 06/06/09 ¦ 23:04:06
/stuvwxyz_1
***************************************************
host123 ¦ Default-Backup-Policy ¦ 2759769 kb ¦ 4.28 min ¦ 06/06/09 ¦ 23:24:09 ¦ 06/06/09 ¦ 23:28:26
/abcdef_2
***************************************************
host123 ¦ Default-Backup-Policy ¦ 548 kb ¦ 0.62 min ¦ 06/06/09 ¦ 23:36:49 ¦ 06/06/09 ¦ 23:37:26
/ghijkl_2
***************************************************
host123 ¦ Default-Backup-Policy ¦ 6640326 kb ¦ 8.43 min ¦ 06/06/09 ¦ 23:00:20 ¦ 06/06/09 ¦ 23:08:46
/mnopqr_2
***************************************************
host123 ¦ Default-Backup-Policy ¦ 6640326 kb ¦ 9.12 min ¦ 06/06/09 ¦ 23:24:09 ¦ 06/06/09 ¦ 23:33:16
/stuvwxyz_2
***************************************************
host123 ¦ Default-Backup-Policy ¦ 8547 kb ¦ 0.63 min ¦ 06/06/09 ¦ 23:11:28 ¦ 06/06/09 ¦ 23:12:06
/abcdefghijklm_3
***************************************************
host123 ¦ Default-Backup-Policy ¦ 8547 kb ¦ 0.63 min ¦ 06/06/09 ¦ 23:35:48 ¦ 06/06/09 ¦ 23:36:26
/nopqrstuvwxyz_3
***************************************************
host123 ¦ image catalog cleanup ¦ 0 kb ¦ 0.02 min ¦ 06/06/09 ¦ 15:19:14 ¦ 06/06/09 ¦ 15:19:15
***************************************************
host123 ¦ image catalog cleanup ¦ 0 kb ¦ 0.02 min ¦ 06/06/09 ¦ 23:38:08 ¦ 06/06/09 ¦ 23:38:09
***************************************************
host123 ¦ image catalog cleanup ¦ 0 kb ¦ 0.02 min ¦ 06/07/09 ¦ 02:04:51 ¦ 06/07/09 ¦ 02:04:52
***************************************************
host123-p ¦ Diff-Inc ¦ 3070549 kb ¦ 4.60 min ¦ 06/07/09 ¦ 02:00:00 ¦ 06/07/09 ¦ 02:04:36
/
***************************************************
host123-s ¦ Diff-Inc ¦ 359900 kb ¦ 1.77 min ¦ 06/07/09 ¦ 02:00:00 ¦ 06/07/09 ¦ 02:01:46
/
***************************************************
-----------------------------------------------
SAMPLE EMAIL REPORT WITH NO ERRORED JOBS - 12 HOUR SPAN!
NetBackup Status Report
-----------------------------------------------
Report Span = 06/07/09 ¦ 01:46:22 through 06/07/09 ¦ 13:46:22
-----------------------------------------------
Jobs Completed Successfully!
***************************************************
Server ¦ Policy ¦ Kb ¦ Min Elapsed ¦ Start Date ¦ Time ¦End Date ¦ Time - File List
***************************************************
host123 ¦ Catalog Backup ¦ 209056 kb ¦ 0.25 min ¦ 06/07/09 ¦ 02:04:35 ¦ 06/07/09 ¦ 02:04:50
***************************************************
host123 ¦ image catalog cleanup ¦ 0 kb ¦ 0.02 min ¦ 06/07/09 ¦ 02:04:51 ¦ 06/07/09 ¦ 02:04:52
***************************************************
host123-p ¦ Diff-Inc ¦ 3070549 kb ¦ 4.60 min ¦ 06/07/09 ¦ 02:00:00 ¦ 06/07/09 ¦ 02:04:36
/
***************************************************
host123-s ¦ Diff-Inc ¦ 359900 kb ¦ 1.77 min ¦ 06/07/09 ¦ 02:00:00 ¦ 06/07/09 ¦ 02:01:46
/
***************************************************
-----------------------------------------------
SAMPLE EMAIL REPORT WITH NO JOBS AT ALL - 6 HOUR SPAN!
NetBackup Status Report
-----------------------------------------------
Report Span = 06/07/09 ¦ 07:46:49 through 06/07/09 ¦ 13:46:49
-----------------------------------------------
No Activity To Report! Check If NetBackup Is Running!
-----------------------------------------------
bpreport.pl by Mike Golvach is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.
Based on a work at linuxshellaccount.blogspot.com.
Permissions beyond the scope of this license may be available at http://linuxshellaccount.blogspot.com.
#!/usr/bin/perl
#
# bpreport.pl - 2009 Mike Golvach - eggi@comcast.net
#
#bpreport.pl by Mike Golvach is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License. Based on a work at linuxshellaccount.blogspot.com. Permissions beyond the scope of this license may be available at http://linuxshellaccount.blogspot.com.
#
$first_tmp = $$;
$offset = $ARGV[0];
if ( -f "tmp.$first_tmp" ) {
unlink("tmp.$first_tmp")
}
if ( $#ARGV != 0 ) {
print "Usage: $0 TimeInSecondsToReportBack\n";
exit(1)
}
system("/usr/openv/netbackup/bin/admincmd/bpdbjobs -all_columns >>tmp.$first_tmp");
open(FT, "<tmp.$first_tmp");
@all_jobs = <FT>;
close(FT);
if ( -f "tmp.$first_tmp" ) {
unlink("tmp.$first_tmp")
}
$global_error_flag = 0;
$separator = "***************************************************";
$hostname=`hostname`;
foreach $job (@all_jobs) {
@file_list="";
@job = split(/,/, $job);
chomp($job[14]);
if ( $job[14] =~ /^$/ ) {
$job[14] = "0";
}
$error_flag = 0;
$report_time = $job[8];
($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime($job[8]);
$today = parse_date($sec,$min,$hour,$mday,$mon,$year,$wday);
($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime($job[10]);
chomp($today_end = parse_date($sec,$min,$hour,$mday,$mon,$year,$wday));
if ( $today_end =~ /12\/31\/69/ ) {
$today_end = "JOB IN PROGRESS";
}
$today_span_pre = ( $job[9] / 60 );
$today_span = sprintf("%.2f min", $today_span_pre);
$actual_time = time;
($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);
$right_now = parse_date($sec,$min,$hour,$mday,$mon,$year,$wday);
($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time-$offset);
$back_then = parse_date($sec,$min,$hour,$mday,$mon,$year,$wday);
report_span($report_time, $actual_time, $offset);
next if $dont_report;
if ( $job[3] > 1 ) {
@error_condition = `/usr/openv/netbackup/bin/admincmd/bperror -S $job[3]`;
chomp($error_condition = $error_condition[0]);
$job[3] = $error_condition;
$error_flag = 1;
$global_error_flag = 1;
}
next if ( $job[6] eq "backuphost" && $job[14] == 0 && ! $error_flag );
$file_list=32;
$file_list_still_going = 1;
$just_starting = 1;
while ($file_list_still_going) {
if ( $job[$file_list] =~ /(\/¦\\¦ALL)/ ) {
if ( $just_starting ) {
push(@file_list, "$job[$file_list] ");
$file_list++;
$just_starting = 0;
} else {
push(@file_list, "¦ $job[$file_list] ");
$file_list++;
}
} else {
$file_list_still_going = 0;
}
}
if ( ! $job[5] && ! $job[6] ) {
if ( $job[42] =~ /^.*(image catalog cleanup).*$/ ) {
$job[6] = $job[25];
$job[5] = $job[42];
$job[5] =~ s/^.*(image catalog cleanup).*$/$1/;
} else {
$job[5] = "Server UNKNOWN";
}
} elsif ( ! $job[5] && $job[6] ) {
if ( $job[42] =~ /^.*(Catalog Backup).*$/ ) {
$job[6] = $job[25];
$job[5] = $job[42];
$job[5] =~ s/^.*(Catalog Backup).*$/$1/;
} else {
$job[5] = "Policy UNKNOWN";
}
}
if ( $#file_list == 0 ) {
push(@job_successes, "$job[6] ¦ $job[5] ¦ $job[14] kb ¦ $today_span ¦ $today ¦ $today_end\n$separator\n");
} elsif ( $error_flag ) {
push(@job_errors, "$job[6] ¦ $job[5] ¦ $job[14] kb ¦ $today_span ¦ $today ¦ $today_end\nERROR: ${error_condition}!!!\n@{file_list}\n$separator\n");
} else {
push(@job_successes, "$job[6] ¦ $job[5] ¦ $job[14] kb ¦ $today_span ¦ $today ¦ $today_end\n@file_list\n$separator\n");
}
}
$¦=1;
## Don't send out okay mails if everything is fine)
#if ( @job_errors ) {
open(MAIL, "¦/usr/lib/sendmail -t");
select(MAIL);
if ( ( $#job_errors < 0 )&& ( $#job_successes < 0 ) ) {
print "Subject: NO JOBS: $hostname NetBackup Status Report - $right_now\n";
} elsif ( @job_errors ) {
print "Subject: ERRORS: $hostname NetBackup Status Report - $right_now\n";
} else {
print "Subject: ALL OK: $hostname NetBackup Status Report - $right_now\n";
}
print "From: root\@yourcompany.com\n";
print "Reply-To: you\@yourcompany.com\n";
print "To: you\@yourcompany.com\n";
print "\n\n";
print "NetBackup Status Report\n";
print "-----------------------------------------------\n";
print "Report Span = $back_then through $right_now\n";
print "-----------------------------------------------\n";
print "\n";
if ( ( $#job_errors < 0 )&& ( $#job_successes < 0 ) ) {
print "No Activity To Report! Check If NetBackup Is Running!\n";
print "-----------------------------------------------\n";
} else {
if ( @job_errors ) {
print "Jobs With Errors To Report!\n\n$separator\n";
print "Server ¦ Policy ¦ Kb ¦ Min Elapsed ¦ Start Date ¦ Time ¦End Date ¦ Time - Error - File List\n";
print "$separator\n\n";
@job_errors = sort @job_errors;
$job_error_count = @job_errors;
if ( $job_error_count ) {
print "@job_errors\n";
} else {
print "NO ERRORS TO REPORT. REPORT MAY BE IN ERROR!!!\n\n"
}
print "-----------------------------------------------\n";
}
if ( @job_successes ) {
print "\nJobs Completed Successfully!\n\n$separator\n\n";
print "Server ¦ Policy ¦ Kb ¦ Min Elapsed ¦ Start Date ¦ Time ¦End Date ¦ Time - File List\n";
print "$separator\n\n";
@job_successes = sort @job_successes;
print "@job_successes\n";
print "-----------------------------------------------\n";
}
}
#} # End of possibly-no-mail section
sub parse_date {
my ($sec,$min,$hour,$mday,$mon,$year,$wday) = @_;
%months = qw(0 Jan 1 Feb 2 Mar 3 Apr 4 May 5 Jun 6 Jul 7 Aug 8 Sep 9 Oct 10 Nov 11 Dec);
%weekdays = qw(0 Sun 1 Mon 2 Tue 3 Wed 4 Thu 5 Fri 6 Sat);
if ( $sec < 10 ) {
$sec = "0" . $sec;
}
if ( $min < 10 ) {
$min = "0" . $min;
}
if ( $hour < 10 ) {
$hour = "0" . $hour;
}
if ( $mday < 10 ) {
$mday = "0" . $mday;
}
if ( $yday < 10 ) {
$yday = "0" . $yday;
}
$mon = $mon + 1;
if ( $mon < 10 ) {
$mon = "0" . $mon;
}
$wday = $weekdays{$wday};
$year = 1900 + $year;
$year =~ s/^..(.*)$/$1/;
$nice_date = "${mon}/${mday}/$year ¦ ${hour}:${min}:$sec";
}
sub report_span {
my ($report_time,$actual_time,$offset) = @_;
$dont_report = 0;
$more_than_offset_hours_old = $actual_time - $report_time;
if ( $more_than_offset_hours_old > $offset ) {
$dont_report = 1;
}
}
, Mike
Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.
