Shutdown, Reboot and Init Process Flow On Solaris Unix

Hey There,

Today's post harkens back to an earlier post we did on clearing up some common misconceptions about Solaris run levels. That post took care of going through the in's and out's of what the differences between boot, reboot, init, shutdown, etc, all mean and has a lot of good practical examples to demonstrate each point.

Branching off from that post, today we're going to add something of an appendix to it. If you're interested in some of the nittier gritty of the Solaris run level changing process, definitely give that older post another look. Maybe it'll go down easier the second time ;)

Below, we've got a nice collection of commands that would (or could) normally be executed by a root (or privileged) user to switch run levels, put together with what run level scripts in what run level script directories get run when those commands get ...executed (almost typed "run" again ;) And, just for flavor, the command and the script/directory execution sequence are further segmented into what run level the Solaris Unix OS switches to at each salient point. Note that, when you see a run level listed in the "run level" column, it's at this point that the run level has become the run level listed. Any points without run levels listed are executed at the run level of the previous entry in the example. Just trying to write a thousand words on how to not have to explain subtle differences in understanding of the blatantly obvious. I'm well on my way to being even more confused than I was when I woke up this morning... ;)

Hope you enjoy this chart and find it useful. Print it out and stick it to your refrigerator with lots of magnets from various pizza parlours and real estate agents. You never know when you'll have an epiphany in the kitchen. ...it's getting late in the day (both technically and figuratively) when you're all alone telling in-jokes ;)

Note: This chart assumes that you're starting from run level 0 (except for the reboot, init 6 and "shutdown -i6" sections). Generally, you can reverse the order of execution if you're starting from, say, run level 3, or combine multiple actions (or their opposites) when transitioning from one particular run level to the next. The stop/start lines for identical run level directories are also dependent on your understanding of which direction you're heading (down to 0 or up to 3, for instance). If this is too confuggling, just let me know :) Again, our previous post on unconfusing Solaris run levels may be of help in conjunction with this little map. Switching to run level 5 has been left out on purpose. Since that level is "power down," everything's going the way of the K ;)

Cheers and pardon the formatting,

---

command       run level directories/scripts      current run level (retrieved from "who -r" if it seems to not make sense)

boot -s      /etc/rcS.d/S* start      0

boot -S      /etc/rcS.d/S* start      0

reboot -- -s      /etc/rcS.d/S* start      3

shutdown -is      /etc/rcS.d/K* stop      S

shutdown -i0      /etc/rc0.d/K* stop      0
            /etc/rc0.d/S* start   

shutdown -i6      /etc/rc0.d/K* stop      6
            /etc/rc0.d/S* start   
            /etc/rcS.d/S* start   
            /etc/rc2.d/S* start      3
            /etc/rc3.d/K* stop   
            /etc/rc3.d/S* start 

init 0      /etc/rc0.d/K* stop      0
            /etc/rc0.d/S* start   

init s      /etc/rcS.d/K* stop      S
            /etc/rcS.d/S* start  

init S      /etc/rcS.d/K* stop      S
            /etc/rcS.d/S* start

init 1      /etc/rc1.d/K* stop      1
            /etc/rc1.d/S* start 
 
init 2      /etc/rc2.d/K* stop      2
            /etc/rc2.d/S* start 
 
init 3      /etc/rc3.d/K* stop      3
            /etc/rc3.d/S* start 

init 6      /etc/rc0.d/K* stop      6
            /etc/rc0.d/S* start   
            /etc/rcS.d/S* start   
            /etc/rc2.d/S* start      3
            /etc/rc3.d/K* stop   
            /etc/rc3.d/S* start

---

, Mike

---

Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

---

Changing Solaris Run Levels And Clearing Up Some Common Misconceptions

Hey there,

Today's post hearkens back to an earlier time, when I was administrating Solaris 2.4 and figured Sun had big plans for run level 4. It was, after all "reserved for future use." I don't know about you, but (to this day) when I see terminology like that, I tend to shy away from getting too comfortable with working in a cheap hack. My thinking was that, at some point (given Sun's policy at the time) they'd come up with a unique use for run level 4 and I'd have a whole lot of stuff set up to launch from there and my world would be thrown into chaos ;)

I no longer worry too much about the chaos (something, somewhere is going to break or I'll eventually be out of a job ;), but I still like to keep tabs on Solaris' "level 4" status updates and their command run-level policies. So, today, we'll take a brief look at "run level 4" and then look at a few commands, used to change run levels, that may not act as you would expect.

1. Run level 4: It is still, technically, reserved for future use on Solaris. However, the subtle difference in meaning over time has been stated more clearly with the last few releases. Now, about half of the places you look (on Sun's Documentation Site) list "run level 4" as being "user-defined." What this means, aside from the fact that any admin can make use of it if he or she deems it worthwhile, is that I spent a lot of time worrying about nothing. Of course, I'm willing to forgive, since I never had any use for "run level 4" anyway.

On most Linux and Unix distro's, run level 4 is considered open and user-definable. In a few instances, on HP-UX and Slakware, among others, run level 4 is the default run level for the X-Windows display manager. Generally, this is found on run level 5. But, before I go crazy over-explaining, check out Wikipedia's entry on run levels for virtually every system known to man :)

The only thing you have to make sure of, for the most part, if you want to use Solaris' run level 4 is that you create an entry for it in /etc/inittab. Something like:

s4:4:wait:/sbin/rc4                     >/dev/msglog 2<>/dev/msglog </dev/console

would probably suffice for distributions up to, and including Solaris 9. If there are minor differences in your distribution, you can simply copy the line for "s3," paste it below and change all the 3's to 4's. Fairly simplistic, but without it, Solaris' init will never know to look there and will not know what you mean when you try to change to run level 4.

On Solaris 10, things are a little goofier, but in a good way. You can mess around with the inittab and set things up that way, but if you use the "out-of-the-box" configuration and stick with the Service Management Facility (SMF), you won't need to worry about that. SMF has no concept of run levels and doesn't need /etc/inittab. Since SMF handles the order of starting services (with dependencies), you don't need to have a "separate" run level to accomplish whatever end you were shooting for. You can add new services with SMF and then goof around, to your heart's content, with svcadm, svccfg, svc.configd, etc. You may find yourself longing for the "old ways" until you get used to it.

2. Some command run level changing commands and what they're limited to: Some of this stuff still surprises me, since it seems counter-intuitive, but I guess I've always been lucky in the way I start stuff up and shut it down (my apologies to Clint Eastwood for bastardizing his famous quote from "Unforgiven" ;) Plus, I'm a big fan of "init" :) We're going to forget run level 4 exists for most of this part since it rarely matters... technically...

a. init: This command can be used to bring Solaris to any run level you specify (0, 1, s, S, 2, 3, 4, 5 and 6). This is a bone of contention, if you ever have to get certified, but Sun says there are only 8 run levels, of which only 7 are used. If you count "s" and "S" as separate (even though they both do the same thing) I would back you up if your answer was 9.

b. reboot: This command can only bring the system to run level 6. However, you can make reboot take you to other places by terminating the getopts routine and adding a flag afterward, so that:

host # reboot -- -s

would allow you to use the reboot command to get into single user mode. Also, another reason this perplexes me as much as it does is that, technically, bringing the machine to run level 6 entails the system going through a series of run levels leading all the way to the default run level (which is 3, if you left it alone). Sun's answer is technically true, but somewhat misleading. The only thing you really "can't" do with reboot is stop at run levels 0, 1, 2 and 5 (unless you make one of them your default run level in /etc/inittab - not a good idea, especially for 0 and 5 ;)

c. shutdown: This command can only bring the system to run levels 0, 1, s/S, 5 or 6. This makes sense to a degree, since run level 6 is reboot, 5 is power-off and 0 is PROM. I don't understand why it can shut the system down to run level 1 (single user with all local filesystems mounted read-write) and s/S (single user with only / mounted read-only) and not shut it down to run level 2 (basic multi-user with network) from run level 3 (same multi-user with nfs and some additional network services) straight up. As noted above, regarding reboot, you can actually make this command take you to any run level except 2 (unless you set that as your default run level in /etc/inittab)

d. poweroff: This command can only bring the system to run level 5. Perfectly sensible.

e. halt: This command can only bring the system to run level 0. This makes sense, too.

f. uadmin: This command can only bring the system to run levels 0, 5 and 6 (see above regarding why run level 6 means you can also use this command to get to run level 3, or your system default run level, as well). This command is restricted to super-user access by default. For the longest time I thought it could get you to any run level straight off, although I never needed to use it except to dump the system. An interesting fact about uadmin is that it can accept arguments. Which means, if you're feeling moderately clever, you can make it go places Mother won't let it go, by fudging it a little. Consider the following, understanding that uadmin converts arguments into integers as such, and is invoked as "uadmin command function optarg":

commands:

1 = No Disk Sync
2 = Sync Disks

functions:

1 = A_REBOOT <-- Run level 6
2 = A_SHUTDOWN <-- Run level 0
5 = A_DUMP <-- Run level 5

optional arguments:

Try whatever you want out, if you can afford to. Be sure to enclose an argument with spaces in double quotes.

host # uadmin 1 1 "-s kernel/unix" <-- Basically any option you can pass to the PROM "boot" command, you can sneak in here!

also, there are multiple versions of uadmin available, depending on your Solaris release, so you may be looking at these set of options, which will get you to run levels 0, s/S, 1, 2, 5 and 6 (which gives you run level 3) as well as allowing you the optional argument:

uadmin 2 0: sync the filesystems and drops system to ok prompt
uadmin 2 1: sync the filesystems and reboots to multi-user mode
uadmin 2 2: sync the filesystems and reboots interactively
uadmin 2 3: sync the filesystems and reboots to single-user mode
uadmin 2 6: sync the filesystems and powers off the system

uadmin 1 0: do not sync filesystems and drops system to ok prompt
uadmin 1 1: do not sync filesystems and reboots to multi-user mode
uadmin 1 2: do not sync filesystems and reboots interactively
uadmin 1 3: do not sync filesystems and reboots to single-user mode
uadmin 1 6: do not sync filesystems and powers off the system

Now, let's kick back and let the raging debate about OS version-and-release accuracy begin ;)

Cheers,

, Mike

---

Please note that this blog accepts comments via email only. See our Mission And Policy Statement for further details.

---

Using chkconfig To Manage Linux Service Run Levels

Hey there,

If you're like me, you've probably gotten used to creating init scripts and all the various linking to run level directories that needs to be done so that certain services will start and stop at the run levels you decide you want them to.

Even so, I'm always impressed with ways to make this sort of setup easier. Especially when it comes to manipulation of start and stop (init) scripts that come with system software. Solaris' mish-mosh of hard and soft linked run level scripts can be a headache, for sure, but Redhat, Fedora and other Linux flavours can be a bit of a pain, as well. The main problem I run into is making a customized change that ends up getting overwritten by an RPM upgrade or things of that nature.

Thankfully, that issue has been addressed, more than adequately, in a few ways. One of my favorites is the chkconfig command (usually located in the /sbin directory). This tool makes it very easy for an administrator to control at what run levels virtually every system-controlled service starts and stops.

Today, we're going to look at a few common ways to use chkconfig to make administration of run levels more convenient.

The first way is the most obvious. If you want to know everything that's going on with every service that's being controlled, simply type:

host # /sbin/chkconfig --list

and you'll get output similar to this:

acpid 0:off 1:off 2:off 3:off 4:on 5:on 6:off
anacron 0:off 1:off 2:on 3:on 4:on 5:on 6:off
...

An easy way to find out what services are "on" is to just pipe the above command to grep. If you want to know what's running "now" (or, at least, what should be ;), you can find your run level with the who command and then use chkconfig to find all the active services:

host # who -r
run-level 3...
host # chkconfig --list|grep "3:on"

or, if you're only interested in one service, you can specify it and chkconfig will query its state at your current run level, like so:

host # chkconfig anacron

More specifically (since that would only show you services that get started at run level 3, you could do this:

host # chkconfig --list|egrep '[123]:on'

and you should see all the services that are running currently (assuming your machine boots normally, through run levels 1 and 2, before getting to 3).

You can also use chkconfig to configure any listed service to start and/or stop at any run level you choose. You can even do multiple run levels at once. So, all of the following commands would be valid (assuming you have the service "anacron" on your machine):

host # chkconfig anacron on
host # chkconfig anacron off
host # chkconfig --level 3 anacron on
host # chkconfig --level 3 anacron off
host # chkconfig --level 235 anacron on
host # chkconfig --level 046 anacron off

Also note that on some systems, on and off only apply to run levels 2, 3, 4 and 5. Setting other levels may not work, but should not cause failure.

An alternative to on or off, is to use reset. This keyword will reset the service to start and stop at the run levels it defaults to (look near the bottom of this post to see from where chkconfig gets that information).

host # chkconfig anacron reset

Some simple scripting can make it so that you can replicate this behaviour for multiple services. At this point, to my knowledge, you can only either make changes globally or per-service. In any event, something like this is simple enough to work out at the command line (after checking out all of the services with --list, put the ones you want to start at runlevels 3 and 5 , and be turned off at all other levels, into a text file, one service per line). Note, that this is just my little command line script, and can be done a lot of different ways, so you should do this however makes the most sense for you:

host # for x in `cat MYSERVICES`;do /sbin/chkconfig --level 35 $x on;/sbin/chkconfig --level 01246 off;done

And the final (and most helpful, since it keeps you from having to remember how to format an init script to work with chkconfig) thing chkconfig does, is allow you to add and delete services. If your service isn't standard, you may still have to write your own init script, but for all packaged (and handled) services, this is a snap.

To remove a service that's currently active, just type:

host # chkconfig --del anacron

And to reinstate it, just type

host # chkconfig --add anacron

And, bingo, you're all set :) Note that the above add/del function really only creates and destroys symlinks from the init directory to the run level directories, so if you've deleted the actual init script, using chkconfig to re-add the service probably won't work. If you still have access to the RPM for that package, you can fix it to replace any missing files that way first.

If you do decide to create your own init script, and want it to be controlled by chkconfig, just make sure that you have these lines near the top of the script, in the comment section, below the shebang line and above any other comments in the script, like this:

#!/bin/sh

#
# chkconfig: 235 10 60

This part of the init script is what chkconfig looks for, and, in this instance, it tells chkconfig that the service should start at run levels 2, 3 and 5, it should be turned off for all other run levels (chkconfig's default assumption) and that it should have a start priority of 10 and a stop priority of 60. The priority numbers that you include in your script are relative and shouldn't generally be an issue. The defaults are usually 20 and 80. You can also add a "#description:" line (directly after the required chkconfig line) to your script if you want to be able to see that in chkconfig output, but it's not necessary.

Here's hoping this helps your run level management step up and get easier, all at the same time! :)

Cheers,

, Mike

Using Last To Its Full Potential On Linux

Hey There,

This probably comes as no surprise to most Unix or Linux administrators out there (at least this first thing), but I find it's always interesting how rarely the "last" command is used to determine anything other than the users logged in "now" and the "last" time a user logged in.

Granted; the last command doesn't offer too much in the extra-functionality department, but it does have one very useful feature. Normally, if you were to run last, you'd see output like the following:

reboot   system boot  2.6.5-7.283-smp  Thu Jan 25 18:06          (00:21)
user1    pts/1        host.xyz.com    Thu Jan 25 08:03 - down   (00:27)
reboot   system boot  2.6.5-7.283-smp  Thu Jan 25 08:01          (00:29)
user1    pts/1        host.xyz.com    Thu Jan 25 07:50 - down   (00:06)

But, if you add the "-x" switch to the "last" command, it gives you a lot more detailed information about system run-level changes, which makes it a more accurate way to determine what happened if, and when, your system ever goes down unexpectedly! Here's output from that same swatch of time using "last -x":

runlevel (to lvl 3)   2.6.5-7.283-smp  Thu Jan 25 18:06 - 18:28  (00:21)
reboot   system boot  2.6.5-7.283-smp  Thu Jan 25 18:06          (00:21)
shutdown system down  2.6.5-7.283-smp  Thu Jan 25 08:31 - 18:28  (09:56)
runlevel (to lvl 6)   2.6.5-7.283-smp  Thu Jan 25 08:31 - 08:31  (00:00)
user1    pts/1        host.xyz.com    Thu Jan 25 08:03 - down   (00:27)
runlevel (to lvl 3)   2.6.5-7.283-smp  Thu Jan 25 08:01 - 08:31  (00:29)
reboot   system boot  2.6.5-7.283-smp  Thu Jan 25 08:01          (00:29)
shutdown system down  2.6.5-7.283-smp  Thu Jan 25 07:57 - 08:31  (00:33)
runlevel (to lvl 6)   2.6.5-7.283-smp  Thu Jan 25 07:57 - 07:57  (00:00)
user1    pts/1        host.xyz.com    Thu Jan 25 07:50 - down   (00:06)

Interestingly enough, the "-x" flag still isn't available in Solaris, even in all the versions of the 10.x strain that I've checked out. There are other methods to get the information, but they are more tedious and require the user, or admin, to do enough work that they may as well script it out (or write a wrapper for "last" that allows for a "-x" flag ;)

Generally, you'll notice that this extra information is assigned to the "user" with the name of your "kernel" revision ( usually the value of "uname -r" or "uname -k." 2.6.5-7.286-smp, in our case) so you can run:

last -x|grep `uname -r`

to restrict your output to this system information and ignore all the user logins/logouts :)

While the information that "last -x" provides may seem extraneous and not generally worthwhile, I'd say that it's exactly the opposite. For instance, in our first, straight-up, last command, we only get the reboot time of (we'll take the last one) January 25th at 8:01 a.m. ( The year is 2008 since we're taking this from the top of the output).

Interestingly enough, again, last does not print the year, although you can get that information if you really want it. For more info on that, check out our previous posts on scripting out user deletion on Unix and the modifications for Linux, which both include Perl routines for tearing open wtmpx so you "can" get the "year" data if you want it :)

With "last -x," for that very same reboot, we know that the reboot command was issued by the system on January 25th at 8:01 a.m. (this helps put into perspective what last, without arguments, is "really" reporting. The "beginning" of the reboot process). We can then see that (and, just as a reminder, we're reading from the bottom of the output up!) the request to switch to "run level 6" (which is "reboot") was actually issued at 7:57 a.m.

The "shutdown" information on the next line is an all-encompassing time. It should always match the entire amount of time spent in all of the states we're looking at. It starts with the switch to "run level 6" at 7:57 a.m. and ends with the switch to "run level 3" (this system's default run level) at 8:31 a.m. Finally, after the "reboot" line, we see the switch to "run level 3" which happens from 8:01 a.m. (the time the "reboot" was called) until 8:31 a.m. (the time the system fully got back to "run level 3").

As you can see, just knowing the "reboot" time doesn't give a very accurate report of the time involved in the reboot, at a glance. We just know that it happened at 8:01 a.m. If we wanted more information, we might need to go look at system logs.

"last -x," however, makes it so that we can, just by reviewing that output, see that the reboot process actually began at 7:57 a.m. and didn't complete until 8:31 a.m. That may not be a long time for this machine (If it is, you'll know to look at the system logs, now :), but the length of time required for a normal reboot is very system-independent and, also, dependant on what sorts of scripts and programs are run on a controlled reboot, etc.

And that's the last I have to say about that ;)

Best wishes,

, Mike