Moving Boards Between Domains On Sun 6800/6900 Servers

Hey There,

For some light weekend fare. I thought I'd put together this quick little tutorial on the simple task of moving boards (CPU, IO) between domains on Sun 6800/6900 servers. For our purposes, we'll assume that the boards are already installed on the physical system and that we want to move one CPU and one I/O board from Domain C to Domain A.

The first thing you want to do is connect to the System Controller and then attach to the Platform Shell. This is basically necessary since you'll be working on two separate Domains. The Platform Shell will allow you to connect to each and go back and forth if you need to.

System Controller 'host-sc0':

    Type  0  for Platform Shell

    Type  1  for domain A console
    Type  2  for domain B console
    Type  3  for domain C console
    Type  4  for domain D console

    Input: 0

Platform Shell

host-sc0:SC>

The first thing you'll want to do is show the board assignments, to make sure you're moving the right devices. You can do this simply with the showboards command below (output trimmed):

host-sc0:SC> showboards

Slot     Pwr Component Type                 State      Status     Domain
----     --- --------------                 -----      ------     ------
/N0/SB0  Off CPU Board                      Assigned   Not tested C
/N0/SB1  Off CPU Board                      Assigned   Not tested C
/N0/SB2  Off CPU Board                      Assigned   Not tested A
/N0/SB3  Off CPU Board                      Assigned   Not tested A
/N0/SB4  Off CPU Board                      Assigned   Not tested C
/N0/SB5  Off CPU Board                      Assigned   Not tested C
/N0/IB6  Off PCI I/O Board                  Assigned   Not tested C
/N0/IB7  Off PCI I/O Board                  Assigned   Not tested C
/N0/IB8  Off PCI I/O Board                  Assigned   Not tested C
/N0/IB9  Off PCI I/O Board                  Assigned   Not tested A

Now assuming we wanted to move IO Board 7 and CPU Board 1 from Domain C to Domain A, you can do so, also simply, using the deleteboard and addboard commands:

host-sc0:SC> deleteboard IB7
host-sc0:SC> deleteboard SB1
host-sc0:SC> addboard -d A IB7
host-sc0:SC> addboard -d A SB1

Then just double check :)

host-sc0:SC> showboards

Slot     Pwr Component Type                 State      Status     Domain
----     --- --------------                 -----      ------     ------
/N0/SB0  Off CPU Board                      Assigned   Not tested C
/N0/SB1  Off CPU Board                      Assigned   Not tested A
/N0/SB2  Off CPU Board                      Assigned   Not tested A
/N0/SB3  Off CPU Board                      Assigned   Not tested A
/N0/SB4  Off CPU Board                      Assigned   Not tested C
/N0/SB5  Off CPU Board                      Assigned   Not tested C
/N0/IB6  Off PCI I/O Board                  Assigned   Not tested C
/N0/IB7  Off PCI I/O Board                  Assigned   Not tested A
/N0/IB8  Off PCI I/O Board                  Assigned   Not tested C
IB9      Off PCI I/O Board                  Available  Not tested A

And repeat as necessary. When you boot up the separate domains (The logical machines) you'll enjoy the benefit of the new hardware. Easy peasy :)

Cheers,

, Mike

linux unix internet technology

Sifting Through the PROM

Every once in a while, like most admins (I hope;), I'll find myself in a situation where I'm working on a downed Solaris box and stuck at the PROM level. For the purposes of this post, we'll assume that diagnostics over and above the PROM level are impossible (Which is sometimes true).

The issue here isn't that I'm stuck at the PROM. That's no big deal and happens often. The problem is, every so often, I'll find myself stuck at the PROM on a Sun system type I've never worked on before. Depending upon how varied the PROM interface and commands are for that system, digging into my old bag of tricks my have me drawing up blanks left and right. The PROM "is" always very courteous about letting you know you got the command you typed in wrong and doesn't offer much in the way of help. For instance:

ok> show-me-the-money
show-me-the-money ?
ok>

The output from the PROM, letting you know it doesn't know what you're talking about, can vary also.

One of the most useful commands I've ever run across, at the PROM level, is called "sifting." Since you really have no help immediately available, and searching for particular commands in manuals or on the net can suck up huge amounts of time, this command is a life saver. Some folks refer to it as a "sifting dump" (the technical name), but the end result and usage are the same.

Best of all, the sifting command is incredibly easy to use (and the "one" command you should remember if you forget everything else ;). The command itself is incredibly simplistic. All it does is find all the OBP/PROM commands that contain the string you've specified as the only argument, and prints them out to the screen. That's it!

So, even if you've got your PROM chops down, when life throws you that curveball, you can still find the solution easily. Consider that you're working on a 4500 series server that hasn't been rebooted in 3 years. On boot up, everything seems to be going fine and, suddenly, you're presented with a poorly worded error about clock's being out of sync. What??? Assuming, still, that you have no access to any reference material of any kind on any medium, you can still figure this one out using sifting.

To continue, once that error pops up, you get dropped back to the PROM. Subsequent attempts to just boot again and hope the problem goes away have equal results. At this point, you know two things from reading the error itself: One clock on the system is out of sync with another clock, and/or the opposite. But how to synchronize them, since that would seem to be all the boot process wants you to do? This is where sifting becomes invaluable. Just like grep, the smaller the search string you provide, the more results you'll get. In this instance, we could do:

sifting clock

and we'd get back, among a few other things that can easily be dismissed:

copy-clock-tod-to-io-boards
copy-io-board-tod-to-clock-tod

I usually opt to copy the system clock to the I/O boards (copy-clock-tod-to-io-boards). My idea of best practices dictates that I'd go about this like so:

setenv auto-boot? false
reset-all
copy-clock-tod-to-io-boards
setenv auto-boot? true
boot

And, magically, the "clocks out of sync" error is gone!

The little example I showed you above is just the beginning. You can use sifting to find any command you need to know the exact name of (the PROM is so unforgiving ;). Try "sifting probe" if you can't remember what that probe command was that you needed to run, or "sifting net" if you've forgotten the exact name of "show-nets" or "test-net" (notice the seemingly arbitrary difference; one is plural and the other is singular). You can run sifting against a single character if you want to. Just get ready to have plenty of options ;)

Enjoy your exploration of the PROM commands available on your system of choice, and remember: sifting through the PROM will eventually lead you to the answer :)

, Mike

linux unix internet technology