Saturday, January 19, 2008

Installing System Boards on SunFire 6800 Servers

This week has been pretty intense (I'm actually in the middle of working another lovely Friday through Saturday maintenance window), so I thought I'd start out the weekend with a quick how-to regarding adding system boards to SunFire 6800 servers. Some of it is manual, but some can be scripted out to run on Solaris hands free. For the most part, when you're dealing with this stuff, I'd recommend not scripting it out. Odds are something important is on a machine that costs more than most people make in a year ;)

The first thing you'll want to do is connect to your system console, via SSH or Telnet (Or direct to the machine - your choice). You can find information on the basics of getting on to the 6800/6900 series machines in this previous post.

Here's a walkthrough of the commands you'd enter at the system console for "Domain B" if you were going to add, say, system board 3 (SB3).

First we'd start out by issuing the following commands at the "Domain B" system console, after having physically inserted the board. Note that all the showboards commands aren't absolutely necessary. I did include them however, because, even if the board you're adding never physically existed in your 6800/6900 SunFire box (Just a placeholder), it's almost always "owned" by a Domain, so it's best to delete the non-existent board before adding it. That doesn't seem to make a whole lot of sense, does it? Count yourself amongst the many ;)

host-sc0:B> showplatform -v
host-sc0:B> showboards -v
host-sc0:B> deleteboard SB3
host-sc0:B> showboards
host-sc0:B> addboard -d C SB3
host-sc0:B> showboards
host-sc0:B> poweron SB3
host-sc0:B> showboards
host-sc0:B> testboard SB3
host-sc0:B> showplatform -v
host-sc0:B> showboards -v

Now, that we've verified that the board is installed correctly and the system recognizes it, we get to the part that we could, theoretically, script out. There are so many variables here, that, maybe we should leave that up to Sun. If we all write them enough, maybe they'll include it in a future release of Solaris. It should raise the going price from an-unbelievable-amount-of-money to an-unbelievable-amount-of-money and 99 cents ;)

Now, we'll take a look at the following output - all should indicate your new system board (SB3) as active and available. This is done from the OS, as opposed to the "Domain Console"

host # usr/platform/sun4u/prtdiag -v
host # psrin

Then, we'll need to make sure that Solaris absolutely positively knows it's there by "configuring" it, like so:

host # cfgadm -c configure N0.SB3

This step can take upwards of 15 to 20 minutes to complete. If you've ever powered down or powered up a 6800, this will seem like the blink of an eye :P

The next step really is quick. We'll use cfgadm again to "connect" the board:

host # cfgadm -c connect N0.SB5

Then, again, we'll just verify that it's there and, to be doubly sure, log back into the system console and ensure that it's been installed without errors as well, with:

host # /usr/platform/sun4u/prtdiag -v
host # psrinfo

And, then connect to the system console and do:

host-sc0:B> showplatform -v
host-sc0:B> showboards -v

And you should be all set, with the board showing up as belonging to your Domain :)

In a future post, we'll look at how horribly things can wrong in this scenario. I'll hope and pray that you don't have to suffer that in this lifetime!


, Mike