Finding Physical Network Devices And Their Paths On Solaris

Hey There,

This week's lazy Sunday post deals with something a lot of Unix sys admins may not have to do. At least not in this particular way. It's not uncommon to be asked where a network interface physically "is" on a server, or where a cable is connected, but that task can sometimes be made difficult if you don't have physical access to the machine. Seeing it, and adding that to the knowledge you've collected on the command line, is almost guaranteed to succeed.

For the most part it's relatively simple, on Solaris Unix, to figure out the device paths to your network cards if you're using Sun hardware. Basically, just the output of "ifconfig -a" will get you started:

host # ifconfig -a
lo0: flags=1000849 mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
qfe0: flags=9040843 mtu 1500 index 2
        inet 99.5.99.99 netmask ffffff00 broadcast 99.5.99.255
        groupname GRP
        ether 8:0:20:**:**:**
qfe0:1: flags=1000843 mtu 1500 index 2
        inet 99.5.99.98 netmask ffffff00 broadcast 99.5.99.255

And you can tell that you only have the loopback device (which doesn't "really" count, although it's really bad "not" to have ;) and the qfe0 device.

Note that, if you have to find all the devices, even if they don't have cables attached, sometimes (if you have the latitude), running:

host # ifconfig -a plumb

Will make them show up in your "ifconfig" output. But you can most definitely find them by grepping the "qfe" network driver instance out of the /etc/path_to_inst file:

host # grep qfe /etc/path_to_inst
"/pci@8,700000/pci@3/SUNW,qfe@0,1" 0 "qfe"
"/pci@8,700000/pci@3/SUNW,qfe@1,1" 1 "qfe"
"/pci@8,700000/pci@3/SUNW,qfe@2,1" 2 "qfe"
"/pci@8,700000/pci@3/SUNW,qfe@3,1" 3 "qfe"

This shows you that there are actually 4 ports available, but, looking at "ifconfig," you can see that only port 0 is in use. Port 0 is supporting two separate IP's, but the qfe0:1 instance is "virtual" and doesn't indicate another physical port even though it does indicate another network address that's up on your machine.

Attached to the end of this post is a tiny script that will go through your "ifconfig -a" output and pick out the "real" devices that are up (no "virtuals" ;) and get the corresponding entry out of /etc/path_to_inst. It's not much, as far as script's go, but it's something you don't want to have to type too often ;)

Now comes the "iffy" part. Worst case, you'll have to go to the online manuals, or hard copy if you have it, to find this stuff out. For instance, on Sun v880's, the output from "prtdiag" does not show the device path for any of the pci devices. This makes it somewhat more difficult to figure out what PCI slot and/or port your network device is on, but it's still possible. Sometimes, it's just a matter of going through each line of output and determining what devices "couldn't" be the one you're looking for (Somewhat like the strategy you might use to pass an unnecessarily covoluted certification test.)

If you're fortunate enough, however, your machine may be able to help you make a direct correlation through the use of "prtdiag" (for instance, SunFire machines, up to at least the v440's, include this information. Some of Sun's newer machines have issues with spewing incorrect error messages when you use prtdiag. This shouldn't be an issue here, but just giving you a heads up in case you see any of those and think something else is wrong with your machine when it's perfectly fine).

For this example, we'll walk step by step through figuring out what PCI slot and port a network device resides on using a V440. First, we'll get the basic info using "ifconfig -a" (All output here will be summarized somewhat to save trees ;)

host # ifconfig -a
lo0: flags=2001000849 mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
ce3: flags=1000843 mtu 1500 index 3
        inet 99.9.99.12 netmask ffffff00 broadcast 99.9.99.255

Now, we know that the device we'll be looking for is "ce3." If there had been another listing up for, say, "ce3:1," we would ignore that, since it's a "virtual" interface.

Now we'll find that NIC's "physical" device path in /etc/path_to_inst:

host # grep ce /etc/path_to_inst|grep -w 3
"/pci@1f,700000/network@1" 3 "ce"

It's good to understand the basic format of /etc/path_to_inst if you need to search it for any information. It's very basic and each line consists of 3 elements, in order. The "physical device path" (the same as you would see if you ran an "ls" on /devices), the "instance number" and the "device driver name." It would probably be more easy to read if the device driver name came before the instance number, but it is what it is ;)

Now we'll check prtdiag to find out what slot this network card is plugged in on (or if it's a built-in port) and what port on the device the cable is (or should be) connected to:

host# prtdiag
System Configuration: Sun Microsystems  sun4u Sun Fire V440
...
================================= IO Devices =================================
Bus     Freq  Slot +      Name +
Type    MHz   Status      Path                          Model
------  ----  ----------  ----------------------------  --------------------
pci     66    MB          pci108e,abba (network)        SUNW,pci-ce
              okay        /pci@1f,700000/network@1

And we find that it's, indeed, a built-in card. In this instance, since "prtdiag" doesn't give us much information about the port, we can derive this from the physical name. As a general rule of thumb, if you look at the end of the "Name + Path" column (or the same thing in /etc/path_to_inst) and see "network@0" it's on port 0, and it's on port 1 if you see "network@1" (Remember that the numbering system begins at 0, so instance 0 is always the first port. Note, also, that the "Path" name had no secondary "pci" designation. Generally, if you see a short path from pci to network, you're dealing with a Built-In port, since the first pci addressin the device name is the board)

Here's a quick excerpt (unrelated) from a machine that provides more information on the port and slot numbers (but not the exact matching path!!!! Can you feel the self-doubt eating away at you? Blow it off ;)

========================= IO Cards =========================
...
     IO   Port Bus       Freq Bus  Dev,
Brd  Type  ID  Side Slot MHz  Freq Func State Name                              Model
---- ---- ---- ---- ---- ---- ---- ---- ----- --------------------------------  ----------------------
...
I/O   PCI   8    B    2    33   33  0,1  ok    SUNW,qfe-pci108e,1001             SUNW,pci-qfe/pci-bridg+

In this situation, you'd have to use some deductive reasoning, as well, to be sure.

As a last resort (or a first one if you prefer), check out Sun's Online Documentation Center where you can read all the technical manuals and copy off each model's device-path-to-PCI-slot/port number mappings.

Yay :)

Best wishes,





This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/bin/ksh

#
# whyDoIHaveToDoThis.sh - find our nic's and their physical device paths in /etc/path_to_inst
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License
#

ifconfig -a|sed 's/^\([^0-9]*\)\([^:]*\):.*/\1 \2/'|egrep -v 'lo|in|gr'|sort -u|while read one two
do
awk '{ if ( $3 ~ /'"$one"'/ && $2 ~ /'"$two"'/ ) print $0 }' /etc/path_to_inst|sed 's/\"//g'
done

, Mike

linux unix internet technology

A Few Linux Networking Tips

Here's a little something for those of us who use Linux (The place I work uses RedHat primarily) on a day to day basis. Lots of shops these days are switching from the more expensive solutions, like those offered by Sun and HP, to cut cost of deployment and maintenance, which opens up a great pathway into the Linux administration field for folks who are eager to learn. I can tell you, after working all angles of the *nix arena for a decade or so, that there's nothing more grating than having to fill out a form and wait for someone to do something you're supposed to be being paid to be good at. I'm not going to sneeze at my paycheck, of course, but I'm worried that, if I work at too many big corporations with big contracts, my mind will atrophy and I'll die a slow miserable death long before my body gives out ;)

In this post, I just wanted to touch on some Linux networking basics. I try to write this blog for users of all skill levels and I think some of my posts assume a lot of pre-knowledge. This isn't a "for dummies" site, by any means (I've never understood how they marketed that series of books so well. I've tried flat-out telling people they're idiots for most of my natural life and it's never ended well ;). That being said, I'm hoping this blog attracts folks with a wealth of experience and is also accessible to those new to the field.

So, today, I'm going to touch on some Linux networking tips at a basic level. As with everything, over time, I'll dive into these subjects in greater detail. But for now, we'll get to the meat of the post. A lot of times, I find, it makes more sense to know what to do and understand it later, rather than the opposite. To that end (This is all pretty much RedHat specific - please don't try this on a Sun box without a healthy ego - the error messages can be blunt ;) here we go:

1. Adding a default router:

You can generally do this one of two different ways:

If the NIC is already configured and UP, all you need to do is use the route command, like so:

host # route add default gw 127.0.0.1 dev eth0

If you want to add a default route permanently, you'll just need to add this line to the existing /etc/sysconfig/network configuration file:

Note that the NETWORKING and HOSTNAME variables should already be in there (If not, assign them values of yes and "whatever your hostname is", respectively. Also, your network may not be up ;)

GATEWAY=127.0.0.1

Of course, if you prefer, you can always use the /etc/init.d/network script to bounce your NIC's and routes (You set these up in the /etc/sysconfig/network and /etc/sysconfig/network-scripts/ifcfg-eth0, etc, scripts). You can also use the service command to bring up your network, bring it down or restart it (which will, again, re-read your configuration).

2. Displaying your NIC's device driver settings:

This is most commonly done with a command called ethtool. To get your NIC's settings you could do the following:

host # ethtool -i eth0
driver: tg3
version: 3.10u6
firmware-version:
bus-info: 05:01.0

You can then use this info to help with problem solving. For instance, if eth0 isn't coming up correctly, perhaps eth0 doesn't have a proper alias setup in /etc/modules.conf, like:

alias eth0 tg3 <--- This line tells us that eth0 is actually an alias referring to the tg3 device driver.

If you look in there and it's:

alias eth0 e1000

you've found the problem right there!)

3. Display your NIC's Speed, Duplex and Negotiation settings (also with ethtool):

This one is just as simple as the command above (ethtool is much nicer than using old-style ifconfig, netstat and/or kstat - although they all have their virtues and are necessary depending on how old your Linux distro is)

host # ethtool eth0
Settings for eth0:
Cannot get device settings: Resource temporarily unavailable
Supports Wake-on: g
Wake-on: d
Current message level: 0x000000ff (255)
Link detected: no

Looks like the link's not up for that one! It's just as easy to spot if all's well (You can sometimes tell from a good distance away ;):

host # ethtool eth1
Settings for eth1:
Supported ports: [ FIBRE ]
Supported link modes: 1000baseT/Half 1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 1000baseT/Half 1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: g
Wake-on: d
Current message level: 0x000000ff (255)

4. Now to configure, or initialize, those settings, we'll use ethtool as well:

Assuming we found out that the device associated with eth0 didn't have the cable connected, and we've got that all set up (along with correcting the alias in /etc/modules.conf, if that was wrong), we could fix it all by doing something like this:

ethtool -s eth0 speed 1000 duplex full autoneg off

You can add as many option/value pairs as you need (e.g. speed 1000) to get the job done. Don't forget to update the /etc/sysconfig/network and /etc/sysconfig/network-scripts/ifcfg-eth0, etc, files with your settings. If you don't, you'll have to do this every time the machine reboots!

Hope this has helped you out some or, at least, helped you get started taking on the Linux world :)

Cheers,

, Mike

linux unix internet technology