Sun Fire V120 as NAS part 7: Gigabit networking

Well, one week ago I received a new Sun Quad GigaSwift PCI card for this server. It's a full lenght PCI card with 64 bits PCI bus, but works in 32 bits PCI bus also.

As you can see in the pictures, the card installation is very easy. Only take out the blank bracket and the small grey plastic, then mount the card in th server slot. Finally return to set the grey plastic which locks the rear part of the GigaSiwft.

The network cards can be configured via the ifconfig utility in Solaris but to make this configurations stored we have to do some more file editting:

1.- The first step is to check if the server detects the GigaSwift card:

# grep ce /etc/path_to_inst
"/pci@1f,0/pci@1/pci@5/pci@0/network@0" 0 "ce"
"/pci@1f,0/pci@1/pci@5/pci@0/network@1" 1 "ce"
"/pci@1f,0/pci@1/pci@5/pci@4/network@2" 2 "ce"
"/pci@1f,0/pci@1/pci@5/pci@4/network@3" 3 "ce"

2.- Second step: create the /etc/hostname.ce0 file containing the name of the server. This name can't be the same in two different ethernet ports. In the 100MB port I'm using v120 as hostname, now I added a 'g' to the name:

# cat /etc/hostname.ce0
v120g

ce0 is the instance number for the port I'm going to use. I only need one port of the four installed with this card. If we want to use more port, more hostname.ce'instance_number' are needed.

3.- Now we have to edit the /etc/inet/hosts file but the vi editor says it's write protected (and I'm doing it beeing root). Like other changes I did before, I uploaded this file to my laptop via FTP, modified with a text editor and then downloaded to the server.

# cd /etc/inet
# cp /export/home/admin/hosts .

# cat /etc/inet/hosts
#
# Internet host table
#
::1     localhost
127.0.0.1       localhost
192.168.2.120   v120    loghost
192.168.5.120   v120g
~

4.- With the two configuration files ready, the card can be turned on:

# ifconfig ce0 plumb 192.168.5.120 up

Unfortunately the trasfer speed is the same as the speed obtained with the 100MB onboard port. Yes, I know there is a lot of parameters to tune, but I will look on the next days. Another problem I found is that: with both Ethernet ports configured (onboard an GigaSwift), when I unplug the onboard port cable, I have a lot of problems to connect with the Gigaswift (well, I can't connect). I think this is because Solaris uses the main port to check against DNS server. Then, I disconfigured the onboard ethernet port:

1.- Unplumb the card:

# ifconfig -a
lo0: flags=2001000849 mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
ce0: flags=1000843 mtu 1500 index 2
        inet 192.168.5.120 netmask ffffff00 broadcast 192.168.5.255
        ether 0:3:ba:5c:2b:7d
eri0: flags=1000843 mtu 1500 index 3
        inet 192.168.2.120 netmask ffffff00 broadcast 192.168.2.255
        ether 0:3:ba:5c:2b:7d

# ifconfig eri0 unplumb

# ifconfig -a
lo0: flags=2001000849 mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
ce0: flags=1000843 mtu 1500 index 2
        inet 192.168.5.120 netmask ffffff00 broadcast 192.168.5.255
        ether 0:3:ba:5c:2b:7d

2.- Delete the /etc/hosts entry (I modified the file in my laptop):

# cd /etc/inet
# cp hosts hosts.old
# cp /export/home/admin/hosts .
# cat hosts

#
# Internet host table
#
::1     localhost
127.0.0.1       localhost
192.168.5.120   v120g   loghost

3. Delete the /etc/hostname.eri0 file (here I moved it):

# cd /etc
# ls hostname*
hostname.ce0   hostname.eri0
# mv hostname.eri0 hostname.ri0.old

4.- Unplumb the ethernet port (and reboot):

# ifconfig eri0 unplumb
# reboot

Now everything works good with only the GigaSwift port 0 attached to the network. Something that is killing me is the low speed transfer via FTP. I can only reach 4.5 MB/s. This is a very low value. While doing a file tranfer I checked the server status with iostat:

# iostat
   tty        sd0           sd3           sd16          sd17           cpu
 tin tout kps tps serv  kps tps serv  kps tps serv  kps tps serv   us sy wt id
   0    1   2   0    4  439   8   28  789   8   26  758   8   31   60 26  0 14

Here I can see that the CPU is hard working. Maybe all the calculations needed by Samba, ftp and the other services joined by the ZFS filesystem are eating a lot of resources. I think I will check to optimize the network parameters for play and then I will reformat the hard disks using SVM instead of ZFS.

P.D.: Do you know what is this? (Check picture)

Fuel hinv

Never published my Fuel hinv. I don't have finished the upgrades/updates but here you are what I have now:
UPDATED 2010-03-02

fuel# hinv -mv
Location: /hw/module/001c01/node
IP34 Board: barcode MSM061 part 030-1707-003 rev -H
Location: /hw/module/001c01/node/cpubus/0
IP34PIMM Board: barcode MSX243 part 030-1836-001 rev -C
Location: /hw/module/001c01/Ibrick/xtalk/13
ASTODYV10 Board: barcode MSX356 part 030-1826-001 rev -B
Location: /hw/module/001c01/Ibrick/xtalk/14
IP34 Board: barcode MSM061 part 030-1707-003 rev -H
Location: /hw/module/001c01/Ibrick/xtalk/15
IP34 Board: barcode MSM061 part 030-1707-003 rev -H
1 600 MHZ IP35 Processor
CPU: MIPS R14000 Processor Chip Revision: 2.4
FPU: MIPS R14010 Floating Point Chip Revision: 2.4
CPU 0 at Module 001c01/Slot 0/Slice A: 600 Mhz MIPS R14000 Processor Chip (enabled)
Processor revision: 2.4. Scache: Size 4 MB Speed 300 Mhz Tap 0xa
Main memory size: 1024 Mbytes
Instruction cache size: 32 Kbytes
Data cache size: 32 Kbytes
Secondary unified instruction/data cache size: 4 Mbytes
Memory at Module 001c01/Slot 0: 1024 MB (enabled)
Bank 0 contains 256 MB (Standard) DIMMS (enabled)
Bank 1 contains 256 MB (Standard) DIMMS (enabled)
Bank 2 contains 256 MB (Standard) DIMMS (enabled)
Bank 3 contains 256 MB (Standard) DIMMS (enabled)
Integral SCSI controller 2: Version IEEE1394 SBP2
Integral SCSI controller 0: Version QL12160, low voltage differential
Disk drive: unit 1 on SCSI controller 0 (unit 1)
Disk drive: unit 2 on SCSI controller 0 (unit 2)
Integral SCSI controller 1: Version QL12160, low voltage differential
IOC3/IOC4 serial port: tty1
IOC3/IOC4 serial port: tty2
IOC3 parallel port: plp1
Graphics board: V10
Gigabit Ethernet: tg1, module 001c01, PCI bus 2 slot 1
Integral Fast Ethernet: ef0, version 1, module 001c01, pci 4
Iris Audio Processor: version EMU revision A4, number 2
PCI Adapter ID (vendor 0x14e4, device 0x1645) PCI slot 1
PCI Adapter ID (vendor 0x1077, device 0x1216) PCI slot 1
PCI Adapter ID (vendor 0x1102, device 0x0004) PCI slot 2
PCI Adapter ID (vendor 0x1102, device 0x7003) PCI slot 2
PCI Adapter ID (vendor 0x1102, device 0x4001) PCI slot 2
PCI Adapter ID (vendor 0x104c, device 0x8024) PCI slot 3
PCI Adapter ID (vendor 0x10a9, device 0x0003) PCI slot 4
PCI Adapter ID (vendor 0x11c1, device 0x5802) PCI slot 5
HUB in Module 001c01/Slot 0: Revision 2 Speed 200.00 Mhz (enabled)
IP35prom in Module 001c01/Slot n0: Revision 6.210
DMediaPro DM10 FW option: unit 0, revision 1.1.0
USB controller: type OHCI
USB Human Interface Device: device id 0 type tablet

Fuel goes Gigabit !!!

From a long time that everybody want his systems with GigaBit EtherNET. In SGI systems there is a few options and usually are expensive. In fact, SGI cards are really 3Com rebranded ones and a mod can be done. All cards have a Tigon 3 chipset (Broadcom 5701) but with different PCI identification in its EEPROM.

First mod was a new driver kernel, you can find all the information in this post of the nekochan forums, but with this kind of mod the card only runs in a patched IRIX OS.

The best solution now is two modify directly the EEPROM with the ethtool software included in linux distributions. You can find how to do this mod at the end of this post. In my case I tried with my old Fedora 7 installation I have in my workstation with no success, then I recovered and old Gentoo LiveCD 00.2006 and applied the modifications.

Now I have my fuel with GigaBit EtherNET but I have to end my network wiring at home!!!!