On Fri, Dec 23, 2005 at 03:09:19PM +1000, Michael Mansour wrote:
> > On Tue, 20 Dec 2005 17:57:30 +0000, John Hearns
> > <[log in to unmask]> wrote:
> > Speaking of SAN's, has anyone gotten the multipath feature in SL42
> > to work? We have an IBM blade equipped with a dual-port Qlogic
> > controller connected through two independent fabrics to a
> > DS400/Adaptec array with a dual-port controller. fdisk -l shows
> > two paths to every logical array, ie. each disk shows up twice.
> > The instructions make it look easy (comment everything out from
> > /etc/multipath.conf, start up multipathd and away you go). Running
> > multipath -v2 -l shows nothing. multipath -v3 shows that it sees
> > all of the devices but I don't know how to decipher where it is
> > going wrong (I'd attach the output but I'm typing this from home).
> > Ideas? -- Jeff
>
> I have been building various RHEL4 U1 blades which boot off SAN. The SAN is an
> HP XP12000 array.
>
> I used the HP Rapid Deployment Pack to build them.
>
> The gotcha for me was when I installed the HP Proliant Support Pack, which
> installs the "updated" qlogic driver (but defaults to single-pathed mode).
The "full" qla driver has a failover mode, that is "multipath" in the
sense that it does path-failover, but cannot do load balancing over
the available paths.
> This was turned out to be the problem for me. When the SAN guys enabled
> multipathing on the SAN, I could see two disks (/dev/sda and /dev/sdb) when I
> should be seeing only /dev/sda (but in multipathed mode).
In multipath mode as in "dm-multipath" you do see multiple /dev/sdX
devices that are mapped by dm-multipath and multipath tools onto a
/dev/dm-X device, e.g.
# for x in sdb sdc sdd sde; do /sbin/scsi_id -g -u -s /block/$x; done
3600508b4000106f30006300000240000
3600508b4000106f30006300000240000
3600508b4000106f30006300000240000
3600508b4000106f30006300000240000
# multipath -l
3600508b4000106f30006300000240000
[size=800 GB][features="0"][hwhandler="0"]
\_ round-robin 0 [active]
\_ 4:0:0:1 sdb 8:16 [active]
\_ 4:0:1:1 sdc 8:32 [active]
\_ round-robin 0 [enabled]
\_ 4:0:2:1 sdd 8:48 [failed]
\_ 4:0:3:1 sde 8:64 [failed]
# dmsetup ls
physik-data (253, 4)
systemzs02-swap (253, 1)
systemzs02-root (253, 0)
3600508b4000106f30006300000240000 (253, 2)
3600508b4000106f30006300000240000p1 (253, 3)
That means that /dev/sd{b,c,d,e} are four independent paths to the
storage, that are detected over the id
3600508b4000106f30006300000240000. These are mapped onto /dev/dm-2 or
whatever udev rules you have setup to manage the naming of this
device.
> When seeing two disks, the "LABEL=/boot" command in the fstab file
> would conflict as "both disks" have the same label.
multipath booting (as in "dm-multipath", not "qla-failover") is a bit
more complicated and not yet directly supported by RH (at least not in
RHELU2, yet). But there are instructions on how to get this going
nonetheless on the homepage of multipath-tools and also promoted by
AXIS (a RH partner).
> When I finally figured out that the HP PSP was not installing the qla2x00
> driver in failover mode, then I made sure it did and the next time I booted
> only /dev/sda would show up, even though two paths were seen.
>
> My point is, because initrd is loaded at the very early stages of the kernel
> boot process, then if initrd has single-pathing enabled (via the driver that's
> built into it), you'll see two disks. Everything after that will cause you
> problems.
It depends on what you want. The qla-failover will present only a
/dev/sdX device and handle failover in the driver layer. The kernel
gods don't want this functionality in the drivers' layer, but prefer a
more generic one in the dm layer, so they rejected the failover code
(as well as the /proc/scsi interface), and set on dm-multipath.
But if you need failover only and no load-balancing, or need booting
off multipath, and only have qla HBAs in the system, then qla failover
is probably the better solution (longer on the market and thus
probably more stable).
Merry Xmas!
--
Axel.Thimm at ATrpms.net
|
