Hot adding Qlogic LUNs – the new method
I have demonstrated how to hot-add LUNs to a Linux system with Qlogic HBA. This has become irrelevant with the newer method, available for RHEL4 Update 3 and above.
The new method is as follow:
echo 1 > /sys/class/fc_host/host/issue_lip
echo "- - -" > /sys/class/scsi_host/host/scan<
Replace "<ID>" with your relevant HBA ID.
Notice - due to the blog formatting, the 2nd line might appear incorrect - these are three dashes, and not some Unicode specialy formatted dash.
I tried this on my system [RHEL 5.1 (Tikanga), kernel 2.6.18-53.1.19.el5 x86_64) which runs multipath and has host3 and host4.
QLogic QLE2460 with driver 8.01.07-k7.
Doing this for both hosts caused the currently mounted volumes to go read-only.
I would have expected the OS to just find the new LUNs and not touch the current ones – have you encountered this issue?
What multipath system were you using? dm-multipath? RDAC? PowerPath? Other?
Sending this command causes a bus reset and rescan (LIP) which results in udev adding new devices. Depending on the multipath system and how it was configured (in case of dm-multipath) a short disconnection can be ignored (dm-multipath continues to queue requests without blocking, so that as soon as the disk is accessible, queued I/Os are being sent to it).
I would have pointed at the multipath system as the one to blame. Could be that it rescanned the entire bus following the LIP, and during that time, either:
1. failed I/Os to the disk (which will result in I/O error and SCSI Bus error messages, and FS turn into RO)
2. Altered the path used for either of the disks (for example – using /dev/sdf instead of /dev/sde) which failed I/O operations to the disk. Mind you that normal Linux behavior is to add the new LUNs as the last device letters (/dev/sdX) even if the added LUN is prior to other LUNs (for example – you have had LUN 1, 2, 5 and you have added LUN 3). Device names will differ after a reboot, as the LUN order will define the device naming order.
So – what multipath system?
We are using dm-multipath:
device-mapper-multipath-0.4.7-12.el5
kernel: device-mapper: multipath: version 1.0.5 loaded
kernel: device-mapper: multipath emc: version 0.0.3 loaded
kernel: device-mapper: multipath round-robin: version 1.0.0 loaded
I believe you are correct in number 1 – my thinking is that the the timeout (or some other config) is set incorrectly.
This issue is being reviewed by EMC under a service request – I’ll try to post their results when I get them.
Thanks for your help.
You need to set EMC to allow “passthrough” for the LUNs through the SPs. What happens is that your RR policy causes dm-multipath to attempt access through the other path – the passive controller. This controller fails IOs and causes the system to get IO errors on the passive (and at that time – the only) path, which causes the system to fail the filesystem.
Can you post your /etc/multipath.conf file here? (exclude the remarks, of course)
I am trying to add a new lun to a rhel 4 release 5 system.
I am really looking for two pieces of information.
1) how to scan for the new lun without having a similar problem above (where th existing filesystems became readonly)
2) how to ensure that the device paths don’t change on reboot. IE lun 101 is /dev/sda and on reboot the new lun 90 becomes /dev/sda and 101 becomes sdb.
I haven’t found adequate information on this myself and would be more than willing to RTM if I could find TM.
We’re using EMC powerpath 4.5 for mutlipathing, this is a symetrix (unsure which model)
If this is EMC Symetrix, you should have no problem. Both interfaces allow passthrough (as this is the entire concept of this storage device). You can scan for new LUNs easily, per my document, and you should not encounter read-only cases.
About naming conventions – PowerPath has persistent naming, so there should be no problem with that either.
Hope it helps.
Ez