Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
Attaching a linux box to a SAN | ||||||||
Line: 39 to 39 | ||||||||
| ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
Line: 54 to 54 | ||||||||
If you have just added a new volume to an array, you will probably need to get the HBA to rescan the bus to detect it. For QLogic cards, the following should work
| ||||||||
Changed: | ||||||||
< < | echo "- - -" > /sys/class/scsi_host/host_id_/scan | |||||||
> > | echo "- - -" > /sys/class/scsi_host/host{id}/scan | |||||||
Though you will have to use the path appropriate to your HBA and port in use. For multipath you will need to do this for each port. | ||||||||
Line: 247 to 247 | ||||||||
Multipath | ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
| ||||||||
Changed: | ||||||||
< < |
| |||||||
> > |
| |||||||
| ||||||||
Line: 283 to 283 | ||||||||
-- AlastairScobie - 27 Jul 2007 | ||||||||
Added: | ||||||||
> > | -- Updated 2013-05-31 |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
Deleted: | ||||||||
< < | ||||||||
Attaching a linux box to a SAN | ||||||||
Line: 268 to 268 | ||||||||
Added: | ||||||||
> > | Jan 09 - Note that for Nexsan products, you may have to add the following to /etc/scsi_id.config in order that the multipath software identifies the two paths as being to the same volume.
vendor="NEXSAN",options=-p 0x80 | |||||||
Added: | ||||||||
> > | ||||||||
-- AlastairScobie - 27 Jul 2007 |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
Added: | ||||||||
> > |
Attaching a linux box to a SANIt is absolutely essential that all fibre connections are removed from a machine during installationIntroduction
Configuring the fibre HBA
#include <lcfg/options/qlogic_qla2300.h> #include <lcfg/options/qlogic_qla2322.h> #include <lcfg/options/qlogic_qla2400.h>
Identifying the required SAN volume(s)The Linux SCSI stack, in particular its naming convention for SCSI devices, is extremely simplistic. Basically, it assigns device names to devices in the order that the kernel discovers them at boot (or module load) time, starting from /dev/sda, then /dev/sdb, through to /dev/sdz. This is fine for static configurations where SCSI disks don't come and go but, in a SAN environment where new volumes and storage arrays are routinely added and removed, this simplistic naming convention creates real problems. If a new volume is created on a storage array, this will often result in many existing volumes being discovered after the new volume, and hence being assigned a different SCSI device name. Under FC5, the SCSI stack now creates a soft link to each partition it finds on any attached disk (volume), whether it be local or on a SAN, in the/dev/disk/by-id directory. These soft links are based on the unique SCSI identifiers for the associated volumes. In effect, the SCSI stack is now automatically doing the work that the FC3 udev service did, with no configuration required; with the slight difference that the links are now per partition rather than being per disk (volume).
In the following example, there are five volumes, the first three of which have just one partition, the fourth has two partitions and the fifth has currently no partitions :-
[blurt]ascobie: ls /dev/disk/by-id scsi-36000402001ec048c6a04fc8b00000000 scsi-36000402001ec048c6a04fc8b00000000-part1 scsi-36000402001ec048c6a04fcbb00000000 scsi-36000402001ec048c6a04fcbb00000000-part1 scsi-36000402001ec048c6a05c0a700000000 scsi-36000402001ec048c6a05c0a700000000-part1 scsi-36000402001ec048c6a05c0f000000000 scsi-36000402001ec048c6a05c0f000000000-part1 scsi-36000402001ec048c6a05c0f000000000-part2 scsi-36000402001ec048c6c37286100000000 [blurt]ascobie:In this example, we will create one partition in a volume on the NexSAN ATAbeast atabeast1.inf.ed.ac.uk . The volume's LUN is 1. The partition will be mounted as /disk/testdata .
lsscsi utility.
[bottle]ascobie: lsscsi -k [0:0:6:0] process PE/PV 1x3 SCSI BP 1.1 - [0:2:0:0] disk MegaRAID LD 0 RAID1 34G 412W /dev/sda [1:0:0:0] disk NEXSAN SATABeast Cr41 /dev/sdb [1:0:1:0] disk NEXSAN SATABeast Cr41 - [1:0:1:1] disk NEXSAN SATABeast Cr41 /dev/sdc [1:0:1:3] disk NEXSAN SATABeast Cr41 /dev/sdd [1:0:2:0] disk NEXSAN ATAboy2F 5p41 /dev/sde [1:0:2:1] disk NEXSAN ATAboy2F 5p41 /dev/sdf [1:0:2:2] disk NEXSAN ATAboy2F 5p41 /dev/sdg [1:0:2:3] disk NEXSAN ATAboy2F 5p41 /dev/sdh [1:0:2:4] disk NEXSAN ATAboy2F 5p41 /dev/sdi [1:0:2:5] disk NEXSAN ATAboy2F 5p41 /dev/sdj [1:0:2:6] disk NEXSAN ATAboy2F 5p41 /dev/sdk [1:0:2:7] disk NEXSAN ATAboy2F 5p41 /dev/sdl [1:0:2:8] disk NEXSAN ATAboy2F 5p41 /dev/sdm [1:0:2:9] disk NEXSAN ATAboy2F 5p41 /dev/sdn [1:0:2:10] disk NEXSAN ATAboy2F 5p41 /dev/sdo [1:0:2:11] disk NEXSAN ATAboy2F 5p41 /dev/sdp [1:0:2:12] disk NEXSAN ATAboy2F 5p41 /dev/sdq [1:0:2:13] disk NEXSAN ATAboy2F 5p41 /dev/sdr [1:0:2:14] disk NEXSAN ATAboy2F 5p41 /dev/sds [1:0:3:0] disk NEXSAN ATAbeastF 8r41 /dev/sdt [1:0:3:1] disk NEXSAN ATAbeastF 8r41 /dev/sdu [1:0:3:2] disk NEXSAN ATAbeastF 8r41 /dev/sdv [1:0:3:3] disk NEXSAN ATAbeastF 8r41 /dev/sdw [1:0:3:4] disk NEXSAN ATAbeastF 8r41 /dev/sdx [1:0:3:5] disk NEXSAN ATAbeastF 8r41 /dev/sdy [1:0:3:6] disk NEXSAN ATAbeastF 8r41 /dev/sdz [1:0:3:7] disk NEXSAN ATAbeastF 8r41 /dev/sdaa [1:0:3:8] disk NEXSAN ATAbeastF 8r41 /dev/sdab [1:0:3:9] disk NEXSAN ATAbeastF 8r41 /dev/sdac [1:0:3:10] disk NEXSAN ATAbeastF 8r41 /dev/sdad [1:0:3:11] disk NEXSAN ATAbeastF 8r41 /dev/sdaeThis command lists the SCSI volumes (devices) available. The 4th number in the bracketed value in the first column indicates the LUN of each volume. The last column indicates the current SCSI device name for each volume. We're after the volume with LUN 1 on the ATAbeast, which lsscsi shows to be currently /dev/sdu .
... [1:0:3:1] disk NEXSAN ATAbeastF 8r41 /dev/sdu ...If you have multiple NEXSAN arrays of the same type, you can no longer rely on using lsscsi to directly identify a specific array. Instead you must use the scli -t command to identify your target array using the "Port Name" of the array and map this to the SCSI stack's target id (the third value in the bracketed value in the first column in the output of lsscsi .
[bottle]root: scli -t .. ------------------------------------------------------------------------------ Path : 0 Target : 3 Device ID : 0x8a Port ID : 65-0F-00 Product Vendor : NEXSAN Product ID : ATAbeastF Product Revision : 8r41 Node Name : 20-01-00-04-02-E8-08-5D Port Name : 50-00-40-20-01-E8-08-5D Product Type : Disk Number of LUN(s) : 12 Status : Online ------------------------------------------------------------------------------ ..In this case, the ATAbeast has target id 3 - this maps onto the third value in the bracketed value in the first column in the output of lsscsi .
fdisk /dev/sdu .
mke2fs -j /dev/sdu1 .
[bottle]root: /sbin/scsi_id -g -s /block/sdu 36000402001e8085d6a1882b400000000Note the use of /block/sdu rather than /dev/sdu here.
/dev/disk/by-id by looking for entries of the form scsi-
unique SCSI identifier
.
[bottle]root: ls /dev/disk/by-id/scsi-36000402001e8085d6a1882b400000000* /dev/disk/by-id/scsi-36000402001e8085d6a1882b400000000 /dev/disk/by-id/scsi-36000402001e8085d6a1882b400000000-part1The second entry here (postfixed by part1 ) is the partition (/dev/sdu1 ) on the volume.
Configuring fstab entriesIdeally, one would create a symlink in the/dev directory to the long-winded partition alias. However, this directory is wiped at boot so this is not possible.
!fstab.entries mADD(testdata) fstab.spec_testdata /dev/disk/by-id/scsi-36000402001e8085d6a1882b400000000-part1 fstab.file_testdata /disk/testdata fstab.vfstype_testdata ext3 fstab.passno_testdata 2(Choose another passno for the next partition)
Multipath
129.215.46.190 (81D72EBE ) is mapped to /dev/mpath/36000402001ec048c6a04fcbb00000000
[budapest]root: /sbin/multipath -l ..... 36000402001ec048c6a04fcbb00000000dm-2 NEXSAN,SATABl(81D72EBE) [size=40G][features=0][hwhandler=0] \_ round-robin 0 [prio=0][active] \_ 1:0:0:4 sdd 8:48 [active][undef] \_ 2:0:0:4 sdi 8:128 [active][undef] 36000402001ec048c6c37286100000000dm-1 NEXSAN,SATABl(81D72EBE) [size=10G][features=0][hwhandler=0] \_ round-robin 0 [prio=0][active] \_ 1:0:0:3 sdc 8:32 [active][undef] \_ 2:0:0:3 sdh 8:112 [active][undef] .... [budapest]root:-- AlastairScobie - 27 Jul 2007 |