Difference: FibreAttach (1 vs. 7)

Revision 72013-05-31 - ascobie

Line: 1 to 1
	Attaching a linux box to a SAN
Line: 39 to 39
	Check the profile is compiled and received by machine.
Changed:
< <	Reboot the machine. This reboot will compile the qlogic driver module for the current kernel. It will then trigger an additional reboot (Step not required for SL5)
> >	Reboot the machine. This reboot will install the qlogic firmware RPMs and rebuild the initramfs image. It will then trigger an additional reboot.

Line: 54 to 54
	If you have just added a new volume to an array, you will probably need to get the HBA to rescan the bus to detect it. For QLogic cards, the following should work
Changed:
< <	echo "- - -" > /sys/class/scsi_host/host_id_/scan
> >	echo "- - -" > /sys/class/scsi_host/host{id}/scan
	Though you will have to use the path appropriate to your HBA and port in use. For multipath you will need to do this for each port.
Line: 247 to 247
	Multipath
Changed:
< <	The `<lcfg/options/multipath.h>` header enables multipath support. Note that no multipath configuration is yet possible; a component may be written in the future if it proves necessary. (Remember to use `<dice/options/multipath.h>` under DICE)
> >	The `<lcfg/options/multipath.h>` header enables multipath support. The lcfg-multipath component can be used to configure multipath behaviour : the default configuration should suffice for the majority of situations. (Remember to use `<dice/options/multipath.h>` under DICE)
	If you are using a qlogic HBA and the qlogic supplied driver, you must disable the driver's failover option (The default is to use the stock kernel driver) The storage array must export the LUNs you want to access on all fibre paths. To avoid confusion, it's a very good idea to have the LUNs map to the same volumes on each path; otherwise it is very tricky to identify which volume is which.
Changed:
< <	The multipath system presents the storage volumes and partitions under `/dev/mpath` rather than under `/dev/disk/by-id`, and without a `scsi` prefix. The volumes are still presented under `/dev/disk/by-id`, but you should not use these devices as they are hard-wired to one fibre path and will not failover if that path fails.
> >	The multipath system presents the storage volumes and partitions under `/dev/mapper` rather than under `/dev/disk/by-id`, and without a `scsi` prefix. The volumes are still presented under `/dev/disk/by-id`, but you should not use these devices as they are hard-wired to one fibre path and will not failover if that path fails.
	Use the `/sbin/multipath -l` command to list the available volumes, instead of `/sbin/lsscsi` and `/sbin/scsi_id` used in the above instructions. Where there are many storage arrays connected, it will still be necessary to use `/sbin/scli` to identify the array required (as documented above). When adding a new volume, see above for how to probe for new volume devices. Once `/sbin/lsscsi` has spotted the new volume(s), check with `/sbin/multipath -ll` that the multipath daemon has spotted the new volume(s). Partition the volume using fdisk, as usual. If the volume appears in `/dev/mpath` but the partitions don't, use `kpartx -a -p p /dev/mpath/vol-id` to create the partition entries and new `/dev/dm-{n}` devices.
Line: 283 to 283
	-- AlastairScobie - 27 Jul 2007
Added:
> >	-- Updated 2013-05-31

Revision 62011-01-21 - ascobie

Line: 1 to 1
	Attaching a linux box to a SAN
Line: 252 to 252
	The storage array must export the LUNs you want to access on all fibre paths. To avoid confusion, it's a very good idea to have the LUNs map to the same volumes on each path; otherwise it is very tricky to identify which volume is which. The multipath system presents the storage volumes and partitions under `/dev/mpath` rather than under `/dev/disk/by-id`, and without a `scsi` prefix. The volumes are still presented under `/dev/disk/by-id`, but you should not use these devices as they are hard-wired to one fibre path and will not failover if that path fails. Use the `/sbin/multipath -l` command to list the available volumes, instead of `/sbin/lsscsi` and `/sbin/scsi_id` used in the above instructions. Where there are many storage arrays connected, it will still be necessary to use `/sbin/scli` to identify the array required (as documented above).
Added:
> >	When adding a new volume, see above for how to probe for new volume devices. Once `/sbin/lsscsi` has spotted the new volume(s), check with `/sbin/multipath -ll` that the multipath daemon has spotted the new volume(s). Partition the volume using fdisk, as usual. If the volume appears in `/dev/mpath` but the partitions don't, use `kpartx -a -p p /dev/mpath/vol-id` to create the partition entries and new `/dev/dm-{n}` devices.
	In the following example, LUN 4 of the NexSAN SATABlade with IP address `129.215.46.190` (`81D72EBE`) is mapped to `/dev/mpath/36000402001ec048c6a04fcbb00000000`

Revision 52009-05-12 - ascobie

Line: 1 to 1
	Attaching a linux box to a SAN
Line: 39 to 39
	Check the profile is compiled and received by machine.
Changed:
< <	Reboot the machine. This reboot will compile the qlogic driver module for the current kernel. It will then trigger an additional reboot.
> >	Reboot the machine. This reboot will compile the qlogic driver module for the current kernel. It will then trigger an additional reboot (Step not required for SL5)

Line: 54 to 54
	If you have just added a new volume to an array, you will probably need to get the HBA to rescan the bus to detect it. For QLogic cards, the following should work
Changed:
< <	echo "scsi-qlascan" > /proc/scsi/qla2xxx/1 echo "- - -" > /sys/class/scsi_host/host/scan or rescan-scsi-bus.sh -c -l -w
> >	echo "- - -" > /sys/class/scsi_host/host_id_/scan

Changed:
< <	Though you will have to use the path appropriate to your HBA and port in use. And obtain the `rescan-scsi-bus.sh` script from the web.
> >	Though you will have to use the path appropriate to your HBA and port in use. For multipath you will need to do this for each port.

Changed:
< <	Under FC5, the SCSI stack now creates a soft link to each partition it finds on any attached disk (volume), whether it be local or on a SAN, in the `/dev/disk/by-id` directory. These soft links are based on the unique SCSI identifiers for the associated volumes. In effect, the SCSI stack is now automatically doing the work that the FC3 udev service did, with no configuration required; with the slight difference that the links are now per partition rather than being per disk (volume).
> >	Since FC5, the SCSI stack creates a soft link to each partition it finds on any attached disk (volume), whether it be local or on a SAN, in the `/dev/disk/by-id` directory. These soft links are based on the unique SCSI identifiers for the associated volumes. In effect, the SCSI stack is now automatically doing the work that the FC3 udev service did, with no configuration required; with the slight difference that the links are now per partition rather than being per disk (volume).
	In the following example, there are five volumes, the first three of which have just one partition, the fourth has two partitions and the fifth has currently no partitions :-
Line: 250 to 248
	The `<lcfg/options/multipath.h>` header enables multipath support. Note that no multipath configuration is yet possible; a component may be written in the future if it proves necessary. (Remember to use `<dice/options/multipath.h>` under DICE)
Changed:
< <	If you are using a qlogic HBA, you must disable the driver's failover option (This is disabled by default for DICE)
> >	If you are using a qlogic HBA and the qlogic supplied driver, you must disable the driver's failover option (The default is to use the stock kernel driver)
	The storage array must export the LUNs you want to access on all fibre paths. To avoid confusion, it's a very good idea to have the LUNs map to the same volumes on each path; otherwise it is very tricky to identify which volume is which. The multipath system presents the storage volumes and partitions under `/dev/mpath` rather than under `/dev/disk/by-id`, and without a `scsi` prefix. The volumes are still presented under `/dev/disk/by-id`, but you should not use these devices as they are hard-wired to one fibre path and will not failover if that path fails. Use the `/sbin/multipath -l` command to list the available volumes, instead of `/sbin/lsscsi` and `/sbin/scsi_id` used in the above instructions. Where there are many storage arrays connected, it will still be necessary to use `/sbin/scli` to identify the array required (as documented above).

Revision 42009-04-29 - neilb

Line: 1 to 1
	Attaching a linux box to a SAN
Line: 55 to 55
	If you have just added a new volume to an array, you will probably need to get the HBA to rescan the bus to detect it. For QLogic cards, the following should work echo "scsi-qlascan" > /proc/scsi/qla2xxx/1
Changed:
< <	rescan-scsi-bus.sh -c -l -w
> >	echo "- - -" > /sys/class/scsi_host/host/scan or rescan-scsi-bus.sh -c -l -w

Changed:
< <	Though you will have to use the path appropriate to your HBA and port in use. And obtains the `rescan-scsi-bus.sh` script from the web.
> >	Though you will have to use the path appropriate to your HBA and port in use. And obtain the `rescan-scsi-bus.sh` script from the web.
	Under FC5, the SCSI stack now creates a soft link to each partition it finds on any attached disk (volume), whether it be local or on a SAN, in the `/dev/disk/by-id` directory. These soft links are based on the unique SCSI identifiers for the associated volumes. In effect, the SCSI stack is now automatically doing the work that the FC3 udev service did, with no configuration required; with the slight difference that the links are now per partition rather than being per disk (volume).

Revision 32009-02-10 - neilb

Line: 1 to 1
Deleted:
< <
	Attaching a linux box to a SAN
Line: 53 to 52
	This is fine for static configurations where SCSI disks don't come and go but, in a SAN environment where new volumes and storage arrays are routinely added and removed, this simplistic naming convention creates real problems. If a new volume is created on a storage array, this will often result in many existing volumes being discovered after the new volume, and hence being assigned a different SCSI device name.
Added:
> >	If you have just added a new volume to an array, you will probably need to get the HBA to rescan the bus to detect it. For QLogic cards, the following should work echo "scsi-qlascan" > /proc/scsi/qla2xxx/1 rescan-scsi-bus.sh -c -l -w Though you will have to use the path appropriate to your HBA and port in use. And obtains the `rescan-scsi-bus.sh` script from the web.
	Under FC5, the SCSI stack now creates a soft link to each partition it finds on any attached disk (volume), whether it be local or on a SAN, in the `/dev/disk/by-id` directory. These soft links are based on the unique SCSI identifiers for the associated volumes. In effect, the SCSI stack is now automatically doing the work that the FC3 udev service did, with no configuration required; with the slight difference that the links are now per partition rather than being per disk (volume). In the following example, there are five volumes, the first three of which have just one partition, the fourth has two partitions and the fifth has currently no partitions :-

Revision 22009-01-05 - ascobie

Line: 1 to 1
Deleted:
< <
	Attaching a linux box to a SAN
Line: 268 to 268

Added:
> >	Jan 09 - Note that for Nexsan products, you may have to add the following to /etc/scsi_id.config in order that the multipath software identifies the two paths as being to the same volume. vendor="NEXSAN",options=-p 0x80

Added:
> >
	-- AlastairScobie - 27 Jul 2007

Revision 12008-02-07 - squinney

Line: 1 to 1

Line: 1 to 1
Added:
> >	Attaching a linux box to a SAN It is absolutely essential that all fibre connections are removed from a machine during installation Introduction It is absolutely essential that all fibre connections are removed from a machine during installation Only the Qlogic 23XX and 24xx series cards are currently supported. Multipath notes There are three stages to configuring a machine to use a SAN :- Configuring the fibre HBA (Host Bus Adapter) Identifying the required SAN volume(s) Configuring the fstab entries to mount the SAN Volume(s) Configuring the fibre HBA Add one of the following include files to your machine's profile (under DICE, remember to use `<dice/options...>`) #include <lcfg/options/qlogic_qla2300.h> #include <lcfg/options/qlogic_qla2322.h> #include <lcfg/options/qlogic_qla2400.h> Check the profile is compiled and received by machine. Reboot the machine. This reboot will compile the qlogic driver module for the current kernel. It will then trigger an additional reboot. Identifying the required SAN volume(s) The Linux SCSI stack, in particular its naming convention for SCSI devices, is extremely simplistic. Basically, it assigns device names to devices in the order that the kernel discovers them at boot (or module load) time, starting from /dev/sda, then /dev/sdb, through to /dev/sdz. This is fine for static configurations where SCSI disks don't come and go but, in a SAN environment where new volumes and storage arrays are routinely added and removed, this simplistic naming convention creates real problems. If a new volume is created on a storage array, this will often result in many existing volumes being discovered after the new volume, and hence being assigned a different SCSI device name. Under FC5, the SCSI stack now creates a soft link to each partition it finds on any attached disk (volume), whether it be local or on a SAN, in the `/dev/disk/by-id` directory. These soft links are based on the unique SCSI identifiers for the associated volumes. In effect, the SCSI stack is now automatically doing the work that the FC3 udev service did, with no configuration required; with the slight difference that the links are now per partition rather than being per disk (volume). In the following example, there are five volumes, the first three of which have just one partition, the fourth has two partitions and the fifth has currently no partitions :- [blurt]ascobie: ls /dev/disk/by-id scsi-36000402001ec048c6a04fc8b00000000 scsi-36000402001ec048c6a04fc8b00000000-part1 scsi-36000402001ec048c6a04fcbb00000000 scsi-36000402001ec048c6a04fcbb00000000-part1 scsi-36000402001ec048c6a05c0a700000000 scsi-36000402001ec048c6a05c0a700000000-part1 scsi-36000402001ec048c6a05c0f000000000 scsi-36000402001ec048c6a05c0f000000000-part1 scsi-36000402001ec048c6a05c0f000000000-part2 scsi-36000402001ec048c6c37286100000000 [blurt]ascobie: In this example, we will create one partition in a volume on the NexSAN ATAbeast `atabeast1.inf.ed.ac.uk`. The volume's LUN is 1. The partition will be mounted as `/disk/testdata`. Determine current SCSI device name for the volume. The first task is to determine what device name the SCSI stack has currently allocated for the volume. The easiest way to do this is to use the `lsscsi` utility. [bottle]ascobie: lsscsi -k [0:0:6:0] process PE/PV 1x3 SCSI BP 1.1 - [0:2:0:0] disk MegaRAID LD 0 RAID1 34G 412W /dev/sda [1:0:0:0] disk NEXSAN SATABeast Cr41 /dev/sdb [1:0:1:0] disk NEXSAN SATABeast Cr41 - [1:0:1:1] disk NEXSAN SATABeast Cr41 /dev/sdc [1:0:1:3] disk NEXSAN SATABeast Cr41 /dev/sdd [1:0:2:0] disk NEXSAN ATAboy2F 5p41 /dev/sde [1:0:2:1] disk NEXSAN ATAboy2F 5p41 /dev/sdf [1:0:2:2] disk NEXSAN ATAboy2F 5p41 /dev/sdg [1:0:2:3] disk NEXSAN ATAboy2F 5p41 /dev/sdh [1:0:2:4] disk NEXSAN ATAboy2F 5p41 /dev/sdi [1:0:2:5] disk NEXSAN ATAboy2F 5p41 /dev/sdj [1:0:2:6] disk NEXSAN ATAboy2F 5p41 /dev/sdk [1:0:2:7] disk NEXSAN ATAboy2F 5p41 /dev/sdl [1:0:2:8] disk NEXSAN ATAboy2F 5p41 /dev/sdm [1:0:2:9] disk NEXSAN ATAboy2F 5p41 /dev/sdn [1:0:2:10] disk NEXSAN ATAboy2F 5p41 /dev/sdo [1:0:2:11] disk NEXSAN ATAboy2F 5p41 /dev/sdp [1:0:2:12] disk NEXSAN ATAboy2F 5p41 /dev/sdq [1:0:2:13] disk NEXSAN ATAboy2F 5p41 /dev/sdr [1:0:2:14] disk NEXSAN ATAboy2F 5p41 /dev/sds [1:0:3:0] disk NEXSAN ATAbeastF 8r41 /dev/sdt [1:0:3:1] disk NEXSAN ATAbeastF 8r41 /dev/sdu [1:0:3:2] disk NEXSAN ATAbeastF 8r41 /dev/sdv [1:0:3:3] disk NEXSAN ATAbeastF 8r41 /dev/sdw [1:0:3:4] disk NEXSAN ATAbeastF 8r41 /dev/sdx [1:0:3:5] disk NEXSAN ATAbeastF 8r41 /dev/sdy [1:0:3:6] disk NEXSAN ATAbeastF 8r41 /dev/sdz [1:0:3:7] disk NEXSAN ATAbeastF 8r41 /dev/sdaa [1:0:3:8] disk NEXSAN ATAbeastF 8r41 /dev/sdab [1:0:3:9] disk NEXSAN ATAbeastF 8r41 /dev/sdac [1:0:3:10] disk NEXSAN ATAbeastF 8r41 /dev/sdad [1:0:3:11] disk NEXSAN ATAbeastF 8r41 /dev/sdae This command lists the SCSI volumes (devices) available. The 4th number in the bracketed value in the first column indicates the LUN of each volume. The last column indicates the current SCSI device name for each volume. We're after the volume with LUN 1 on the ATAbeast, which `lsscsi` shows to be currently `/dev/sdu`. ... [1:0:3:1] disk NEXSAN ATAbeastF 8r41 /dev/sdu ... If you have multiple NEXSAN arrays of the same type, you can no longer rely on using `lsscsi` to directly identify a specific array. Instead you must use the `scli -t` command to identify your target array using the "Port Name" of the array and map this to the SCSI stack's target id (the third value in the bracketed value in the first column in the output of `lsscsi`. [bottle]root: scli -t .. ------------------------------------------------------------------------------ Path : 0 Target : 3 Device ID : 0x8a Port ID : 65-0F-00 Product Vendor : NEXSAN Product ID : ATAbeastF Product Revision : 8r41 Node Name : 20-01-00-04-02-E8-08-5D Port Name : 50-00-40-20-01-E8-08-5D Product Type : Disk Number of LUN(s) : 12 Status : Online ------------------------------------------------------------------------------ .. In this case, the ATAbeast has target id 3 - this maps onto the third value in the bracketed value in the first column in the output of `lsscsi`. Partition the volume Partition the volume as you would any SCSI device. In this example by calling `fdisk /dev/sdu`. Create any filesystems for new partitions. Create any filesystems. In this example by calling, eg, `mke2fs -j /dev/sdu1`. Determine the unique SCSI identifier for the volume. [bottle]root: /sbin/scsi_id -g -s /block/sdu 36000402001e8085d6a1882b400000000 Note the use of `/block/sdu` rather than `/dev/sdu` here. Check entries in `/dev/disk/by-id` Check that there are entries in `/dev/disk/by-id` by looking for entries of the form `scsi-` unique SCSI identifier . [bottle]root: ls /dev/disk/by-id/scsi-36000402001e8085d6a1882b400000000* /dev/disk/by-id/scsi-36000402001e8085d6a1882b400000000 /dev/disk/by-id/scsi-36000402001e8085d6a1882b400000000-part1 The second entry here (postfixed by `part1`) is the partition (`/dev/sdu1`) on the volume. Configuring fstab entries Ideally, one would create a symlink in the `/dev` directory to the long-winded partition alias. However, this directory is wiped at boot so this is not possible. Add `fstab.entries` entries to add to `/etc/fstab` !fstab.entries mADD(testdata) fstab.spec_testdata /dev/disk/by-id/scsi-36000402001e8085d6a1882b400000000-part1 fstab.file_testdata /disk/testdata fstab.vfstype_testdata ext3 fstab.passno_testdata 2 (Choose another passno for the next partition) reboot (after checking resources shipped) Multipath The `<lcfg/options/multipath.h>` header enables multipath support. Note that no multipath configuration is yet possible; a component may be written in the future if it proves necessary. (Remember to use `<dice/options/multipath.h>` under DICE) If you are using a qlogic HBA, you must disable the driver's failover option (This is disabled by default for DICE) The storage array must export the LUNs you want to access on all fibre paths. To avoid confusion, it's a very good idea to have the LUNs map to the same volumes on each path; otherwise it is very tricky to identify which volume is which. The multipath system presents the storage volumes and partitions under `/dev/mpath` rather than under `/dev/disk/by-id`, and without a `scsi` prefix. The volumes are still presented under `/dev/disk/by-id`, but you should not use these devices as they are hard-wired to one fibre path and will not failover if that path fails. Use the `/sbin/multipath -l` command to list the available volumes, instead of `/sbin/lsscsi` and `/sbin/scsi_id` used in the above instructions. Where there are many storage arrays connected, it will still be necessary to use `/sbin/scli` to identify the array required (as documented above). In the following example, LUN 4 of the NexSAN SATABlade with IP address `129.215.46.190` (`81D72EBE`) is mapped to `/dev/mpath/36000402001ec048c6a04fcbb00000000` [budapest]root: /sbin/multipath -l ..... 36000402001ec048c6a04fcbb00000000dm-2 NEXSAN,SATABl(81D72EBE) [size=40G][features=0][hwhandler=0] \_ round-robin 0 [prio=0][active] \_ 1:0:0:4 sdd 8:48 [active][undef] \_ 2:0:0:4 sdi 8:128 [active][undef] 36000402001ec048c6c37286100000000dm-1 NEXSAN,SATABl(81D72EBE) [size=10G][features=0][hwhandler=0] \_ round-robin 0 [prio=0][active] \_ 1:0:0:3 sdc 8:32 [active][undef] \_ 2:0:0:3 sdh 8:112 [active][undef] .... [budapest]root: -- AlastairScobie - 27 Jul 2007

Added:

>
>

Attaching a linux box to a SAN

It is absolutely essential that all fibre connections are removed from a machine during installation

Introduction

It is absolutely essential that all fibre connections are removed from a machine during installation
Only the Qlogic 23XX and 24xx series cards are currently supported.
Multipath notes

There are three stages to configuring a machine to use a SAN :-

Configuring the fibre HBA (Host Bus Adapter)
Identifying the required SAN volume(s)
Configuring the fstab entries to mount the SAN Volume(s)

Configuring the fibre HBA

Add one of the following include files to your machine's profile (under DICE, remember to use <dice/options...>)


#include <lcfg/options/qlogic_qla2300.h>
#include <lcfg/options/qlogic_qla2322.h>
#include <lcfg/options/qlogic_qla2400.h>

Check the profile is compiled and received by machine.
Reboot the machine. This reboot will compile the qlogic driver module for the current kernel. It will then trigger an additional reboot.

Identifying the required SAN volume(s)

The Linux SCSI stack, in particular its naming convention for SCSI devices, is extremely simplistic. Basically, it assigns device names to devices in the order that the kernel discovers them at boot (or module load) time, starting from /dev/sda, then /dev/sdb, through to /dev/sdz.

This is fine for static configurations where SCSI disks don't come and go but, in a SAN environment where new volumes and storage arrays are routinely added and removed, this simplistic naming convention creates real problems. If a new volume is created on a storage array, this will often result in many existing volumes being discovered after the new volume, and hence being assigned a different SCSI device name.

Under FC5, the SCSI stack now creates a soft link to each partition it finds on any attached disk (volume), whether it be local or on a SAN, in the /dev/disk/by-id directory. These soft links are based on the unique SCSI identifiers for the associated volumes. In effect, the SCSI stack is now automatically doing the work that the FC3 udev service did, with no configuration required; with the slight difference that the links are now per partition rather than being per disk (volume).

In the following example, there are five volumes, the first three of which have just one partition, the fourth has two partitions and the fifth has currently no partitions :-


[blurt]ascobie: ls /dev/disk/by-id
scsi-36000402001ec048c6a04fc8b00000000
scsi-36000402001ec048c6a04fc8b00000000-part1
scsi-36000402001ec048c6a04fcbb00000000
scsi-36000402001ec048c6a04fcbb00000000-part1
scsi-36000402001ec048c6a05c0a700000000
scsi-36000402001ec048c6a05c0a700000000-part1
scsi-36000402001ec048c6a05c0f000000000
scsi-36000402001ec048c6a05c0f000000000-part1
scsi-36000402001ec048c6a05c0f000000000-part2
scsi-36000402001ec048c6c37286100000000
[blurt]ascobie:

In this example, we will create one partition in a volume on the NexSAN ATAbeast atabeast1.inf.ed.ac.uk. The volume's LUN is 1. The partition will be mounted as /disk/testdata.

Determine current SCSI device name for the volume.

The first task is to determine what device name the SCSI stack has currently allocated for the volume. The easiest way to do this is to use the lsscsi utility.


[bottle]ascobie: lsscsi -k
[0:0:6:0]    process PE/PV    1x3 SCSI BP      1.1   -       
[0:2:0:0]    disk    MegaRAID LD 0 RAID1   34G 412W  /dev/sda
[1:0:0:0]    disk    NEXSAN   SATABeast        Cr41  /dev/sdb
[1:0:1:0]    disk    NEXSAN   SATABeast        Cr41  -       
[1:0:1:1]    disk    NEXSAN   SATABeast        Cr41  /dev/sdc
[1:0:1:3]    disk    NEXSAN   SATABeast        Cr41  /dev/sdd
[1:0:2:0]    disk    NEXSAN   ATAboy2F         5p41  /dev/sde
[1:0:2:1]    disk    NEXSAN   ATAboy2F         5p41  /dev/sdf
[1:0:2:2]    disk    NEXSAN   ATAboy2F         5p41  /dev/sdg
[1:0:2:3]    disk    NEXSAN   ATAboy2F         5p41  /dev/sdh
[1:0:2:4]    disk    NEXSAN   ATAboy2F         5p41  /dev/sdi
[1:0:2:5]    disk    NEXSAN   ATAboy2F         5p41  /dev/sdj
[1:0:2:6]    disk    NEXSAN   ATAboy2F         5p41  /dev/sdk
[1:0:2:7]    disk    NEXSAN   ATAboy2F         5p41  /dev/sdl
[1:0:2:8]    disk    NEXSAN   ATAboy2F         5p41  /dev/sdm
[1:0:2:9]    disk    NEXSAN   ATAboy2F         5p41  /dev/sdn
[1:0:2:10]   disk    NEXSAN   ATAboy2F         5p41  /dev/sdo
[1:0:2:11]   disk    NEXSAN   ATAboy2F         5p41  /dev/sdp
[1:0:2:12]   disk    NEXSAN   ATAboy2F         5p41  /dev/sdq
[1:0:2:13]   disk    NEXSAN   ATAboy2F         5p41  /dev/sdr
[1:0:2:14]   disk    NEXSAN   ATAboy2F         5p41  /dev/sds
[1:0:3:0]    disk    NEXSAN   ATAbeastF        8r41  /dev/sdt
[1:0:3:1]    disk    NEXSAN   ATAbeastF        8r41  /dev/sdu
[1:0:3:2]    disk    NEXSAN   ATAbeastF        8r41  /dev/sdv
[1:0:3:3]    disk    NEXSAN   ATAbeastF        8r41  /dev/sdw
[1:0:3:4]    disk    NEXSAN   ATAbeastF        8r41  /dev/sdx
[1:0:3:5]    disk    NEXSAN   ATAbeastF        8r41  /dev/sdy
[1:0:3:6]    disk    NEXSAN   ATAbeastF        8r41  /dev/sdz
[1:0:3:7]    disk    NEXSAN   ATAbeastF        8r41  /dev/sdaa
[1:0:3:8]    disk    NEXSAN   ATAbeastF        8r41  /dev/sdab
[1:0:3:9]    disk    NEXSAN   ATAbeastF        8r41  /dev/sdac
[1:0:3:10]   disk    NEXSAN   ATAbeastF        8r41  /dev/sdad
[1:0:3:11]   disk    NEXSAN   ATAbeastF        8r41  /dev/sdae

This command lists the SCSI volumes (devices) available.

The 4th number in the bracketed value in the first column indicates the LUN of each volume. The last column indicates the current SCSI device name for each volume.

We're after the volume with LUN 1 on the ATAbeast, which lsscsi shows to be currently /dev/sdu.


...
[1:0:3:1]    disk    NEXSAN   ATAbeastF        8r41  /dev/sdu
...

If you have multiple NEXSAN arrays of the same type, you can no longer rely on using lsscsi to directly identify a specific array. Instead you must use the scli -t command to identify your target array using the "Port Name" of the array and map this to the SCSI stack's target id (the third value in the bracketed value in the first column in the output of lsscsi.


[bottle]root: scli -t
..
------------------------------------------------------------------------------
Path                       : 0
Target                     : 3
Device ID                  : 0x8a
Port ID                    : 65-0F-00
Product Vendor             : NEXSAN  
Product ID                 : ATAbeastF       
Product Revision           : 8r41
Node Name                  : 20-01-00-04-02-E8-08-5D
Port Name                  : 50-00-40-20-01-E8-08-5D
Product Type               : Disk
Number of LUN(s)           : 12
Status                     : Online
------------------------------------------------------------------------------
..

In this case, the ATAbeast has target id 3 - this maps onto the third value in the bracketed value in the first column in the output of lsscsi.

Partition the volume

Partition the volume as you would any SCSI device. In this example by calling fdisk /dev/sdu.

Create any filesystems for new partitions.

Create any filesystems. In this example by calling, eg, mke2fs -j /dev/sdu1.

Determine the unique SCSI identifier for the volume.


[bottle]root: /sbin/scsi_id -g -s /block/sdu
36000402001e8085d6a1882b400000000

Note the use of /block/sdu rather than /dev/sdu here.

Check entries in /dev/disk/by-id

Check that there are entries in /dev/disk/by-id by looking for entries of the form scsi- unique SCSI identifier .


[bottle]root: ls  /dev/disk/by-id/scsi-36000402001e8085d6a1882b400000000*
/dev/disk/by-id/scsi-36000402001e8085d6a1882b400000000
/dev/disk/by-id/scsi-36000402001e8085d6a1882b400000000-part1

The second entry here (postfixed by part1) is the partition (/dev/sdu1) on the volume.

Configuring fstab entries

Ideally, one would create a symlink in the /dev directory to the long-winded partition alias. However, this directory is wiped at boot so this is not possible.

Add fstab.entries entries to add to /etc/fstab


!fstab.entries          mADD(testdata)
fstab.spec_testdata     /dev/disk/by-id/scsi-36000402001e8085d6a1882b400000000-part1
fstab.file_testdata     /disk/testdata
fstab.vfstype_testdata  ext3
fstab.passno_testdata   2

(Choose another passno for the next partition)

reboot (after checking resources shipped)

Multipath

The <lcfg/options/multipath.h> header enables multipath support. Note that no multipath configuration is yet possible; a component may be written in the future if it proves necessary. (Remember to use <dice/options/multipath.h> under DICE)
If you are using a qlogic HBA, you must disable the driver's failover option (This is disabled by default for DICE)
The storage array must export the LUNs you want to access on all fibre paths. To avoid confusion, it's a very good idea to have the LUNs map to the same volumes on each path; otherwise it is very tricky to identify which volume is which.
The multipath system presents the storage volumes and partitions under /dev/mpath rather than under /dev/disk/by-id, and without a scsi prefix. The volumes are still presented under /dev/disk/by-id, but you should not use these devices as they are hard-wired to one fibre path and will not failover if that path fails.
Use the /sbin/multipath -l command to list the available volumes, instead of /sbin/lsscsi and /sbin/scsi_id used in the above instructions. Where there are many storage arrays connected, it will still be necessary to use /sbin/scli to identify the array required (as documented above).

In the following example, LUN 4 of the NexSAN SATABlade with IP address 129.215.46.190 (81D72EBE) is mapped to /dev/mpath/36000402001ec048c6a04fcbb00000000


[budapest]root: /sbin/multipath -l
.....
36000402001ec048c6a04fcbb00000000dm-2 NEXSAN,SATABl(81D72EBE)
[size=40G][features=0][hwhandler=0]
\_ round-robin 0 [prio=0][active]
 \_ 1:0:0:4  sdd 8:48  [active][undef]
 \_ 2:0:0:4  sdi 8:128 [active][undef]
36000402001ec048c6c37286100000000dm-1 NEXSAN,SATABl(81D72EBE)
[size=10G][features=0][hwhandler=0]
\_ round-robin 0 [prio=0][active]
 \_ 1:0:0:3  sdc 8:32  [active][undef]
 \_ 2:0:0:3  sdh 8:112 [active][undef]
....
[budapest]root:

-- AlastairScobie - 27 Jul 2007

View topic | History: r7 < r6 < r5 < r4 | More topic actions...