Scanning LUNs

Discovering storage on the host

LUNs on your storage system appear as disks to the Linux host. When you add new LUNs, you must manually rescan the associated disks to discover them. The host does not automatically discover new LUNs.

Steps

Scan the LUNs by running the # rescan-scsi-bus.sh command from a terminal window on the host.

This command works only if sg3_utils is installed. Alternatively, you can reboot the host or manually scan each SCSI host.

# echo “- - -“

> /sys/class/scsi_host/host3/scan

Verify disk discovery by running the # sanlun lun show -p command.

Example

# sanlun lun show

controller(7mode/E-Series)

<column headings deleted for readability>

--------------------------------------------------------

ictm1619s01c01-SRP /dev/sddi host14 SRP E-Series

ictm1619s01c01-SRP /dev/sddh host14 SRP E-Series

ictm1619s01c01-SRP /dev/sddf host14 SRP E-Series

=====================

Resolution

Yes, as of Red Hat Enterprise Linux 5.0, it is possible to make changes to the SCSI I/O subsystem without rebooting. There are a number of methods that can be used to accomplish this. Some perform changes explicitly, one device at a time, or one bus at a time. Others are potentially more disruptive, causing bus resets, or potentially causing a large number of configuration changes at the same time. If the less-disruptive methods are used, then it is not necessary to pause I/O while the change is being made. If one of the more disruptive methods are used then, as a precaution, it is necessary to pause I/O on each of the SCSI buses that are involved in the change.

Removing a Storage Device

Before removing access to the storage device itself, you may want to copy data from the device. When that is done, then you must stop and flush all I/O, and remove all operating system references to the device, as described below. If this is a multipath device then you must do this for the multipath pseudo device, and each of the identifiers that represent a path to the device.

Removal of a storage device is not recommended when the system is under memory pressure, since the I/O flush will add to the load. To determine the level of memory pressure run the command:

# vmstat 1 100

Device removal is not recommended if swapping is active (non-zero "si" and "so" columns in the vmstat output), and free memory is less than 5% of the total memory in more than 10 samples per 100. (The total memory can be obtained with the "free" command.)

The general procedure for removing all access to a device is as follows:

Close all users (applications, etc.) of the device. Copy (backup) data from the device, as needed.

Use umount to unmount any file systems that mounted the device.

Remove the device from any md and LVM volume that is using it.

If the device is a member of an LVM Volume group, then it may be necessary to move data off the device using the pvmove command, then use the vgreduce command to remove the physical volume, and (optionally) pvremove to remove the LVM metadata from the disk.

If you are removing a multipath device, run multipath -l and take note of all the paths to the device. When this has been done, use the command below to remove the multipath device where multipath-device name may be mpath0, for example.

# multipath -f multipath-device

NOTE: This command may fail with "map in use" if the multipath device is still in use (for example, a partition is on the device).

Use the following command to flush any pending or outstanding I/O to all paths to the device. This is particularly important for devices, where there is no umount or vgreduce operation to cause an I/O flush.

# blockdev --flushbufs device

Remove any reference to the device's path-based name, like /dev/sd or /dev/disk/by-path or the major:minor number, in applications, scripts, or utilities on the system. This is important to ensure that a different device, when added in the future, will not be mistaken for the current device.

The final step is to remove each path to the device from the SCSI subsystem. Use the command below to remove a path where device-name may be sde, for example. Perform this step for each path to the device.

# echo 1 > /sys/block/device-name/device/delete

Another variation of this operation is:

# echo 1 > /sys/class/scsi_device/h:c:t:l/device/delete

Where h is the HBA number, c is the channel on the HBA, t is the SCSI target ID, and l is the LUN.

You can determine the device-name and the h:c:t:l for a device from various commands, such as lsscsi, scsi_id, multipath -l, and ls -l /dev/disk/by-*

One more variation of this operation when the device to be deleted has a known scsi generic special file name:

# echo 1 > /sys/class/scsi_generic/sg-name/device/delete

where sg-name may be sg18, for example. The output of lsscsi -g can show the scsi generic names for different devices, along with the corresponding scsi addresses.

Lastly, the slightly different path used in deletion of a tape device:

# echo 1 > /sys/class/scsi_tape/tape-name/device/delete

If each of the steps above are followed, then a device can safely be removed from a running system. It is not necessary to stop I/O to other devices while this is done.

Other procedures, such as the physical removal of the device, followed by a rescan of the SCSI bus using rescan-scsi-bus or issue_lip to cause the operating system state to be updated to reflect the change, are not recommended. This may cause delays due to I/O timeouts, and devices may be removed/replaced unexpectedly. If it is necessary to perform a rescan of an interconnect, it must be done while I/O is paused.

As a point of clarification: The "physical removal" of a device referenced above means that rescan will end up pausing and going through recovery efforts up to and including resetting the adapter, which ends up causing the same side effects that issue_lip does. This is because the device is still known to the host. The proper procedure for removal would be to remove the device from the operating system first before physically removing the device in storage to avoid error recovery kicking in on the non-responsive device, etc.

Adding a Storage Device or a Path

When adding a device, be aware that the path-based device name (the “sd” name, the major:minor number, and /dev/disk/by-path name, for example) that the system assigns to the new device may have been previously in use by a device that has since been removed. Ensure that all old references to the path-based device name have been removed. Otherwise the new device may be mistaken for the old device.

The first step is to physically enable access to the new storage device, or a new path to an existing device. This may involve installing cables, disks, and vendor-specific commands at the FC or iSCSI storage server. When you do this, take note of the LUN value for the new storage that will be presented to your host.

Next, make the operating system aware of the new storage device, or path to an existing device. The preferred command is:

# echo "c t l" > /sys/class/scsi_host/hostH/scan

where H is the HBA number, c is the channel on the HBA, t is the SCSI target ID, and l is the LUN.

You can determine the H,c,t by referring to another device that is already configured on the same path as the new device. This can be done with commands such as lsscsi, scsi_id, multipath -l, and ls -l /dev/disk/by-*. This information, plus the LUN number of the new device, can be used as shown above to probe and configure that path to the new device.

Note: In some Fibre Channel hardware configurations, when a new LUN is created on the RAID array it may not be visible to the operating system until after a LIP (Loop Initialization Protocol) operation is performed. Refer to the manuals for instructions on how to do this. If a LIP is required, it will be necessary to stop I/O while this operation is done.

As of Red Hat Enterprise Linux 5.6, it is also possible to use the wildcard character "-" in place of c, t and/or l in the command shown above. In this case, it is not necessary to stop I/O while this command executes. In versions prior to 5.6, the use of wildcards in this command requires that I/O be paused as a precaution.

After adding all the SCSI paths to the device, execute the multipath command, and check to see that the device has been properly configured. At this point, the device is available to be added to md, LVM, mkfs, or mount, for example.

Other commands, that cause a SCSI bus reset, LIP, or a system-wide rescan, that may result in multiple add/remove/replace operations, are not recommended. If these commands are used, then I/O to the effected SCSI buses must be paused and flushed prior to the operation.

As of release 5.4, a script called /usr/bin/rescan-scsi-bus.sh is available as part of the sg3_utils package. This can make rescan operations easier. Additionally, cleanup of unused devices can be made easier by using the -r flag which enables device removal.

Diagnostic Steps

If scanning for new devices doesn't work -- that is the expected lun or luns were not seen and new devices were not created -- then obtaining the output from sg_luns from the sg3_utils package may help understand the issue.

There are several common results after scanning for a newly added lun:

new lun added within storage and is seen/discovered by a scan as expected,

new lun added to existing list of luns within storage (lun 9 is added to lun 0-8 for example), is not seen by host, and no errors are logged,

new lun added to existing list of luns within storage, is not seen by host, but errors are logged during scanning such as:

scsi: host 0 channel 0 id 2 lun16643 has a LUN larger than allowed by the host adapter,

new lun added to newly zoned/presented storage port (lun 0 is presented for example) and is not seen.

new lun added to existing list of luns, is not seen, no errors logged

In the case where the new lun isn't seen and no errors are being logged, typically the cause is the lun isn't being returned by storage. The list of luns behind a storage port is obtained by the kernel by sending a scsi REPORT LUNS command to a device on the storage port (the target rather than the lun answers this command request, so all luns behind a storage target port will return the same lun list). We can obtain the same list from storage using the sg_luns command.

For example, we have two HBAs connected to two storage ports each with the following luns.

$ lsscsi

[0:0:0:0] disk XYZ TT3250310AS N4.A /dev/sda

[0:0:0:1] disk XYZ TT3250311AS N4.A /dev/sdb

[0:0:1:0] disk XYZ TT3250312AS N4.A /dev/sdc

[1:0:0:0] disk XYZ TT3250315AS N4.A /dev/sdd

[1:0:1:0] disk XYZ TT3250314AS N4.A /dev/sde

We've added and presented lun '2' to this host within storage. It is expected the new lun will show up as [0:0:0:2] after a scan but no new devices appear. To see the same information the kernel sees during a lun rescan we can run the following commands so that each of the storage ports ([0:0:0:*] [0:0:1:*] [1:0:0:*] and [1:0:1:*]) are checked. This way, if we don't know specifically which storage port the new lun is suppose to show up behind, we've covered all of the possibilities.

$ sg_luns -d -vv /dev/sda

$ sg_luns -d -vv /dev/sdc

$ sg_luns -d -vv /dev/sdd

$ sg_luns -d -vv /dev/sde

The output from the above commands is examined to see if lun 2 has shown up as expected. For example if we expect the new lun to show up as 0:0:0:2, we'd query another device with the same 0:0:0:* scsi address. Such devices all are behind the same storage target. In this example, either sda or sdb can be used.

$ sg_luns -d -vv /dev/sda

:

Report luns [select_report=0]:

0000000000000000

Peripheral device addressing: lun=0

0001000000000000

Peripheral device addressing: lun=1

In the above case, the expected lun 2 was not returned by storage so the host can't see it during discovered. Check to make sure all the storage vendor specific steps needed to present the lun from storage to our host hba were followed.

new lun added to existing list of luns, is not seen, but errors are logged

If the error message has to do with lun number being too big, but the lun added shouldn't have a lun id greater than 255, then the likely problem is that storage is returning the lun numbers in a format other the 00b.

In this case the sg_luns output might include 01b format luns which aren't supported by the linux kernel. This results in the incorrect decode of the lun number. Storage configuration needs to be changed to return the luns within the correct 00b format.

$ sg_luns -d -vv /dev/sda

:

Report luns [select_report=0]:

4000000000000000 01b format

Flat space addressing: lun=0

4001000000000000 01b format

Flat space addressing: lun=1

Note in this case the lun (number) access method is 01b or the "flat space addressing" vs the 00b lun access method ("Peripheral device addressing") as seen in the previous example. Linux currently only supports the 00b lun access method. See the T10 "SCSI Architecture Model" specification for more information on lun number formats/access methods.

======================================================================================================================

Hot add, remove, rescan of SCSI devices on Linux

Finding informations about SCSI devices

The first problem when working with SCSI devices might be to map informations you got from /proc/scsi/scsi to the device names the kernel uses like /dev/sda and so on.

Or you can use lsscsi utlity. Here's an example:

# lsscsi

[0:0:0:0] disk ATA SEAGATE ST31000N SU0E /dev/sda

[0:0:1:0] disk ATA SEAGATE ST31000N SU0E /dev/sdb

...

[1:0:1:0] disk ATA SEAGATE ST31000N SU0E /dev/sdj

[1:0:2:0] disk ATA SEAGATE ST31000N SU0E /dev/sdk

[1:0:3:0] disk ATA HITACHI H7210CA3 A3CB /dev/sdl

[1:0:4:0] disk ATA HITACHI HUA7210S AC5A /dev/sdm

...

[5:0:7:0] disk ATA SEAGATE ST31000N SU0D /dev/sdav

[6:0:0:0] mediumx IBM TS3500 0104 /dev/sch0

[6:0:1:0] tape IBM ULT3580-TD5 0104 /dev/st0

[6:0:2:0] tape IBM ULT3580-TD5 0104 /dev/st1

You can find same info by checking /proc/scsi/scsi

# cat /proc/scsi/scsi

Attached devices:

Host: scsi0 Channel: 00 Id: 00 Lun: 00

Vendor: ATA Model: SEAGATE ST31000N Rev: SU0E

Type: Direct-Access ANSI SCSI revision: 05

Host: scsi0 Channel: 00 Id: 01 Lun: 00

Vendor: ATA Model: SEAGATE ST31000N Rev: SU0E

Type: Direct-Access ANSI SCSI revision: 05

Host: scsi0 Channel: 00 Id: 02 Lun: 00

Vendor: ATA Model: SEAGATE ST31000N Rev: SU0E

Type: Direct-Access ANSI SCSI revision: 05

...

In the example above

Host: scsi0 Channel: 00 Id: 01 Lun: 00

is the same device of [0:0:0:0] in the output of lsscsi

Where the output

h == hostadapter id (first one being 0)

c == SCSI channel on hostadapter (first one being 0)

t == ID

l == LUN (first one being 0)

Rescan of a SCSI bus

Surely reboot a server will let the scsi device get rescanned. But this is not the prefered way. The easiest way is to rescan the whole SCSI bus which will enable the Linux kernel to detect new devices!

To issue a SCSI bus rescan you must know on which bus you've added the device. If you don't know which bus and if there are mutliple buses on the system you can rescan each bus which will be somehow annoying but will not interrupt the system.

To initiate a SCSI bus rescan type

echo "- - -" > /sys/class/scsi_host/host<H>/scan

where H stands for the SCSI bus you want to scan.

# lsscsi

...

[1:0:0:7] disk IBM 1814 FAStT 1060 /dev/sdb

[1:0:0:8] disk IBM 1814 FAStT 1060 /dev/sdc

[1:0:0:10] disk IBM 1814 FAStT 1060 /dev/sde

# echo "- - -" > /sys/class/scsi_host/host1/scan

# lsscsi

...

[1:0:0:7] disk IBM 1814 FAStT 1060 /dev/sdb

[1:0:0:8] disk IBM 1814 FAStT 1060 /dev/sdc

[1:0:0:9] disk IBM 1814 FAStT 1060 /dev/sdd

[1:0:0:10] disk IBM 1814 FAStT 1060 /dev/sde

Well done, we got /dev/sdd back.

By the way this does not only work for disks but also for SCSI CD/DVD devices.

You can still use the old way, to get sdd back

echo "scsi add-single-device 1 0 0 9" > /proc/scsi/scsi

For more devices, you have to repeat the same for all devices

Deletion of a SCSI Device

There might be the situation where you must remove a SCSI device from the system.

Easy by using

echo 1 > /sys/class/scsi_device/h:c:t:l/device/delete

or

echo 1 > /sys/block/<dev>/device/delete

where <dev> is like sda or sdb etc..

Here's an example:

# echo 1 > /sys/block/sdau/device/delete

# lsscsi | grep sdau

#

Now you see the device sdau is gone.

In old way, this is what you would like to do

echo "scsi remove-single-device h c t l" > /proc/scsi/scsi

Add a SCSI Device back

Surely this can be done just rescan the SCSI bus that the disk seats inuse wildcard like below

echo "- - -" > /sys/class/scsi_host/host<h>/scan

You can also do this if you know the device SCSI location h:c:t:l

echo "c t l" > /sys/class/scsi_host/host<h>/scan

Old way

echo "scsi add-single-device a b c d" > /proc/scsi/scsi

Rescan of a SCSI Device

The problem of a SCSI bus rescan is, that it will only detect new devices. Can a existing or missing device be rescanned?

Sure, here is it

echo 1 > /sys/block/sdau/device/rescan

or

echo 1 > /sys/class/scsi_device/h:c:t:l/device/rescan

================================================================

How to scan HBA’s to acquire newly added storage:

Loop Initialization Protocol (issue_lip): Loop Initialization Protocol is a method to perform bus reset which scan the scsi bus and updates the scsi layer to reflect all the scsi devices. This is a asynchronous operation and may take longer time to complete. Usually after executing the command to trigger issue_lip, you would get a command prompt immediately, but in actual it might take up longer time to complete the bus scan.

Loop Initialization Protocol method to scan the HBA’s can cause delay and I/O timeouts if the HBA/Device is in use and can also remove devices unexpectedly. Hence performing the scan using this method is not recommended on any production server where the SAN Devices are already configured in use. This type of scan is recommended on a newly built server to scan all the LUNS/Devices.

To perform the bus scan using this method, execute the following command as root.

echo “1″ > /sys/class/fc_host/host*/issue_lip // this command will scan all the HBA’s

If you want to scan any specific HBA, then execute the command as below

echo “1″ > /sys/class/fc_host/hostN/issue_lip //replace N with the number of HBA to scan

Scanning specific device/path :

If you have a running server where you need to scan specific device or a path to the device, it is not recommended to use issue_lip method and you should use the below method to scan for the updated device information.

To scan a specific device/lun, you need the following information at minimum.

1. WWNN Number of the HBA to scan

2. Lun number (which is assigned from SAN)

Once you have the above mentioned information, you can identify the HBA number, HBA Channel and SCSI Target ID information by executing the following command.

grep 1111111111111111 /sys/class/fc_transport/*/node_name //replace ’1111111111111111′ with actual WWNN Number.

This command would display output as below

/sys/class/fc_transport/target4:0:1/node_name:0×5006016090203181

In this case, the HBA number (Host) is 4, HBA Channel is 0 and SCSI Target id is 1. If the Lun Number is 11, execute the following command to scan this device.

echo “0 1 11″ >/sys/class/scsi_host/host4/scan //here 4 is the HBA Number.

If you do not have the HBA Number, HBA Channel, SCSI Target ID or Lun ID Information, and wanted to scan all devices on specific HBA, execute the following command.

echo “- – -” > /sys/class/scsi_host/hostN/scan //replace N with the HBA Number.

===============================

Storage management, the joys! Storage administration is one of those tasks that every admin does, and depending on your growth it may be something that consumes a lot of your time. If your servers are connected to a Fibre channel or iSCSI SAN, you probably need to periodically poke your systems to see new storage that is made available to them. There are several tools that can be used to do this. QLogic provides the scli utility and a rescan shell script, the sg3utils package comes with the rescan-scsi-bus.sh script and you can always roll your own script to do this.

I use a mix of all of the tools above. On older systems I use the QLogic scli utility, on servers that have Emulex adapters or recent Operating Systems I use rescan-scsi-bun.sh, and in other cases I use the following script to prod the sysfs scan and issue_lip entries directly:

#!/bin/bash

SLEEP_INTERVAL=300

echo "Scanning all fibre channel host adapters"

for i in `ls /sys/class/fc_host`; do

echo "Rescanning /sys/class/fc_host/${i}:"

echo " Issuing a loop initialization on ${i}:"

echo "1" > /sys/class/fc_host/${i}/issue_lip

echo " Scanning ${i} for new devices:"

echo "- - -" > "/sys/class/scsi_host/${i}/scan"

echo "Sleeping for ${SLEEP_INTERVAL} seconds"

sleep ${SLEEP_INTERVAL}

done

This is the nice thing about Linux. There is a tool to do just about everything, and if there isn’t one available you can usually cobble something together in short order to do what you need. Isn’t life as a Linux admin grand!

=========================

1. Scanning new FC LUNs in Centos or Redhat Linux or RHEL.

a. Get the visible disks on system.

[root@nglinux ~]# fdisk -l | grep ^Disk | egrep -v "dm-|identifier"

Disk /dev/sdb: 8589 MB, 8589934592 bytes

Disk /dev/sda: 34.4 GB, 34359738368 bytes

[root@nglinux ~]#

b. Check the available FC host channel.

[root@nglinux ~]# ls /sys/class/fc_host/

host0

[root@nglinux ~]#

Now we need to scan host0 HBA for the new LUNs.

c. Reset the bus by using LIP(Loop Initiation protocol) and scan for new devices by echoing “—“(just like controller, target and disk in Unix).

[root@nglinux ~]# echo "1" > /sys/class/fc_host/host0/issue_lip

[root@nglinux ~]# echo "- - -" > /sys/class/scsi_host/host0/scan

[root@nglinux ~]#

We can monitor the “issue_lip” messages in /var/log/messages file to determine if the scan is complete.

d. Scan the disks again if new disk is detected.

[root@nglinux ~]# fdisk -l | grep ^Disk | egrep -v "dm-|identifier"

Disk /dev/sdb: 8589 MB, 8589934592 bytes

Disk /dev/sda: 34.4 GB, 34359738368 bytes

[root@nglinux ~]#

2. Scanning new SCSI LUNs in Centos or Redhat Linux or RHEL.

a. Get the total number of disks available.

[root@nglinux ~]# fdisk -l | grep ^Disk | egrep -v "dm-|identifier"

Disk /dev/sdb: 8589 MB, 8589934592 bytes

Disk /dev/sda: 34.4 GB, 34359738368 bytes

[root@nglinux ~]#

b. Find out number of scsi hosts available.

[root@nglinux ~]# ls /sys/class/scsi_host/

host0 host1 host2 host3

[root@nglinux ~]#

c. Scan all four targets using below command.

[root@nglinux ~]# echo "- - -" > /sys/class/scsi_host/host0/scan

[root@nglinux ~]# echo "- - -" > /sys/class/scsi_host/host1/scan

[root@nglinux ~]# echo "- - -" > /sys/class/scsi_host/host2/scan

[root@nglinux ~]# echo "- - -" > /sys/class/scsi_host/host3/scan

d. Verify if new disks is available.

[root@nglinux ~]# fdisk -l | grep ^Disk | egrep -v "dm-|identifier"

Disk /dev/sdb: 8589 MB, 8589934592 bytes

Disk /dev/sda: 34.4 GB, 34359738368 bytes

=====================================

How to scan new FC LUNS and SCSI disks in Linux ?

How to scan new FC LUNS and SCSI disks in Redhat Linux without rebooting the server? Most of the Linux beginners have wondering how to do this and this article will be for them.It may look like very simple as we perform this in daily operation to scan luns but system has many work to do in background when you execute storage scanning commands. Redhat says this type of scan can be distributive,since it can cause delays while I/O operation timeout and remove devices unexpectedly from OS.So perform this scan when really you want to scan the disks and LUNS.

Scanning FC-LUN’s in Redhat Linux

1.First find out how many disks are visible in “fdisk -l” .

# fdisk -l 2>/dev/null | egrep '^Disk' | egrep -v 'dm-' | wc -l

2.Find out how many host bus adapter configured in the Linux box.you can use “systool -fc_host -v” to verify available FC in the system.

# ls /sys/class/fc_host

host0 host1

In this case,you need to scan host0 & host1 HBA.

3.If the system virtual memory is too low ,then do not proceed further.If you have enough free virtual memory,then you can proceed with below command to scan new LUNS.

# echo "1" > /sys/class/fc_host/host0/issue_lip

# echo "- - -" > /sys/class/scsi_host/host0/scan

# echo "1" > /sys/class/fc_host/host1/issue_lip

# echo "- - -" > /sys/class/scsi_host/host1/scan

Note: You need to monitor the “issue_lip” in /var/log/messages to determine when the scan will complete.This operation is an asynchronous operation.

You can also use rescan-scsi-bus.sh script to detect new LUNS.

# yum install sg3_utils

# ./rescan-scsi-bus.sh

4.Verify if the new LUN is visible or not by counting the available disks.

# fdisk -l 2>/dev/null | egrep '^Disk' | egrep -v 'dm-' | wc -l

If any new LUNS added ,then you can see more count is more then before scanning the LUNS.

Scanning SCSI DISKS in Redhat Linux

1.Finding the existing disk from fdisk.

[root@mylinz1 ~]# fdisk -l |egrep '^Disk' |egrep -v 'dm-'

Disk /dev/sda: 21.5 GB, 21474836480 bytes

2.Find out how many SCSI controller configured.

[root@mylinz1 ~]# ls /sys/class/scsi_host/host

host0 host1 host2

In this case,you need to scan host0,host1 & host2.

3.Scan the SCSI disks using below command.

[root@mylinz1 ~]# echo "- - -" > /sys/class/scsi_host/host0/scan

[root@mylinz1 ~]# echo "- - -" > /sys/class/scsi_host/host1/scan

[root@mylinz1 ~]# echo "- - -" > /sys/class/scsi_host/host2/scan

4.Verify if the new disks are visible or not.

[root@mylinz1 ~]# fdisk -l |egrep '^Disk' |egrep -v 'dm-'

Disk /dev/sda: 21.5 GB, 21474836480 bytes

Disk /dev/sdb: 1073 MB, 1073741824 bytes

Disk /dev/sdc: 1073 MB, 1073741824 bytes

From Redhat Linux 5.4 onwards, redhat introduced “/usr/bin/rescan-scsi-bus.sh” script to scan all the SCSI bus and update the SCSI layer to reflect new devices.

How to discover, login, and logout iSCSI targets

ORACLE SOLUTIONS

Applies to:

----------------------------------------------------------------------------------------------------------------------

Operating Systems - RHEL 5.x, RHEL 6.x, OL 5.x, OL 6.x, Oracle VM 2.x

Platform - Applies to all Dell PowerEdge Servers

Goal:

----------------------------------------------------------------------------------------------------------------------

To discover iSCSI targets, login targets, and logout targets from your EqualLogic iSCSI storage array.

Solution:---------------------------------------------------------------------------------------------------------------------

Once the process of installing the iSCSI initiator is completed as seen in the wiki article,http://en.community.dell.com/dell-groups/enterprise_solutions/w/oracle_solutions/3-2-1-1-1-how-do-i-install-and-start-iscsi-initiator-utils.aspx, the next step is to discover your iSCSI targets.

Discovering iSCSI Targets:

Once you have the iSCSI service running you will use the 'iscsiadm' userspace utility to discover, login and logout iSCSI targets.

To get a list of available targets the following command can be used:

#iscsiadm -m discovery -t st -p <Group IP address>:3260

NOTE The group IP address is the IP address of the EqualLogic storage group.

Example:

# iscsiadm -m discovery -t st -p 172.23.10.240:3260

172.23.10.240:3260,1 iqn.2001-05.com.equallogic:0-8a0906-83bcb3401-16e0002fd0a46f3d-rhel5-test

The example shows that the 'rhel5-test' volume has been found.

Logging in iSCSI Targets:

Once you have discovered your iSCSI targets, you can log in the following target in one of two ways.

Issuing the following command will login all iSCSI targets found. The command to login all iSCSI targets at once is the following:

#iscsiadm -m node -l

If you prefer to login an individual iSCSI target the following command can be issued:

#iscsiadm -m node -T <Complete Target Name> -l -p <Group IP>:3260

Example:

#iscsiadm -m node -l -T iqn.2001-05.com.equallogic:83bcb3401-16e0002fd0a46f3d-rhel5-test -p 172.23.10.240:3260

Logging out iSCSI Targets:

Logging out iSCSI targets also can be accomplished in one of two ways. The process of logging out all iSCSI targets found, can be done with the following command:

#iscsiadm -m node -u

The process of logging out individual iSCSI target uses the following command:

#iscsiadm -m node -u -T <Complete Target Name>-p <Group IP address>:3260

Example:

#iscsiadm -m node -u -T iqn.2001-05.com.equallogic:83bcb3401-16e0002fd0a46f3d-rhel5-test -p 172.23.10.240:3260

==========================================

Scanning FC-LUN’s in Redhat Linux

1.First, find out how many disks are visible in “fdisk -l” .

# fdisk -l 2>/dev/null | egrep '^Disk' | egrep -v 'dm-' | wc -l

2.Find out how many host bus adapter configured in the Linux box.you can use “systool -fc_host -v” to verify available FC in the system.

# ls /sys/class/fc_host

host0 host1

In this case,you need to scan host0 & host1 HBA.

3.If the system virtual memory is too low ,then do not proceed further.If you have enough free virtual memory,then you can proceed with below command to scan new LUNS.

# echo "1" > /sys/class/fc_host/host0/issue_lip

# echo "- - -" > /sys/class/scsi_host/host0/scan

# echo "1" > /sys/class/fc_host/host1/issue_lip

# echo "- - -" > /sys/class/scsi_host/host1/scan

Note: You need to monitor the “issue_lip” in /var/log/messages to determine when the scan will complete.This operation is an asynchronous operation.

You can also use rescan-scsi-bus.sh script to detect new LUNS.

# yum install sg3_utils

# ./rescan-scsi-bus.sh

4. Verify if the new LUN is visible or not by counting the available disks.

# fdisk -l 2>/dev/null | egrep '^Disk' | egrep -v 'dm-' | wc -l

If any new LUNS added , then you can see more count is more then before scanning the LUNS.

Scanning SCSI DISKS in Redhat Linux

1. Finding the existing disk from fdisk.

[root@mylinz1 ~]# fdisk -l |egrep '^Disk' |egrep -v 'dm-'

Disk /dev/sda: 21.5 GB, 21474836480 bytes

2. Find out how many SCSI controller configured.

[root@mylinz1 ~]# ls /sys/class/scsi_host/host

host0 host1 host2

In this case, you need to scan host0,host1 & host2.

3. Scan the SCSI disks using below command.

[root@mylinz1 ~]# echo "- - -" > /sys/class/scsi_host/host0/scan

[root@mylinz1 ~]# echo "- - -" > /sys/class/scsi_host/host1/scan

[root@mylinz1 ~]# echo "- - -" > /sys/class/scsi_host/host2/scan

4. Verify if the new disks are visible or not.

[root@mylinz1 ~]# fdisk -l |egrep '^Disk' |egrep -v 'dm-'

Disk /dev/sda: 21.5 GB, 21474836480 bytes

Disk /dev/sdb: 1073 MB, 1073741824 bytes

Disk /dev/sdc: 1073 MB, 1073741824 bytes

From Redhat Linux 5.4 onwards, Red hat introduced “/usr/bin/rescan-scsi-bus.sh” script to scan all the SCSI bus and update the SCSI layer to reflect new devices.

===================================================

Scan new FC LUNS and SCSI disks in Linux

How to scan new FC LUNS and SCSI disks in Redhat Linux without rebooting the server? Most of the Linux beginners have wondering how to do this and this article will be for them.It may look like very simple as we perform this in daily operation to scan luns but system has many work to do in background when you execute storage scanning commands. Redhat says this type of scan can be distributive,since it can cause delays while I/O operation timeout and remove devices unexpectedly from OS.So perform this scan when really you want to scan the disks and LUNS.

1. Identify the existing disks

fdisk -l 2>/dev/null | egrep '^Disk' | egrep -v 'dm-|type|identifier'

2. Identify the number of HBA adapters

systool -c fc_host -v

or

ls /sys/class/fc_host

host0

Note the number of hosts available in the server.

3. Use the below command to scan the LUNs

echo "1" > /sys/class/fc_host/host0/issue_lip

echo "- - -" > /sys/class/scsi_host/host0/scan

If you have more number of hosts file under the directory /sys/class/fc_host, then use the command for each hosts file by replacing the “host0”.

From Redhat Linux 5.4 onwards, redhat introduced “/usr/bin/rescan-scsi-bus.sh” script to scan all the SCSI bus and update the SCSI layer to reflect new devices.

We can also use the “rescan-scsi-bus.sh” script to detect new LUNs without rebooting the server.

This script is available with sg3-utils package. So install the sg3-utils package.

yum install sg3_utils

rescan-scsi-bus.sh

4. Verify the Detected disks from fdisk again

fdisk -l 2>/dev/null | egrep '^Disk' | egrep -v 'dm-|type|identifier'

Follow the process to rescan the new added SCSI Disks:

1. Identify the existing disks

fdisk -l 2>/dev/null | egrep '^Disk' | egrep -v 'dm-|type|identifier'

2. Identify the number of SCSI Controllers detected

ls /sys/class/scsi_host/

host0

Note the number of hosts available in the server.

3. Scan the SCSI Disk for each hosts

echo "- - -" > /sys/class/scsi_host/host0/scan

If you have more number of hosts file under the directory /sys/class/scsi_host, then use the command for each hosts file by replacing the “host0”.

4. Verify the Detected disks from fdisk again

fdisk -l 2>/dev/null | egrep '^Disk' | egrep -v 'dm-|type|identifier'