Intresting articles on queue depth

While doing some research on understanding Queue depth from an Oracle database running on AIX perspective, I came across these two articles which have done an excellent job explaining it.

http://dsstos.blogspot.com/2008/12/hostlun-queue-depths-command-taq.html

http://www-01.ibm.com/support/docview.wss?uid=tss1td105745

http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/TD105745

Mapping Storage Array(SAN) Volumes to AIX host

I have an older post where I talk about mapping SAN volumes to a Linux host, and on how to identify the volumes that were created on the array.

In this post I will show how to identify a volumes created on a Storwize V7000 Storage Array on the AIX host.

From the Storwize V7000 GUI we can see that I created 1 volume. Volume test_UID with UID 60050768027F0001F00000000000016C was created, the volume was then mapped to an AIX host.

On the AIX host run the lspv command this will list the current volumes available on the host. If you don't see the newly created volumes, run the cfgmgr command from the command line. Next run the lspv command, and you will see the two new volumes that were created and mapped onto the host.

isvp14_ora> lspv hdisk0          00f62a6b98742dad                    rootvg          active hdisk59         00f62a6bf28975ed                    swapa           active hdisk230        none                                None

Running the lsattr command on the AIX host as shown below will show lots of information about the LUN, including the unique_id       3321360050768027F0001F00000000000016C04214503IBMfcp

From highlighted part of the unique_id 3321360050768027F0001F00000000000016C04214503IBMfcp
field we can find that this LUN maps to the the newly created volume on the storage array.

isvp14_ora> lsattr -El hdisk229 PCM             PCM/friend/fcpother                                 Path Control Module              False algorithm       fail_over                                           Algorithm                        True clr_q           no                                                  Device CLEARS its Queue on error True dist_err_pcnt   0                                                   Distributed Error Percentage     True dist_tw_width   50                                                  Distributed Error Sample Time    True hcheck_cmd      test_unit_rdy                                       Health Check Command             True hcheck_interval 60                                                  Health Check Interval            True hcheck_mode     nonactive                                           Health Check Mode                True location                                                            Location Label                   True lun_id          0x14000000000000                                    Logical Unit Number ID           False lun_reset_spt   yes                                                 LUN Reset Supported              True max_retry_delay 60                                                  Maximum Quiesce Time             True max_transfer    0x40000                                             Maximum TRANSFER Size            True node_name       0x50050768020000d2                                  FC Node Name                     False pvid            none                                                Physical volume identifier       False q_err           yes                                                 Use QERR bit                     True q_type          simple                                              Queuing TYPE                     True queue_depth     8                                                   Queue DEPTH                      True reassign_to     120                                                 REASSIGN time out value          True reserve_policy  single_path                                         Reserve Policy                   True rw_timeout      30                                                  READ/WRITE time out value        True scsi_id         0x10900                                             SCSI ID                          False start_timeout   60                                                  START unit time out value        True unique_id       3321360050768027F0001F00000000000016C04214503IBMfcp Unique device identifier         False ww_name         0x50050768022000d2                                  FC World Wide Name               False

We can use the below script to show all the UID of the luns on the AIX host

#!/usr/ksh for disk in $(lsdev -Cc disk|awk '{print $1}') do echo $disk: lsattr -EHl $disk -a unique_id echo --------------------------------------- done

Everything you wanted to know about your HBA but were afraid to ask!!!

Well, as usual I was provided with a remote access to our lab machines at the beginning of the project, but needed to collect a whole lot of information about the server during the course of the project.

One such piece of information was about the HBA on the RedHat Linux box that I was working on.

You can find all the information you need about the HBA under /sys/class/fc_host

[root@isvx7 host5]# pwd
/sys/class/fc_host

[root@isvx7 fc_host]# ls
host5  host6

[root@isvx7 host5]# ls
device       port_id     speed              supported_speeds  uevent
fabric_name  port_name   statistics         symbolic_name
issue_lip    port_state  subsystem          system_hostname
node_name    port_type   supported_classes  tgtid_bind_type
[root@isvx7 host5]# cat speed
8 Gbit

[root@isvx7 host5]# cat supported_speeds
1 Gbit, 2 Gbit, 4 Gbit, 8 Gbit

[root@isvx7 host5]# lspci |grep -i -e fib -e hba
15:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
15:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
[root@isvx7 host5]#


[root@isvx7 proc]# systool -c fc_host -v
Class = "fc_host"

Class Device = "host5"
Class Device path = "/sys/class/fc_host/host5"
fabric_name         = "0x100000051ef510a4"
issue_lip           = 
node_name           = "0x20000024ff21d2ee"
port_id             = "0x020200"
port_name           = "0x21000024ff21d2ee"
port_state          = "Online"
port_type           = "NPort (fabric via point-to-point)"
speed               = "8 Gbit"
supported_classes   = "Class 3"
supported_speeds    = "1 Gbit, 2 Gbit, 4 Gbit, 8 Gbit"
symbolic_name       = "QLE2562 FW:v5.03.02 DVR:v8.03.01.04.05.05-k"
system_hostname     = ""
tgtid_bind_type     = "wwpn (World Wide Port Name)"
uevent              = 

Device = "host5"
Device path = "/sys/devices/pci0000:00/0000:00:03.0/0000:15:00.0/host5"
ct                  =
edc                 = 
els                 =
fw_dump             =
nvram               = "ISP "
optrom_ctl          = 
optrom              =
reset               = 
sfp                 = ""
uevent              = 
vpd                 = "("


Class Device = "host6"
Class Device path = "/sys/class/fc_host/host6"
fabric_name         = "0x10000005331842a7"
issue_lip           = 
node_name           = "0x20000024ff21d2ef"
port_id             = "0x020200"
port_name           = "0x21000024ff21d2ef"
port_state          = "Online"
port_type           = "NPort (fabric via point-to-point)"
speed               = "8 Gbit"
supported_classes   = "Class 3"
supported_speeds    = "1 Gbit, 2 Gbit, 4 Gbit, 8 Gbit"
symbolic_name       = "QLE2562 FW:v5.03.02 DVR:v8.03.01.04.05.05-k"
system_hostname     = ""
tgtid_bind_type     = "wwpn (World Wide Port Name)"
uevent              = 

Device = "host6"
Device path = "/sys/devices/pci0000:00/0000:00:03.0/0000:15:00.1/host6"
ct                  =
edc                 = 
els                 =
fw_dump             =
nvram               = "ISP "
optrom_ctl          = 
optrom              =
reset               = 
sfp                 = ""
uevent              = 
vpd                 = "("


[root@isvx7 proc]#

Mapping Storage Array(SAN) Volumes to RedHat 5.5 Linux host

I created 6 volumes on my Storwize V7000 Storage Array, and mapped them to my RedHat Linux 5.5 host. Each volume that I created has a unique UID assigned to it eg. vol01 has UID 60050768018103ABF000000000000001, while vol02 has UID 60050768018103ABF000000000000002 and so on.

Find the HBA host ids of the Linux system.

[root@isvx3 ~]# ls /sys/class/fc_host
host5  host6

Next scan the SCSI but to detect the new volumes.

[root@isvx3 ~]# echo "- - -" > /sys/class/scsi_host/host5/scan
[root@isvx3 ~]# echo "- - -" > /sys/class/scsi_host/host6/scan

On my linux host side I enabled the native multipathing(Device Mapper-Multipath). Now on doing "multipath -d -l" from the command line I see the following:

[root@isvx3 vdbench]# multipath -d -l
mpath1 (3600507680192828b8800000000000002) dm-3 IBM,2145
[size=250G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
 \_ 6:0:4:0 sdad 65:208 [active][undef]
 \_ 6:0:5:0 sdae 65:224 [active][undef]
 \_ 6:0:6:0 sdaf 65:240 [active][undef]
 \_ 6:0:7:0 sdag 66:0   [active][undef]
 \_ 5:0:4:0 sdn  8:208  [active][undef]
 \_ 5:0:5:0 sdo  8:224  [active][undef]
 \_ 5:0:6:0 sdp  8:240  [active][undef]
 \_ 5:0:7:0 sdq  65:0   [active][undef]
mpath9 (360050768018103abf000000000000006) dm-2 IBM,2145
[size=500G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
 \_ 6:0:1:5 sdac 65:192 [active][undef]
 \_ 5:0:0:5 sdg  8:96   [active][undef]
 \_ 5:0:1:5 sdm  8:192  [active][undef]
 \_ 6:0:0:5 sdw  65:96  [active][undef]
mpath8 (360050768018103abf000000000000001) dm-1 IBM,2145
[size=500G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
 \_ 6:0:1:4 sdab 65:176 [active][undef]
 \_ 5:0:0:4 sdf  8:80   [active][undef]
 \_ 5:0:1:4 sdl  8:176  [active][undef]
 \_ 6:0:0:4 sdv  65:80  [active][undef]
mpath7 (360050768018103abf000000000000005) dm-0 IBM,2145
[size=500G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
 \_ 6:0:1:3 sdaa 65:160 [active][undef]
 \_ 5:0:0:3 sde  8:64   [active][undef]
 \_ 5:0:1:3 sdk  8:160  [active][undef]
 \_ 6:0:0:3 sdu  65:64  [active][undef]
mpath6 (360050768018103abf000000000000004) dm-6 IBM,2145
[size=500G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
 \_ 5:0:0:2 sdd  8:48   [active][undef]
 \_ 5:0:1:2 sdj  8:144  [active][undef]
 \_ 6:0:0:2 sdt  65:48  [active][undef]
 \_ 6:0:1:2 sdz  65:144 [active][undef]
mpath5 (360050768018103abf000000000000003) dm-5 IBM,2145
[size=500G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
 \_ 5:0:0:1 sdc  8:32   [active][undef]
 \_ 5:0:1:1 sdi  8:128  [active][undef]
 \_ 6:0:0:1 sds  65:32  [active][undef]
 \_ 6:0:1:1 sdy  65:128 [active][undef]
mpath4 (360050768018103abf000000000000002) dm-4 IBM,2145
[size=500G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
 \_ 5:0:0:0 sdb  8:16   [active][undef]
 \_ 5:0:1:0 sdh  8:112  [active][undef]
 \_ 6:0:0:0 sdr  65:16  [active][undef]
 \_ 6:0:1:0 sdx  65:112 [active][undef]
[root@isvx3 vdbench]#

From this we can see that volume vol01 maps to dm-1, vol02 to dm-4, vol03 to dm-5, vol04 to dm-6, vol05 to dm-0, and vol06 to dm-2

After changing the owner and group of the volumes to oracle and dba:

[root@isvx3 ~]# chown -R oracle:dba /dev/mapper/mpath6
[root@isvx3 ~]# chown -R oracle:dba /dev/mapper/mpath7
[root@isvx3 ~]# chown -R oracle:dba /dev/mapper/mpath8
[root@isvx3 ~]# chown -R oracle:dba /dev/mapper/mpath9

I then used 4 of these volumes to create an Oracle ASM Disk Group during my 11g R2 Grid Infrastructure installation.