Next Previous Contents

8. SRP

SRP (SCSI Remote protocol or SCSI RDMA protocol) is a protocol that allows the use of SCSI devices across infiniband. If you have infiniband storage, use can use SRP to acess the devices.

8.1 Configuration

Ensure that your infiniband storage is presented to the host in question. Check your storage controller documentation. Ensure that the ib_srp kernel module is loaded and that the srptools package is installed.

modprobe ib_srp

8.2 SRP daemon configuration

srp_daemon is responsible for discovering and connecting to SRP targets. The default configuration shipped with srp_daemon is to ignore all presented devices; this is a failsafe to prevent devices from being mounted by accident on the wrong hosts.

The srp_daemon config file /etc/srp_daemon.conf has a simply syntax, and is described in the srp_daemon(1) manpage. Each line in this file is a rule which can be either to allow connection or to disallow connection according to the first character in the line (a or d accordingly) and ID of the storage device.

Determine the IDs of presented devices

You can determine the IDs of SRP devices presented to your hosts by running the ibsrpdm -c command.

# ibsrpdm -c
id_ext=50001ff10005052a,ioc_guid=50001ff10005052a,dgid=fe8000000000000050001ff10005052a,pkey=ffff,service_id=2a050500f11f0050

Configure srp_deamon to connect to the devices

Once we have the IDs of the devices, we can add them to /etc/srp_daemon.conf. You can also specify other srp related options for the target, such as max_cmd_per_lun and Max_sect. These are storage specific; check your vendor documentation for reccomended values.

# This rule allows connection to our target
a id_ext=50001ff10005052a,ioc_guid=50001ff10005052a,max_cmd_per_lun=32,max_sect=65535
# This rule disallows everything else
d
Restart the srp_daemon and the storage target should now become visible; check the kernel log to see if the disk has been detected.

/etc/init.d/srptools restart

In the example kernel log output the disk has been descovered as scsi device sdb.

scsi 3:0:0:1: Direct-Access     IBM      DCS9900          5.03 PQ: 0 ANSI: 5
sd 3:0:0:1: [sdb] 1953458176 4096-byte hardware sectors (8001365 MB)
sd 3:0:0:1: [sdb] Write Protect is off
sd 3:0:0:1: [sdb] Mode Sense: 97 00 10 08
sd 3:0:0:1: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
sd 3:0:0:1: [sdb] 1953458176 4096-byte hardware sectors (8001365 MB)
sd 3:0:0:1: [sdb] Write Protect is off
sd 3:0:0:1: [sdb] Mode Sense: 97 00 10 08
sd 3:0:0:1: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
 sdb:<6>scsi4 : SRP.T10:50001FF10005052A
 unknown partition table
sd 3:0:0:1: [sdb] Attached SCSI disk
sd 3:0:0:1: Attached scsi generic sg5 type 0

8.3 Multipathing, LVM and formatting

The newly detected SRP device can be treated as an other scsi device. If you have multiple infiniband adapters you can use multipath-tools on top of the SRP devices to protects against a network failure. If you are not using multipathed IO you can simply format the device as normal.


Next Previous Contents