iscsi-scst-howto.txt

This is a very quick HOWTO intended to provide a quick reference in how
to setup and configure iSCSI-SCST. For a deeper info please refer to
SCST and iSCSI-SCST README files.


Installing and using iSCSI-SCST with scstadmin
==============================================

1. Download, build and install iSCSI-SCST.

Start with downloading the iSCSI-SCST source code. You can either download the
released version from the following URL:

  http://scst.sourceforge.net/downloads.html

or you can download the latest development version by running the following
command in a shell:

  svn co https://scst.svn.sourceforge.net/svnroot/scst/trunk scst

While SCST and iSCSI-SCST work fine with an unpatched Linux kernel with
the same or better speed as other open source iSCSI targets, if you want
even better performance you need to patch and rebuild the kernel. Select
the patch according to your kernel version (2.6.38.x in the example
below):

  cd /usr/src/kernels/linux-2.6.38.8
  patch -p1 < $HOME/scst/iscsi-scst/kernel/patches/put_page_callback-2.6.38.patch
  make clean

If you can't find put_page_callback for your kernel, then the patching
is not needed for it, so you can skip this and next (kernel rebuild and
reboot) steps.

Next, build and install the kernel:

  make && make modules
  make modules_install && make install

For some Linux distributions (e.g. Ubuntu) the above steps do not rebuild the
initial RAM disk. Make sure you regenerate the RAM disk before rebooting.
Please look up the documentation of the update-initrd, update-initramfs or the
mkinitrd command supplied with your distribution.

Now reboot your system such that the newly built kernel is loaded.

Once the system is again up and running the next step is to build and install
SCST, iSCSI-SCST and scstadmin. With most distros (including RHEL 6) you can
do that by running the following command from the SCST directory:

  make 2release
  make scst scst_install iscsi iscsi_install scstadm scstadm_install

When using RHEL 5 or CentOS 5, use these commands instead:

  make 2release
  make enable_proc
  make scst scst_install iscsi iscsi_install scstadm scstadm_install

iSCSI-SCST includes the following components:
   * kernel modules: iscsi-scst.ko scst.ko scst_vdisk.ko
   * iscsi-scstd - daemon
   * man pages
   * Obsolete configuration files located under iscsi-scst/etc/obsolete, need to be copied to
     /etc if you want to use them. In iSCSI-SCST 2.0 it is recommended to use scstadmin
      and SCST sysfs interface for that:
      * initiators.allow - used for assigning specific initiators to targets
      * initiators.deny - used for assigning specific initiators to targets
      * iscsi-scstd.conf - list of targets and their properties

2. Set up /etc/scst.conf

The scst.conf configuration file is loaded at system boot time by the
shell script /etc/init.d/scst via scstadmin. The target, device and LUN
configuration is defined in this file.

Generally, SCST and its components, including iSCSI-SCST, are kernel
modules, hence they are using kernel interface for configuring, sysfs.
However, using sysfs interface directly is cumbersome. Scstadmin utility
allows to use regular plain text interface instead. You can load a
config file into SCST sysfs using "scstadmin -config" and save currently
running configuration into scst.conf using "scstadmin -write_config".
See below the second section, if you want to use SCST sysfs interface
directly.

Here is the sample config file. Copy it in /etc/scst.conf

HANDLER vdisk_fileio {
	DEVICE disk01 {
		filename /dev/ram0
		nv_cache 1
	}
	DEVICE disk02 {
		filename /dev/ram1
		nv_cache 1
	}
}

TARGET_DRIVER iscsi {
	enabled 1

	TARGET iqn.2006-10.net.vlnb:tgt {
		LUN 0 disk01
		LUN 1 disk02

		enabled 1
	}
}

Note that for each initiator LUN 0 must exist, hence you can't create
targets and initiator groups without LUN 0. This is required by the SCSI
protocol and is also documented in SCST's README file.

Then to start scst and iscsi-scst services run the following command:

  /etc/init.d/scst start

You should have the working iSCSI-SCST now. If at any later time you
need to change scst.conf and reload it, you can run scstadmin directly:

  scstadmin -config /etc/scst.conf

or restart SCST:

  /etc/init.d/scst restart

Do the following steps if you need to automatically configure SCST upon reboot.

3. Create soft links in /etc/init.d

This will make SCST to start automatically upon system boot. Use either
chkconfig or update-rc.d, depending on the Linux distribution you are using.
An example for Debian systems such as Ubuntu:

  update-rc.d scst defaults

On openSUSE you can let SCST start automatically through the following commands:

  insserv scst

4. Restart SCST and iSCSI-SCST

If you don't want to reboot now, you can make the above settings effective via
the following commands:

  /etc/init.d/scst restart


Installing and using iSCSI-SCST without using scstadmin via /sys interface
==========================================================================

This is advanced section you should not need for basic services.
Scstadmin should be fully sufficient for you. Use those steps only if
you need to do something too advanced for scstadmin. For instance, keep
your configuration is a database instead of plain file.

In an iSCSI-SCST system, three components interact to export your
devices and file systems to iSCSI initiators on remote hosts: Targets,
LUNS, and Devices. A Target represents the host and acts as a container
for LUNs. A LUN (Logical Unit Number) represents the address of a Device
and provides access control for initiators, as well as acting as a
container for Devices. A Device represents the actual block device or
file system you wish to make available to initiators. You have to create
targets first, then create LUNs, then assign LUNs to targets, then
define devices and assign them to a LUN. Finally, you have to enable
your Targets and the SCST system. Once the steps (below) have been
completed, you should be able to find and access your exported devices
from authorized remote initiators.

First repeat step 1 the first section but leave out "scstadm
scstadm_install" from the make command in step 1.

Next load the required kernel modules as follows:

modprobe scst_disk
modprobe scst_vdisk
modprobe iscsi-scst

Module scst_disk allows to export local SCSI (SAS, etc.) devices in
pass-through mode, scst_vdisk allows to create virtual LUNs based on
files and block devices, module iscsi-scst is the main iSCSI-SCST
module.

Then run iSCSI-SCST user space service:

iscsi-scstd


Creating targets
----------------

echo "add_target iqn.2007-05.com.example:storage.iscsi-scst-1" > /sys/kernel/scst_tgt/targets/iscsi/mgmt

This will add target iqn.2007-05.com.example:storage.iscsi-scst-1.


Changing targets' parameters
----------------------------

echo "8192" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example\:storage.iscsi-scst-1/MaxRecvDataSegmentLength

This will change MaxRecvDataSegmentLength parameter of target
iqn.2007-05.com.example:storage.iscsi-scst-1 to 8192.

You can read it then by:

# cat /sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example\:storage.iscsi-scst-1/MaxRecvDataSegmentLength
8192
[key]

The mark "[key]" shows high level management tools, like scstadmin, that
this attribute has not default value. You can ignore it.


Add vdisk devices
-----------------

dd if=/dev/zero of=/disk1 bs=1M count=1024

This will create a backend file /disk1 for our future virtual device
disk1.

echo "add_device disk1 filename=/disk1; nv_cache=1" >/sys/kernel/scst_tgt/handlers/vdisk_fileio/mgmt

This will add new FILEIO device disk1 with backend file /disk1 and
NV_CACHE option enabled.

For a block device usually it is more efficient to use BLOCKIO handler:

echo "add_device dev1 filename=/dev/dev1" >/sys/kernel/scst_tgt/handlers/vdisk_blockio/mgmt


Listing currently registered SCST devices
-----------------------------------------

You can see the list of available devices:

# ls -1 /sys/kernel/scst_tgt/devices
0:0:0:0
1:0:0:0
3:0:0:0
3:0:0:1
3:0:0:2
3:0:0:3
3:0:0:4
3:0:0:5
3:0:0:6
3:0:0:7
2:0:0:0
4:0:0:0
4:0:0:5
4:0:0:6
4:0:0:7
4:0:0:8
4:0:0:9
4:0:1:0
4:0:1:1
4:0:1:2
4:0:1:3
4:0:1:4
5:0:0:0
5:0:0:5
5:0:0:6
5:0:0:7
5:0:0:8
5:0:0:9
5:0:1:0
5:0:1:1
5:0:1:2
5:0:1:3
5:0:1:4
disk1


Defining LUN masking
--------------------

LUN masking defines which LUNs which initiator can see.

In order to associate specific LUNs with target
iqn.2007-05.com.example:storage.iscsi-scst-1, do the following:

echo "add 4:0:0:8 0" > /sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example\:storage.iscsi-scst-1/luns/mgmt

This will assign the LUN 0 for pass-through device 4:0:0:8 to the target
iqn.2007-05.com.example:storage.iscsi-scst-1.

Note, you must have LUN 0 for each group. That's a SCSI requirement
(documented in SCST's README).

echo "add disk1 1 read_only=1" > /sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example\:storage.iscsi-scst-1/luns/mgmt

This will assign the read only LUN 1 for virtual device disk1 to the
target iqn.2007-05.com.example:storage.iscsi-scst-1.


Deleting a LUN from a group
---------------------------

Run the following command:

echo "del X" > /sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example\:storage.iscsi-scst-1/luns/mgmt

This will remove the LUN X from the target iqn.2007-05.com.example:storage.iscsi-scst-1.

For more information about LUN masking, refer to SCST README, section
"Access and devices visibility management (LUN masking)".


Enabling targets
----------------

After you created and configured target, you should enable it:

echo 1 >/sys/kernel/scst_tgt/targets/iscsi/iqn.2007-05.com.example\:storage.iscsi-scst-1/enabled

Then enable the iSCSI-SCST driver:

echo 1 >/sys/kernel/scst_tgt/targets/iscsi/enabled


Assigning targets to specific initiators
----------------------------------------

In order to assign targets to specific initiators, you should use LUN
masking defined above. If an initiator assigned to an empty security
group of a target, the initiator will not see this target in the
SendTargets discovery and will not be able to connect to it.

Additionally, you can make the target be visible only via selected
portals using allowed_portal sysfs attribute. Read iSCSI-SCST's README
file for more details about this facility.


Installing and using iSCSI-SCST without using scstadmin via /proc interface
===========================================================================

First repeat steps 1 and 2 from the previous section but leave out
"scstadm scstadm_install" from the make command in step 1.

Next set up /etc/iscsi-scstd.conf

You can do this by e.g. running the following shell commands:

  echo "Target $(sed -n 's/InitiatorName=//p' /etc/iscsi/initiatorname.iscsi):storage" >/etc/iscsi-scstd.conf
  cat /etc/iscsi-scstd.conf

Next load the scst_disk kernel module as follows:

  modprobe scst_disk

The shell script /etc/init.d/iscsi-scst can now be used to start/stop/restart or
check the status of iSCSI-SCST.


Selecting devices to be used by SCST
------------------------------------

You can see the list of available devices:

# cat /proc/scsi_tgt/scsi_tgt
Device (host:ch:id:lun or name)                             Device handler
0:0:0:0                                                     dev_disk
1:0:0:0                                                     dev_disk
3:0:0:0                                                     dev_disk
3:0:0:1                                                     dev_disk
3:0:0:2                                                     dev_disk
3:0:0:3                                                     dev_disk
3:0:0:4                                                     dev_disk
3:0:0:5                                                     dev_disk
3:0:0:6                                                     dev_disk
3:0:0:7                                                     dev_disk
2:0:0:0                                                     none
4:0:0:0                                                     none
4:0:0:5                                                     dev_disk
4:0:0:6                                                     dev_disk
4:0:0:7                                                     dev_disk
4:0:0:8                                                     dev_disk
4:0:0:9                                                     dev_disk
4:0:1:0                                                     dev_disk
4:0:1:1                                                     dev_disk
4:0:1:2                                                     dev_disk
4:0:1:3                                                     dev_disk
4:0:1:4                                                     dev_disk
5:0:0:0                                                     none
5:0:0:5                                                     dev_disk
5:0:0:6                                                     dev_disk
5:0:0:7                                                     dev_disk
5:0:0:8                                                     dev_disk
5:0:0:9                                                     dev_disk
5:0:1:0                                                     dev_disk
5:0:1:1                                                     dev_disk
5:0:1:2                                                     dev_disk
5:0:1:3                                                     dev_disk
5:0:1:4                                                     dev_disk


LUN masking
-----------

SCST defines security groups. For each group, you can add LUNs. 

If you want to have all LUNs available for all targets, just add them to
the "Default" group:

# echo "add 5:0:0:8 0" >/proc/scsi_tgt/groups/Default/devices
# cat /proc/scsi_tgt/groups/Default/devices
Device (host:ch:id:lun or name)                             Virtual lun  Options
5:0:0:8                                                        0

Now, the LUN "5:0:0:8" was added to the "Default" group as LUN #0.

Note, that you must have LUN 0. That's a SCSI requirement (documented in SCST's README).


Defining LUN masking
--------------------

In order to associate specific LUNs with specific targets, do the following:

   * Create a group for the target:

echo "add_group Default_iqn.2007-05.com.example:storage.iscsi-scst-1" >/proc/scsi_tgt/scsi_tgt

   * Add LUNs to the group:

echo "add 4:0:0:8 0" > /proc/scsi_tgt/groups/Default_iqn.2007-05.com.example\:storage.iscsi-scst-1/devices

This will assign the LUN "4:0:0:8" to the target iqn.2007-05.com.example:storage.iscsi-scst-1.
Again, you must have LUN 0 for each group.


Deleting a LUN from a group
---------------------------

Run the following command:

echo "del 4:0:0:8 0" > /proc/scsi_tgt/groups/Default_iqn.2007-05.com.example\:storage.iscsi-scst-1/devices

This will remove the LUN "4:0:0:8" from the target iqn.2007-05.com.example:storage.iscsi-scst-1.


Deleting a group
----------------

Run the following command:

echo "del_group Default_iqn.2007-05.com.example:storage.iscsi-scst-1" >/proc/scsi_tgt/scsi_tgt

For more information about LUN masking, refer to SCST README, section
"Access and devices visibility management (LUN masking)".


Creating targets using iscsi-scstd.conf
---------------------------------------

The easiest way to create targets is to define them in
/etc/iscsi-scstd.conf. An example can be found in etc/iscsi-scstd.conf.
You need to have this file under /etc/ before starting iSCSI-SCST.


Assigning targets to specific initiators
----------------------------------------

In order to assign targets to specific initiators, you need to have
/etc/initiators.allow  and /etc/initiators.deny. You can find
example files in etc/initiators.allow and etc/initiators.deny.

Note that all targets are allowed to all initiators by default, so if
you want to use /etc/initiators.allow, you will need to have
/etc/initiators.deny that looks like this:

ALL ALL

This will deny all initiators expect for those defined in /etc/initiators.allow.

Useful examples of iscsi-scst-adm:

Show a specific target:

# iscsi-scst-adm --op show --tid=1
QueuedCommands=0

Show the configured parameters for a specific target:

# iscsi-scst-adm --op show --tid=1 --sid=0
InitialR2T=No
ImmediateData=Yes
MaxConnections=1
MaxRecvDataSegmentLength=2097152
MaxXmitDataSegmentLength=131072
MaxBurstLength=2097152
FirstBurstLength=262144
DefaultTime2Wait=2
DefaultTime2Retain=0
MaxOutstandingR2T=1
DataPDUInOrder=Yes
DataSequenceInOrder=Yes
ErrorRecoveryLevel=0
HeaderDigest=None
DataDigest=None
OFMarker=No
IFMarker=No
OFMarkInt=Reject
IFMarkInt=Reject


Bart Van Assche
Erez Zilber
Vladislav Bolkhovitin