This post demonstrates how to enable bidirectional or mutual CHAP on iSCSI luns between FreeBSD 10.1 and NetApp storage. The aggregate, lun and disk sizes are small in this HOWTO to keep it simple.  Special thanks to FreeBSD developer Edward Napierala for his help with testing -CURRENT updates and syntax issues.

1) On the NetApp filer, create the volume that will hold the iscsi luns. This command assumes you have aggregate aggr1 already created. If not, use an aggregate that has enough room for your volume.
netapp> vol create MCHAPVOL aggr1 10g

2) On the NetApp filer, create a lun in the volume. Type solaris is used to be compatible with UFS.
netapp> lun create -s 5g -t solaris /vol/MCHAPVOL/FBSD10_iSCSI_MCHAP_0

3) Obtain the NetApp target nodename.
netapp> iscsi nodename
iSCSI target nodename: iqn.1992-08.com.netapp:sn.4055372815

4) On the FreeBSD server, create an iSCSI session to the NetApp. Unfortunately, there is no way of obtaining the FreeBSD server iqn in advance. Hopefully this
function will be in the next release.
server> iscsictl -A -p 10.10.10.141 -t iqn.1992-08.com.netapp:sn.4055372815

5) Now that a basic session is established, obtain your server’s iqn.
server> iscsictl -Lv | grep “Initiator name”
Initiator name:   iqn.1994-09.org.freebsd:fbsd101

6) On the NetApp filer, create an iGroup and add the FreeBSD iscsi nodename or iqn from step 5 above. The iGroup type is Solaris because its the most similar to FreeBSD. BSD is not an iGroup option.
netapp> igroup create -i -t solaris ISCSI_MCHAP_FBSD10
netapp> igroup add ISCSI_MCHAP_FBSD10 iqn.1994-09.org.freebsd:fbsd101
netapp> igroup show

ISCSI_MCHAP_FBSD10 (iSCSI) (ostype: solaris):
iqn.1994-09.org.freebsd:fbsd101 (not logged in)

7) Map the lun to the iGroup and give it lun ID 0. Important note: as of FreeBSD 10.1, there is a lun enumeration limit that requires you to start your lun ID with 0. You cannot start with 1. This is fixed in the current baseline and will appear in 10.2.
netapp> lun map /vol/MCHAPVOL/FBSD10_iSCSI_MCHAP_0 ISCSI_MCHAP_FBSD10 0

8) Set the CHAP secret on the NetApp controller.
netapp> iscsi security add -i iqn.1994-09.org.freebsd:fbsd101 -s chap -p FREEBSD -n iqn.1994-09.org.freebsd:fbsd101 -o NETAPP -m iqn.1992-08.com.netapp:sn.4055372815

netapp> iscsi security show
init: iqn.1994-09.org.freebsd:fbsd101 auth: CHAP Local Inbound password: **** Inbound username: iqn.1994-09.org.freebsd:fbsd101 Outbound password: ****  Outbound username: iqn.1992-08.com.netapp:sn.4055372815

9) Configure iSCSI on the server.
a) Start iSCSI when booted:
server> echo iscsid_enable=”YES” >> /etc/rc.conf

b) This tells it to connect to all targets when booted.
server> echo iscsictl_enable=”YES” >> /etc/rc.conf

c) Create the iscsi.conf file. Explanation for the example below:

Example:
t0                       = Represents the target ID. Start with 0.
TargetAddress   = 10.10.10.141 (your NetApp controller)
TargetName      = iqn.1992-08.com.netapp:sn.4055372815 (your NetApp iqn)
AuthMethod      = CHAP (the authentication type)
chapIName       = iqn.1994-09.org.freebsd:fbsd101 (your FreeBSD server iqn)
chapSecret      = FREEBSD (your server’s chap secret)
tgtChapName    = iqn.1992-08.com.netapp:sn.4055372815 (your Netapp iqn)
tgtChapSecret    = NETAPP (your NetApp chap secret)

Actual:
server> vi /etc/iscsi.conf

t0 {
TargetAddress   = 10.10.10.141
TargetName      = iqn.1992-08.com.netapp:sn.4055372815
AuthMethod      = CHAP
chapIName       = iqn.1994-09.org.freebsd:fbsd101
chapSecret      = FREEBSD
tgtChapName    = iqn.1992-08.com.netapp:sn.4055372815
tgtChapSecret    = NETAPP
}

wq!

d) Change permissions on the file.
> chmod 500 /etc/iscsi.conf

e) Start the iSCSI server on the server.
server> service iscsid start
Starting iscsid.

f) Reboot to test boot and session parameters. The session should connect and your lun will be visible.
server> reboot

Note: if you can’t reboot, run the following:
server> iscsictl -An t0

10) Verify your new lun.
server>  cat /var/log/messages | grep da1
Feb  1 10:38:12 fbsd101 kernel: da1 at iscsi1 bus 0 scbus3 target 0 lun 0
Feb  1 10:38:12 fbsd101 kernel: da1: <NETAPP LUN 811a> Fixed Direct Access SCSI-4 device
Feb  1 10:38:12 fbsd101 kernel: da1: Serial Number BQVJ3]DxwBcF
Feb  1 10:38:12 fbsd101 kernel: da1: 150.000MB/s transfers
Feb  1 10:38:12 fbsd101 kernel: da1: Command Queueing enabled
Feb  1 10:38:12 fbsd101 kernel: da1: 5120MB (10485760 512 byte sectors: 255H 63S/T 652C)

server> camcontrol reportluns /dev/da1
1 LUN found
0

server> camcontrol devlist -v | grep NETAPP
<NETAPP LUN 811a> at scbus3 target 0 lun 0 (da1,pass2)

server> camcontrol inquiry /dev/da1
pass2: <NETAPP LUN 811a> Fixed Direct Access SCSI-4 device
pass2: Serial Number BQVJ3]DxwBcF
pass2: 150.000MB/s transfers, Command Queueing Enabled

11) Obtain session details on the server.
server> iscsictl -Lv
Session ID:       1
Initiator name:   iqn.1994-09.org.freebsd:fbsd101
Initiator portal:
Initiator alias:
Target name:      iqn.1992-08.com.netapp:sn.4055372815
Target portal:    10.10.10.141
Target alias:
User:             iqn.1994-09.org.freebsd:fbsd101
Secret:           FREEBSD
Mutual user:      iqn.1992-08.com.netapp:sn.4055372815
Mutual secret:    NETAPP
Session type:     Normal
Session state:    Connected
Failure reason:
Header digest:    None
Data digest:      None
DataSegmentLen:   65536
ImmediateData:    Yes
iSER (RDMA):      No
Device nodes:     da1

a) Verify the iSCSI session on the filer:
netapp> iscsi session show
Session 1
Initiator Information
Initiator Name: iqn.1994-09.org.freebsd:fbsd101
ISID: 80:33:9b:8b:a9:6d

12) From the server, format the new lun (new disk). Since you know the device ID from step 10 above (/dev/da1), perform the following steps.

a) Create the partition.
server> gpart create -s gpt da1
da1 created

b) Display the partition.
server> gpart show da1
=>      40  10485680  da1  GPT  (5.0G)
40  10485680  – free –  (5.0G)

c) Add a slice.
server> gpart add -t freebsd da1
da1s1 added

d) Format the slice.
server> newfs -L ntaplun -b 4096 /dev/da1s1
/dev/da1s1: 5120.0MB (10485680 sectors) block size 4096, fragment size 4096
using 107 cylinder groups of 48.12MB, 12320 blks, 6160 inodes.
super-block backups (for fsck_ffs -b #) at:
144, 98704, 197264, 295824, 394384, 492944, 591504, 690064, 788624, 887184, 985744, 1084304, 1182864,
1281424, 1379984, 1478544, 1577104, 1675664, 1774224, 1872784, 1971344, 2069904, 2168464, 2267024,
2365584, 2464144, 2562704, 2661264, 2759824, 2858384, 2956944, 3055504, 3154064, 3252624, 3351184,
3449744, 3548304, 3646864, 3745424, 3843984, 3942544, 4041104, 4139664, 4238224, 4336784, 4435344,
4533904, 4632464, 4731024, 4829584, 4928144, 5026704, 5125264, 5223824, 5322384, 5420944, 5519504,
5618064, 5716624, 5815184, 5913744, 6012304, 6110864, 6209424, 6307984, 6406544, 6505104, 6603664,
6702224, 6800784, 6899344, 6997904, 7096464, 7195024, 7293584, 7392144, 7490704, 7589264, 7687824,
7786384, 7884944, 7983504, 8082064, 8180624, 8279184, 8377744, 8476304, 8574864, 8673424, 8771984,
8870544, 8969104, 9067664, 9166224, 9264784, 9363344, 9461904, 9560464, 9659024, 9757584, 9856144,
9954704, 10053264, 10151824, 10250384, 10348944, 10447504

13)  Create the mount point and manually mount the directory.
server> mkdir /newiscsilun
server> mount /dev/da1s1 /newiscsilun
server> df -h | grep newiscsilun
/dev/da1s1    4.8G    8.0K    4.5G     0%    /newiscsilun

14) Add the new mount point to /etc/fstab.
server>  echo “/dev/da1s1 /newiscsilun ufs rw 1 1” >> /etc/fstab

15) Test that it survives a reboot by rebooting the server now.
server> reboot

a) After the reboot, login and check the lun mount point.
server> df -h | grep newiscsilun
/dev/da1s1    4.8G    8.0K    4.5G     0%    /newiscsilun

16) On the NetApp storage you can verify the lun and the server’s session.
netapp> lun show -v /vol/MCHAPVOL/FBSD10_iSCSI_MCHAP_0
lun show -v /vol/MCHAPVOL/FBSD10_iSCSI_MCHAP_0
/vol/MCHAPVOL/FBSD10_iSCSI_MCHAP_0      5g (5368709120)    (r/w, online, mapped)
Serial#: BQVJ3]DxwBcF
Share: none
Space Reservation: enabled
Multiprotocol Type: solaris
Maps: ISCSI_MCHAP_FBSD10=0
Occupied Size:    3.8m (4022272)
Creation Time: Sun Feb  1 10:10:17 EST 2015
Cluster Shared Volume Information: 0x0

a) On the NetApp controller, get stats with the command below.
netapp>  iscsi session show -v
Session 2
Initiator Information
Initiator Name: iqn.1994-09.org.freebsd:fbsd101
ISID: 80:0d:2c:82:e2:2a

Session Parameters
SessionType=Normal
TargetPortalGroupTag=1000
MaxConnections=1
ErrorRecoveryLevel=0
AuthMethod=CHAP
HeaderDigest=None
DataDigest=None
ImmediateData=Yes
InitialR2T=Yes
FirstBurstLength=65536
MaxBurstLength=65536
Initiator MaxRecvDataSegmentLength=65536
Target MaxRecvDataSegmentLength=65536
DefaultTime2Wait=0
DefaultTime2Retain=0
MaxOutstandingR2T=1
DataPDUInOrder=Yes
DataSequenceInOrder=Yes
Command Window Size: 64

Connection Information
Connection 0
Remote Endpoint: 10.10.10.61:46881
Local Endpoint: 10.10.10.141:3260
Local Interface: e0a
TCP recv window size: 131400

Command Information
No commands active