This post demonstrates how to enable Bidirectional or Mutual CHAP on iSCSI luns between Oracle Enterprise Linux 7 and NetApp storage. The aggregate, lun and disk sizes are small in this HOWTO to keep it simple.
1) If not already installed, install the iSCSI initiator on your server.
> yum install iscsi-initiator*
2) Display your server’s new iSCSI initiator or iqn nodename.
> cat /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.1988-12.com.oracle:77ff4f784c55
3) On the NetApp filer, create the volume that will hold the iscsi luns. This command assumes you have aggregate aggr1 already created. If not, use an aggregate that has enough room for your volume.
netapp> vol create MCHAPVOL aggr1 10g
4) Create the lun in the volume.
netapp> lun create -s 5g -t linux /vol/MCHAPVOL/OEL7_iSCSI_MCHAP_01
5) Create an igroup and add the Linux iscsi nodename or iqn from step 2 above to the new igroup.
netapp> igroup create -i -t linux ISCSI_MCHAP_OEL7
netapp> igroup add ISCSI_MCHAP_OEL7 iqn.1988-12.com.oracle:77ff4f784c55
netapp> igroup set ISCSI_MCHAP_OEL7 report_scsi_name yes
netapp> igroup show ISCSI_MCHAP_OEL7
ISCSI_MCHAP_OEL7 (iSCSI) (ostype: linux):
iqn.1988-12.com.oracle:77ff4f784c55 (not logged in)
6) Map the lun to the igroup and give it lun ID 01.
netapp> lun map /vol/MCHAPVOL/OEL7_iSCSI_MCHAP_01 ISCSI_MCHAP_OEL7 01
7) Obtain the NetApp target nodename.
netapp> iscsi nodename
iSCSI target nodename: iqn.1992-08.com.netapp:sn.4055372815
8) Set the CHAP secret on the NetApp controller.
netapp> iscsi security add -i iqn.1988-12.com.oracle:77ff4f784c55 -s chap -p OEL7 -n iqn.1988-12.com.oracle:77ff4f784c55 -o NETAPPMCHAP -m iqn.1992-08.com.netapp:sn.4055372815
netapp> iscsi security show
Default sec is None
init: iqn.1986-03.com.sun:01:e00000000000.52bcad1c auth: CHAP Local Inbound password: **** Inbound username: iqn.1986-03.com.sun:01:e000000000bound password: **** Outbound username: iqn.1992-08.com.netapp:sn.4055372815
init: iqn.1988-12.com.oracle:77ff4f784c55 auth: CHAP Local Inbound password: **** Inbound username: iqn.1988-12.com.oracle:77ff4f784c55 Outbou** Outbound username: iqn.1992-08.com.netapp:sn.4055372815
9) On the server, edit your /etc/iscsi/iscsi.conf file and set the parameters below.
> vi /etc/iscsi/iscsid.conf
node.startup = automatic
node.session.auth.authmethod = CHAP
node.session.auth.username = iqn.1988-12.com.oracle:77ff4f784c55
node.session.auth.password = OEL7
node.session.auth.username_in = iqn.1992-08.com.netapp:sn.4055372815
node.session.auth.password_in = NETAPPMCHAP
discovery.sendtargets.auth.authmethod = CHAP
discovery.sendtargets.auth.username = iqn.1988-12.com.oracle:77ff4f784c55
discovery.sendtargets.auth.password = OEL7
discovery.sendtargets.auth.username_in = iqn.1992-08.com.netapp:sn.4055372815
discovery.sendtargets.auth.password_in = NETAPPMCHAP
> wq!
10) On the server, restart the service and discover your iSCSI target (your storage system).
> service iscsi restart
Redirecting to /bin/systemctl restart iscsi.service
a) Verify the target.
> iscsiadm -m discovery -t st -p 10.10.10.141
10.10.10.141:3260,1000 iqn.1992-08.com.netapp:sn.4055372815
> iscsiadm -m node (this should display the same as above)
10.10.10.141:3260,1000 iqn.1992-08.com.netapp:sn.4055372815
11) On the server, manually login to the iSCSI target (your storage array). Note there are two dashes “- -” in front of targetname and login.
> iscsiadm -m node –targetname “iqn.1992-08.com.netapp:sn.4055372815” –login
Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.4055372815, portal: 10.10.10.141,3260] (multiple)
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.4055372815, portal: 10.10.10.141,3260] successful.
a) On the NetApp storage console you should see the iSCSI session:
[netapp:iscsi.notice:notice]: ISCSI: New session from initiator iqn.1988-12.com.oracle:77ff4f784c55 at IP addr 10.10.10.201
b) Verify the iSCSI session on the filer:
netapp> iscsi session show
Session 4
Initiator Information
Initiator Name: iqn.1988-12.com.oracle:77ff4f784c55
ISID: 00:02:3d:06:00:00
Initiator Alias: localhost.localdomain
12) From the server , check your session.
> iscsiadm -m session -P 1
Target: iqn.1992-08.com.netapp:sn.4055372815 (non-flash)
Current Portal: 10.10.10.141:3260,1000
Persistent Portal: 10.10.10.141:3260,1000
**********
Interface:
**********
Iface Name: default
Iface Transport: tcp
Iface Initiatorname: iqn.1988-12.com.oracle:77ff4f784c55
Iface IPaddress: 10.10.10.201
Iface HWaddress: <empty>
Iface Netdev: <empty>
SID: 6
iSCSI Connection State: LOGGED IN
iSCSI Session State: LOGGED_IN
Internal iscsid Session State: NO CHANGE
13) From the server, check the NetApp iSCSI details. Note there are two dashes “- -” in front of mode, targetname and portal.
> iscsiadm –mode node –targetname “iqn.1992-08.com.netapp:sn.4055372815″ –portal 10.10.10.141:3260
# BEGIN RECORD 6.2.0.873-21
node.name = iqn.1992-08.com.netapp:sn.4055372815
node.tpgt = 1000
node.startup = automatic
node.leading_login = No
iface.hwaddress = <empty>
iface.ipaddress = <empty>
iface.iscsi_ifacename = default
iface.net_ifacename = <empty>
iface.transport_name = tcp
iface.initiatorname = <empty>
iface.state = <empty>
iface.vlan_id = 0
iface.vlan_priority = 0
iface.vlan_state = <empty>
iface.iface_num = 0
iface.mtu = 0
iface.port = 0
iface.bootproto = <empty>
iface.subnet_mask = <empty>
iface.gateway = <empty>
iface.dhcp_alt_client_id_state = <empty>
iface.dhcp_alt_client_id = <empty>
iface.dhcp_dns = <empty>
iface.dhcp_learn_iqn = <empty>
iface.dhcp_req_vendor_id_state = <empty>
iface.dhcp_vendor_id_state = <empty>
iface.dhcp_vendor_id = <empty>
iface.dhcp_slp_da = <empty>
iface.fragmentation = <empty>
iface.gratuitous_arp = <empty>
iface.incoming_forwarding = <empty>
iface.tos_state = <empty>
iface.tos = 0
iface.ttl = 0
iface.delayed_ack = <empty>
iface.tcp_nagle = <empty>
iface.tcp_wsf_state = <empty>
iface.tcp_wsf = 0
iface.tcp_timer_scale = 0
iface.tcp_timestamp = <empty>
iface.redirect = <empty>
iface.def_task_mgmt_timeout = 0
iface.header_digest = <empty>
iface.data_digest = <empty>
iface.immediate_data = <empty>
iface.initial_r2t = <empty>
iface.data_seq_inorder = <empty>
iface.data_pdu_inorder = <empty>
iface.erl = 0
iface.max_receive_data_len = 0
iface.first_burst_len = 0
iface.max_outstanding_r2t = 0
iface.max_burst_len = 0
iface.chap_auth = <empty>
iface.bidi_chap = <empty>
iface.strict_login_compliance = <empty>
iface.discovery_auth = <empty>
iface.discovery_logout = <empty>
node.discovery_address = 10.10.10.141
node.discovery_port = 3260
node.discovery_type = send_targets
node.session.initial_cmdsn = 0
node.session.initial_login_retry_max = 8
node.session.xmit_thread_priority = -20
node.session.cmds_max = 128
node.session.queue_depth = 32
node.session.nr_sessions = 1
node.session.auth.authmethod = CHAP
node.session.auth.username = iqn.1988-12.com.oracle:77ff4f784c55
node.session.auth.password = ********
node.session.auth.username_in = iqn.1992-08.com.netapp:sn.4055372815
node.session.auth.password_in = ********
node.session.timeo.replacement_timeout = 120
node.session.err_timeo.abort_timeout = 15
node.session.err_timeo.lu_reset_timeout = 30
node.session.err_timeo.tgt_reset_timeout = 30
node.session.err_timeo.host_reset_timeout = 60
node.session.iscsi.FastAbort = Yes
node.session.iscsi.InitialR2T = No
node.session.iscsi.ImmediateData = Yes
node.session.iscsi.FirstBurstLength = 262144
node.session.iscsi.MaxBurstLength = 16776192
node.session.iscsi.DefaultTime2Retain = 0
node.session.iscsi.DefaultTime2Wait = 2
node.session.iscsi.MaxConnections = 1
node.session.iscsi.MaxOutstandingR2T = 1
node.session.iscsi.ERL = 0
node.conn[0].address = 10.10.10.141
node.conn[0].port = 3260
node.conn[0].startup = manual
node.conn[0].tcp.window_size = 524288
node.conn[0].tcp.type_of_service = 0
node.conn[0].timeo.logout_timeout = 15
node.conn[0].timeo.login_timeout = 15
node.conn[0].timeo.auth_timeout = 45
node.conn[0].timeo.noop_out_interval = 5
node.conn[0].timeo.noop_out_timeout = 5
node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
node.conn[0].iscsi.HeaderDigest = None
node.conn[0].iscsi.IFMarker = No
node.conn[0].iscsi.OFMarker = No
# END RECORD
14) From the server, find and format the new lun (new disk). Your fdisk commands are in bold red below.
> cat /var/log/messages | grep “unknown partition table”
localhost kernel: sdb: unknown partition table
> fdisk /dev/sdb
Welcome to fdisk (util-linux 2.23.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x195fbc72.
The device presents a logical sector size that is smaller than
the physical sector size. Aligning to a physical sector (or optimal
I/O) size boundary is recommended, or performance may be impacted.
Command (m for help): w
> fdisk /dev/sdb
Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-10485759, default 2048): <press enter>
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-10485759, default 10485759): <press enter>
Using default value 10485759
Partition 1 of type Linux and of size 5 GiB is set
Command (m for help): p
Disk /dev/sdb: 5368 MB, 5368709120 bytes, 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 65536 bytes
Disk label type: dos
Disk identifier: 0xa1c2729d
Device Boot Start End Blocks Id System
/dev/sdb1 2048 10485759 5241856 83 Linux
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
15) On the server, create the Linux file system on the new partition.
> mkfs -t ext4 /dev/sdb1
mke2fs 1.42.9 (28-Dec-2013)
Discarding device blocks: done
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=16 blocks
327680 inodes, 1310464 blocks
65523 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1342177280
40 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
16) Verify the partition.
> blkid /dev/sdb1
/dev/sdb1: UUID=”eb7fa074-50d8-47d1-83aa-7b69568020e4″ TYPE=”ext4″
17) Create the mount point and manually mount the directory.
> mkdir /newiscsilun
> mount /dev/sdb1 /newiscsilun
> df -h | grep newiscsilun
/dev/sdb1 4.8G 20M 4.6G 1% /newiscsilun
18) Add the new mount point to /etc/fstab.
> vi /etc/fstab
/dev/sdb1 /newiscsilun ext4 _netdev 0 0
> wq!
Note: the _netdev option is important so that it doesn’t try mounting the target before the network is available.
19) Test that it survives a reboot by rebooting the server. With the _netdev set, iscsi starts and your CHAP logins should take place before it attempts to mount. After the reboot, login and verify its mounted.
> df -h | grep newiscsilun
/dev/sdb1 4.8G 20M 4.6G 1% /newiscsilun
20) On the server you can check session stats.
> iscsiadm -m session -s
Stats for session [sid: 6, target: iqn.1992-08.com.netapp:sn.4055372815, portal: 10.10.10.141,3260]
iSCSI SNMP:
txdata_octets: 137976652
rxdata_octets: 3841684
noptx_pdus: 0
scsicmd_pdus: 1127
tmfcmd_pdus: 0
login_pdus: 0
text_pdus: 0
dataout_pdus: 1827
logout_pdus: 0
snack_pdus: 0
noprx_pdus: 0
scsirsp_pdus: 1127
tmfrsp_pdus: 0
textrsp_pdus: 0
datain_pdus: 793
logoutrsp_pdus: 0
r2t_pdus: 1827
async_pdus: 0
rjt_pdus: 0
digest_err: 0
timeout_err: 0
iSCSI Extended:
tx_sendpage_failures: 0
rx_discontiguous_hdr: 0
eh_abort_cnt: 0
21) As root, change permissions on /etc/iscsi/iscsid.conf. I’m not sure why they haven’t fixed this clear text CHAP password in a file issue so just make sure only root can read/write the file.
> chmod 600 /etc/iscsi/iscsid.conf
22) On the NetApp storage you can verify the Lun and the server’s session.
netapp> lun show -v /vol/MCHAPVOL/OEL7_iSCSI_MCHAP_01
/vol/MCHAPVOL/OEL7_iSCSI_MCHAP_01 5g (5368709120) (r/w, online, mapped)
Serial#: BQVJ3]DxwBc-
Share: none
Space Reservation: enabled
Multiprotocol Type: linux
Maps: ISCSI_MCHAP_OEL7=1
Occupied Size: 134.0m (140546048)
Creation Time: Sat Aug 30 12:14:47 EST 2014
Cluster Shared Volume Information: 0x0
netapp> iscsi session show -v
Session 6
Initiator Information
Initiator Name: iqn.1988-12.com.oracle:77ff4f784c55
ISID: 00:02:3d:01:00:00
Initiator Alias: localhost.localdomain
Session Parameters
SessionType=Normal
TargetPortalGroupTag=1000
MaxConnections=1
ErrorRecoveryLevel=0
AuthMethod=CHAP
HeaderDigest=None
DataDigest=None
ImmediateData=Yes
InitialR2T=No
FirstBurstLength=65536
MaxBurstLength=65536
Initiator MaxRecvDataSegmentLength=65536
Target MaxRecvDataSegmentLength=65536
DefaultTime2Wait=2
DefaultTime2Retain=0
MaxOutstandingR2T=1
DataPDUInOrder=Yes
DataSequenceInOrder=Yes
Command Window Size: 64
Connection Information
Connection 0
Remote Endpoint: 10.10.10.201:41613
Local Endpoint: 10.10.10.141:3260
Local Interface: e0a
TCP recv window size: 131400
Command Information
No commands active
No commands active