HOWTO Secure iSCSI Luns Between FreeBSD 10.1 and NetApp Storage with Mutual CHAP

01 Sunday Feb 2015

Posted by Slice2 in FreeBSD, iSCSI, Security

Tags

This post demonstrates how to enable bidirectional or mutual CHAP on iSCSI luns between FreeBSD 10.1 and NetApp storage. The aggregate, lun and disk sizes are small in this HOWTO to keep it simple. Special thanks to FreeBSD developer Edward Napierala for his help with testing -CURRENT updates and syntax issues.

1) On the NetApp filer, create the volume that will hold the iscsi luns. This command assumes you have aggregate aggr1 already created. If not, use an aggregate that has enough room for your volume.
netapp> vol create MCHAPVOL aggr1 10g

2) On the NetApp filer, create a lun in the volume. Type solaris is used to be compatible with UFS.
netapp> lun create -s 5g -t solaris /vol/MCHAPVOL/FBSD10_iSCSI_MCHAP_0

3) Obtain the NetApp target nodename.
netapp> iscsi nodename
iSCSI target nodename: iqn.1992-08.com.netapp:sn.4055372815

4) On the FreeBSD server, create an iSCSI session to the NetApp. Unfortunately, there is no way of obtaining the FreeBSD server iqn in advance. Hopefully this
function will be in the next release.
server> iscsictl -A -p 10.10.10.141 -t iqn.1992-08.com.netapp:sn.4055372815

5) Now that a basic session is established, obtain your server’s iqn.
server> iscsictl -Lv | grep “Initiator name”
Initiator name: iqn.1994-09.org.freebsd:fbsd101

6) On the NetApp filer, create an iGroup and add the FreeBSD iscsi nodename or iqn from step 5 above. The iGroup type is Solaris because its the most similar to FreeBSD. BSD is not an iGroup option.
netapp> igroup create -i -t solaris ISCSI_MCHAP_FBSD10
netapp> igroup add ISCSI_MCHAP_FBSD10 iqn.1994-09.org.freebsd:fbsd101
netapp> igroup show

ISCSI_MCHAP_FBSD10 (iSCSI) (ostype: solaris):
iqn.1994-09.org.freebsd:fbsd101 (not logged in)

7) Map the lun to the iGroup and give it lun ID 0. Important note: as of FreeBSD 10.1, there is a lun enumeration limit that requires you to start your lun ID with 0. You cannot start with 1. This is fixed in the current baseline and will appear in 10.2.
netapp> lun map /vol/MCHAPVOL/FBSD10_iSCSI_MCHAP_0 ISCSI_MCHAP_FBSD10 0

8) Set the CHAP secret on the NetApp controller.
netapp> iscsi security add -i iqn.1994-09.org.freebsd:fbsd101 -s chap -p FREEBSD -n iqn.1994-09.org.freebsd:fbsd101 -o NETAPP -m iqn.1992-08.com.netapp:sn.4055372815

netapp> iscsi security show
init: iqn.1994-09.org.freebsd:fbsd101 auth: CHAP Local Inbound password: **** Inbound username: iqn.1994-09.org.freebsd:fbsd101 Outbound password: **** Outbound username: iqn.1992-08.com.netapp:sn.4055372815

9) Configure iSCSI on the server.
a) Start iSCSI when booted:
server> echo iscsid_enable=”YES” >> /etc/rc.conf

b) This tells it to connect to all targets when booted.
server> echo iscsictl_enable=”YES” >> /etc/rc.conf

c) Create the iscsi.conf file. Explanation for the example below:

Example:
t0                       = Represents the target ID. Start with 0.
TargetAddress   = 10.10.10.141 (your NetApp controller)
TargetName      = iqn.1992-08.com.netapp:sn.4055372815 (your NetApp iqn)
AuthMethod      = CHAP (the authentication type)
chapIName       = iqn.1994-09.org.freebsd:fbsd101 (your FreeBSD server iqn)
chapSecret      = FREEBSD (your server’s chap secret)
tgtChapName   = iqn.1992-08.com.netapp:sn.4055372815 (your Netapp iqn)
tgtChapSecret   = NETAPP (your NetApp chap secret)

Actual:
server> vi /etc/iscsi.conf

t0 {
TargetAddress   = 10.10.10.141
TargetName      = iqn.1992-08.com.netapp:sn.4055372815
AuthMethod      = CHAP
chapIName       = iqn.1994-09.org.freebsd:fbsd101
chapSecret      = FREEBSD
tgtChapName   = iqn.1992-08.com.netapp:sn.4055372815
tgtChapSecret   = NETAPP
}

wq!

d) Change permissions on the file.
> chmod 500 /etc/iscsi.conf

e) Start the iSCSI server on the server.
server> service iscsid start
Starting iscsid.

f) Reboot to test boot and session parameters. The session should connect and your lun will be visible.
server> reboot

Note: if you can’t reboot, run the following:
server> iscsictl -An t0

10) Verify your new lun.
server> cat /var/log/messages | grep da1
Feb 1 10:38:12 fbsd101 kernel: da1 at iscsi1 bus 0 scbus3 target 0 lun 0
Feb 1 10:38:12 fbsd101 kernel: da1: <NETAPP LUN 811a> Fixed Direct Access SCSI-4 device
Feb 1 10:38:12 fbsd101 kernel: da1: Serial Number BQVJ3]DxwBcF
Feb 1 10:38:12 fbsd101 kernel: da1: 150.000MB/s transfers
Feb 1 10:38:12 fbsd101 kernel: da1: Command Queueing enabled
Feb 1 10:38:12 fbsd101 kernel: da1: 5120MB (10485760 512 byte sectors: 255H 63S/T 652C)

server> camcontrol reportluns /dev/da1
1 LUN found
0

server> camcontrol devlist -v | grep NETAPP
<NETAPP LUN 811a> at scbus3 target 0 lun 0 (da1,pass2)

server> camcontrol inquiry /dev/da1
pass2: <NETAPP LUN 811a> Fixed Direct Access SCSI-4 device
pass2: Serial Number BQVJ3]DxwBcF
pass2: 150.000MB/s transfers, Command Queueing Enabled

11) Obtain session details on the server.
server> iscsictl -Lv
Session ID:       1
Initiator name:   iqn.1994-09.org.freebsd:fbsd101
Initiator portal:
Initiator alias:
Target name:      iqn.1992-08.com.netapp:sn.4055372815
Target portal:    10.10.10.141
Target alias:
User:             iqn.1994-09.org.freebsd:fbsd101
Secret:           FREEBSD
Mutual user:      iqn.1992-08.com.netapp:sn.4055372815
Mutual secret:    NETAPP
Session type:     Normal
Session state:    Connected
Failure reason:
Header digest:    None
Data digest:      None
DataSegmentLen:   65536
ImmediateData:    Yes
iSER (RDMA):      No
Device nodes:     da1

a) Verify the iSCSI session on the filer:
netapp> iscsi session show
Session 1
Initiator Information
Initiator Name: iqn.1994-09.org.freebsd:fbsd101
ISID: 80:33:9b:8b:a9:6d

12) From the server, format the new lun (new disk). Since you know the device ID from step 10 above (/dev/da1), perform the following steps.

a) Create the partition.
server> gpart create -s gpt da1
da1 created

b) Display the partition.
server> gpart show da1
=> 40 10485680 da1 GPT (5.0G)
40 10485680 – free – (5.0G)

c) Add a slice.
server> gpart add -t freebsd da1
da1s1 added

d) Format the slice.
server> newfs -L ntaplun -b 4096 /dev/da1s1
/dev/da1s1: 5120.0MB (10485680 sectors) block size 4096, fragment size 4096
using 107 cylinder groups of 48.12MB, 12320 blks, 6160 inodes.
super-block backups (for fsck_ffs -b #) at:
144, 98704, 197264, 295824, 394384, 492944, 591504, 690064, 788624, 887184, 985744, 1084304, 1182864,
1281424, 1379984, 1478544, 1577104, 1675664, 1774224, 1872784, 1971344, 2069904, 2168464, 2267024,
2365584, 2464144, 2562704, 2661264, 2759824, 2858384, 2956944, 3055504, 3154064, 3252624, 3351184,
3449744, 3548304, 3646864, 3745424, 3843984, 3942544, 4041104, 4139664, 4238224, 4336784, 4435344,
4533904, 4632464, 4731024, 4829584, 4928144, 5026704, 5125264, 5223824, 5322384, 5420944, 5519504,
5618064, 5716624, 5815184, 5913744, 6012304, 6110864, 6209424, 6307984, 6406544, 6505104, 6603664,
6702224, 6800784, 6899344, 6997904, 7096464, 7195024, 7293584, 7392144, 7490704, 7589264, 7687824,
7786384, 7884944, 7983504, 8082064, 8180624, 8279184, 8377744, 8476304, 8574864, 8673424, 8771984,
8870544, 8969104, 9067664, 9166224, 9264784, 9363344, 9461904, 9560464, 9659024, 9757584, 9856144,
9954704, 10053264, 10151824, 10250384, 10348944, 10447504

13) Create the mount point and manually mount the directory.
server> mkdir /newiscsilun
server> mount /dev/da1s1 /newiscsilun
server> df -h | grep newiscsilun
/dev/da1s1 4.8G 8.0K 4.5G 0% /newiscsilun

14) Add the new mount point to /etc/fstab.
server> echo “/dev/da1s1 /newiscsilun ufs rw 1 1” >> /etc/fstab

15) Test that it survives a reboot by rebooting the server now.
server> reboot

a) After the reboot, login and check the lun mount point.
server> df -h | grep newiscsilun
/dev/da1s1 4.8G 8.0K 4.5G 0% /newiscsilun

16) On the NetApp storage you can verify the lun and the server’s session.
netapp> lun show -v /vol/MCHAPVOL/FBSD10_iSCSI_MCHAP_0
lun show -v /vol/MCHAPVOL/FBSD10_iSCSI_MCHAP_0
/vol/MCHAPVOL/FBSD10_iSCSI_MCHAP_0 5g (5368709120) (r/w, online, mapped)
Serial#: BQVJ3]DxwBcF
Share: none
Space Reservation: enabled
Multiprotocol Type: solaris
Maps: ISCSI_MCHAP_FBSD10=0
Occupied Size: 3.8m (4022272)
Creation Time: Sun Feb 1 10:10:17 EST 2015
Cluster Shared Volume Information: 0x0

a) On the NetApp controller, get stats with the command below.
netapp> iscsi session show -v
Session 2
Initiator Information
Initiator Name: iqn.1994-09.org.freebsd:fbsd101
ISID: 80:0d:2c:82:e2:2a

Session Parameters
SessionType=Normal
TargetPortalGroupTag=1000
MaxConnections=1
ErrorRecoveryLevel=0
AuthMethod=CHAP
HeaderDigest=None
DataDigest=None
ImmediateData=Yes
InitialR2T=Yes
FirstBurstLength=65536
MaxBurstLength=65536
Initiator MaxRecvDataSegmentLength=65536
Target MaxRecvDataSegmentLength=65536
DefaultTime2Wait=0
DefaultTime2Retain=0
MaxOutstandingR2T=1
DataPDUInOrder=Yes
DataSequenceInOrder=Yes
Command Window Size: 64

Connection Information
Connection 0
Remote Endpoint: 10.10.10.61:46881
Local Endpoint: 10.10.10.141:3260
Local Interface: e0a
TCP recv window size: 131400

Command Information
No commands active

HOWTO Secure iSCSI Luns Between Ubuntu Server 14.10 and NetApp Storage with Mutual CHAP

29 Monday Dec 2014

Posted by Slice2 in iSCSI, Linux, Security

≈ Leave a comment

Tags

iSCSI, Linux, Security

This post demonstrates how to enable two-way or mutual CHAP on iSCSI luns between Ubuntu Server 14.10 and NetApp storage. The aggregate, lun and disk sizes are small in this HOWTO to keep it simple. Note that Ubuntu follows the no root model so you must use sudo on privileged command. I will not show every “enter password” prompt in the doc. When you see it, just enter your password.

1) Install open-iscsi on your server.
> sudo apt-get install open-iscsi
> sudo reboot (don’t argue with me, just do it!)

2) Display your server’s new iscsi initiator or iqn nodename.
> sudo cat /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.1993-08.org.debian:01:ce45f9ecc9b

3) On the NetApp filer, create the volume that will hold the iscsi luns. This command assumes you have aggregrate aggr1 already created. If not use an aggregate that has enough room for your volume.
netapp> vol create MCHAPVOL aggr1 10g

4) On the NetApp filer, create the lun in the volume.
netapp> lun create -s 5g -t linux /vol/MCHAPVOL/UB1410_iSCSI_MCHAP_01

5) On the NetApp filer, create an igroup and add the Linux iscsi nodename or iqn from step 2 above to it.
netapp> igroup create -i -t linux ISCSI_MCHAP_UB1410
netapp> igroup add ISCSI_MCHAP_UB1410 iqn.1993-08.org.debian:01:ce45f9ecc9b
netapp> igroup show

ISCSI_MCHAP_UB1410 (iSCSI) (ostype: linux):
iqn.1993-08.org.debian:01:ce45f9ecc9b (not logged in)

6) Map the lun to the iscsi-group and give it lun ID 01.
netapp> lun map /vol/MCHAPVOL/UB1410_iSCSI_MCHAP_01 ISCSI_MCHAP_UB1410 01

7) Obtain the NetApp target nodename.
netapp> iscsi nodename
iSCSI target nodename: iqn.1992-08.com.netapp:sn.4055372815

8) Set the CHAP secret on the NetApp controller.
netapp> iscsi security add -i iqn.1993-08.org.debian:01:ce45f9ecc9b -s chap -p MCHAPUB1410 -n iqn.1993-08.org.debian:01:ce45f9ecc9b -o NETAPPMCHAP -m iqn.1992-08.com.netapp:sn.4055372815

netapp> iscsi security show
init: iqn.1993-08.org.debian:01:ce45f9ecc9b auth: CHAP Local Inbound password: **** Inbound username: iqn.1993-08.org.debian:01:ce45f9ecc9b Outbound
password: **** Outbound username: iqn.1992-08.com.netapp:sn.4055372815

9) On the server, edit your /etc/iscsi/iscsi.conf file and set the parameters below. You can just copy this into the file under iscsid.startup = /usr/sbin/iscsid but make sure you comment out node.startup = manual just below it. Since you want it to start automatically comment out node.leading_login = No.

> sudo vi /etc/iscsi/iscsid.conf:
node.startup = automatic
node.session.auth.authmethod = CHAP
node.session.auth.username = iqn.1993-08.org.debian:01:ce45f9ecc9b
node.session.auth.password = MCHAPUB1410
node.session.auth.username_in = iqn.1992-08.com.netapp:sn.4055372815
node.session.auth.password_in = NETAPPMCHAP
discovery.sendtargets.auth.authmethod = CHAP
discovery.sendtargets.auth.username = iqn.1993-08.org.debian:01:ce45f9ecc9b
discovery.sendtargets.auth.password = MCHAPUB1410
discovery.sendtargets.auth.username_in = iqn.1992-08.com.netapp:sn.4055372815
discovery.sendtargets.auth.password_in = NETAPPMCHAP
> wq!

10) On the server, discover your iSCSI target (your NetApp storage system).
> sudo iscsiadm -m discovery -t st -p 10.10.10.141
10.10.10.141:3260,1000 iqn.1992-08.com.netapp:sn.4055372815

> sudo iscsiadm -m node (this should display the same as above)
10.10.10.141:3260,1000 iqn.1992-08.com.netapp:sn.4055372815

11) On the server, manually login to the iSCSI target (your storage array). Note there are two “- -” dashed in front of targetname and login.
> sudo iscsiadm -m node –-targetname “iqn.1992-08.com.netapp:sn.4055372815” -–login

Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.4055372815, portal: 10.10.10.141,3260] (multiple)
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.4055372815, portal: 10.10.10.141,3260] successful.

On the NetApp storage console you should see the iSCSI sessions:
[iscsi.notice:notice]: ISCSI:
New session from initiator iqn.1993-08.org.debian:01:ce45f9ecc9b at IP addr 10.10.10.128

Verify the iSCSI session on the filer:
netapp> iscsi session show
Session 1
Initiator Information
Initiator Name: iqn.1993-08.org.debian:01:ce45f9ecc9b
ISID: 00:02:3d:01:00:00
Initiator Alias: ub1410

12) Stop and start the iscsi service on the server.
> sudo service open-iscsi stop
Pause for 10 seconds and then run the next command.
> sudo service open-iscsi start

13) From the server, check your session.
> sudo iscsiadm -m session -P 1

14) From the server, check the NetApp iSCSI details. Note that mode, targetname and portal have two “- -” dashes in front of them.
> sudo iscsiadm –mode node –targetname “iqn.1992-08.com.netapp:sn.4055372815” –portal 10.10.10.141:3260

15) From the server, find and format the new lun (new disk). The command below will find the device. In this case its sdb.
> dmesg | grep “unknown partition table”
[ 1930.949065] sdb: unknown partition table
[ 2167.186068] sdb: unknown partition table

> sudo fdisk /dev/sdb (note: commands are in bold red below)

Welcome to fdisk (util-linux 2.25.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0xe4775fd7.

Command (m for help): w

The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.

> sudo fdisk /dev/sdb

Command (m for help): n
Partition type
p primary (0 primary, 0 extended, 4 free)
e extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-10485759, default 2048): press enter
Last sector, +sectors or +size{K,M,G,T,P} (2048-10485759, default 10485759): press enter

Created a new partition 1 of type ‘Linux’ and of size 5 GiB.

Command (m for help): p
Disk /dev/sdb: 5 GiB, 5368709120 bytes, 10485760 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 65536 bytes
Disklabel type: dos
Disk identifier: 0xc68508a4

Device Boot Start End Sectors Size Id Type
/dev/sdb1 2048 10485759 10483712 5G 83 Linux

Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.

16) On the server, create the Linux file system on the new partition.
> sudo mkfs -t ext4 /dev/sdb1
mke2fs 1.42.10 (18-May-2014)
Discarding device blocks: done
Creating filesystem with 1310464 4k blocks and 327680 inodes
Filesystem UUID: d125b8ff-a690-4cbb-925d-645764d41172
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

17) Verify the partition.
> sudo blkid /dev/sdb1
/dev/sdb1: UUID=”d125b8ff-a690-4cbb-925d-645764d41172″ TYPE=”ext4″ PARTUUID=”c68508a4-01″

18) Create the mount point and manually mount the directory.
> sudo mkdir /newiscsilun
> sudo mount /dev/sdb1 /newiscsilun
> df -h | grep newiscsilun
/dev/sdb1 4.8G 10M 4.6G 1% /newiscsilun

19) Add the new mount point to /etc/fstab.
> sudo vi /etc/fstab
/dev/sdb1 /newiscsilun ext4 _netdev 0 0
> wq!

Note: the _netdev option is important so that it doesn’t try mounting the target before the network is available.

20) Test that it survives a reboot by rebooting the server now. With the _netdev set, iscsi starts and your CHAP logins should take place before it attempts to mount. After the reboot, login and verify that it is mounted.

> df -h | grep newiscsilun
/dev/sdb1 4.8G 10M 4.6G 1% /newiscsilun

21) On the server you can check session stats.
> sudo iscsiadm -m session -s

22) As root, change permissions on /etc/iscsi/iscsid.conf. I’m not sure why they haven’t fixed this clear text CHAP password in a file issue so just make sure only root can read/write the file.
> sudo chmod 600 /etc/iscsi/iscsid.conf

23) On the NetApp storage you can verify the lun and the server’s session.
netapp> lun show -v /vol/MCHAPVOL/UB1410_iSCSI_MCHAP_01
/vol/MCHAPVOL/UB1410_iSCSI_MCHAP_01 5g (5368709120) (r/w, online, mapped)
Serial#: BQVJ3]DxwBcB
Share: none
Space Reservation: enabled
Multiprotocol Type: linux
Maps: ISCSI_MCHAP_UB1410=1
Occupied Size: 132.8m (139202560)
Creation Time: Mon Dec 29 13:33:18 EST 2014
Cluster Shared Volume Information: 0x0

You can also get stats with the command below.
netapp> iscsi session show -v

HOWTO Secure iSCSI Luns Between Oracle Enterprise Linux 7 and NetApp Storage with Mutual CHAP

01 Monday Sep 2014

Posted by Slice2 in iSCSI, Linux, Oracle, Security

≈ Leave a comment

Tags

iSCSI, Linux, Oracle, Security

This post demonstrates how to enable Bidirectional or Mutual CHAP on iSCSI luns between Oracle Enterprise Linux 7 and NetApp storage. The aggregate, lun and disk sizes are small in this HOWTO to keep it simple.

1) If not already installed, install the iSCSI initiator on your server.
> yum install iscsi-initiator*

2) Display your server’s new iSCSI initiator or iqn nodename.
> cat /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.1988-12.com.oracle:77ff4f784c55

3) On the NetApp filer, create the volume that will hold the iscsi luns. This command assumes you have aggregate aggr1 already created. If not, use an aggregate that has enough room for your volume.
netapp> vol create MCHAPVOL aggr1 10g

4) Create the lun in the volume.
netapp> lun create -s 5g -t linux /vol/MCHAPVOL/OEL7_iSCSI_MCHAP_01

5) Create an igroup and add the Linux iscsi nodename or iqn from step 2 above to the new igroup.
netapp> igroup create -i -t linux ISCSI_MCHAP_OEL7
netapp> igroup add ISCSI_MCHAP_OEL7 iqn.1988-12.com.oracle:77ff4f784c55
netapp> igroup set ISCSI_MCHAP_OEL7 report_scsi_name yes
netapp> igroup show ISCSI_MCHAP_OEL7

ISCSI_MCHAP_OEL7 (iSCSI) (ostype: linux):
iqn.1988-12.com.oracle:77ff4f784c55 (not logged in)

6) Map the lun to the igroup and give it lun ID 01.
netapp> lun map /vol/MCHAPVOL/OEL7_iSCSI_MCHAP_01 ISCSI_MCHAP_OEL7 01

7) Obtain the NetApp target nodename.
netapp> iscsi nodename
iSCSI target nodename: iqn.1992-08.com.netapp:sn.4055372815

8) Set the CHAP secret on the NetApp controller.
netapp> iscsi security add -i iqn.1988-12.com.oracle:77ff4f784c55 -s chap -p OEL7 -n iqn.1988-12.com.oracle:77ff4f784c55 -o NETAPPMCHAP -m iqn.1992-08.com.netapp:sn.4055372815

netapp> iscsi security show
Default sec is None
init: iqn.1986-03.com.sun:01:e00000000000.52bcad1c auth: CHAP Local Inbound password: **** Inbound username: iqn.1986-03.com.sun:01:e000000000bound password: **** Outbound username: iqn.1992-08.com.netapp:sn.4055372815
init: iqn.1988-12.com.oracle:77ff4f784c55 auth: CHAP Local Inbound password: **** Inbound username: iqn.1988-12.com.oracle:77ff4f784c55 Outbou** Outbound username: iqn.1992-08.com.netapp:sn.4055372815

9) On the server, edit your /etc/iscsi/iscsi.conf file and set the parameters below.
> vi /etc/iscsi/iscsid.conf
node.startup = automatic
node.session.auth.authmethod = CHAP
node.session.auth.username = iqn.1988-12.com.oracle:77ff4f784c55
node.session.auth.password = OEL7
node.session.auth.username_in = iqn.1992-08.com.netapp:sn.4055372815
node.session.auth.password_in = NETAPPMCHAP
discovery.sendtargets.auth.authmethod = CHAP
discovery.sendtargets.auth.username = iqn.1988-12.com.oracle:77ff4f784c55
discovery.sendtargets.auth.password = OEL7
discovery.sendtargets.auth.username_in = iqn.1992-08.com.netapp:sn.4055372815
discovery.sendtargets.auth.password_in = NETAPPMCHAP
> wq!

10) On the server, restart the service and discover your iSCSI target (your storage system).
> service iscsi restart
Redirecting to /bin/systemctl restart iscsi.service

a) Verify the target.
> iscsiadm -m discovery -t st -p 10.10.10.141
10.10.10.141:3260,1000 iqn.1992-08.com.netapp:sn.4055372815

> iscsiadm -m node (this should display the same as above)
10.10.10.141:3260,1000 iqn.1992-08.com.netapp:sn.4055372815

11) On the server, manually login to the iSCSI target (your storage array). Note there are two dashes “- -” in front of targetname and login.
> iscsiadm -m node –targetname “iqn.1992-08.com.netapp:sn.4055372815” –login
Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.4055372815, portal: 10.10.10.141,3260] (multiple)
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.4055372815, portal: 10.10.10.141,3260] successful.

a) On the NetApp storage console you should see the iSCSI session:
[netapp:iscsi.notice:notice]: ISCSI: New session from initiator iqn.1988-12.com.oracle:77ff4f784c55 at IP addr 10.10.10.201

b) Verify the iSCSI session on the filer:
netapp> iscsi session show
Session 4
Initiator Information
Initiator Name: iqn.1988-12.com.oracle:77ff4f784c55
ISID: 00:02:3d:06:00:00
Initiator Alias: localhost.localdomain

12) From the server , check your session.
> iscsiadm -m session -P 1
Target: iqn.1992-08.com.netapp:sn.4055372815 (non-flash)
Current Portal: 10.10.10.141:3260,1000
Persistent Portal: 10.10.10.141:3260,1000
**********
Interface:
**********
Iface Name: default
Iface Transport: tcp
Iface Initiatorname: iqn.1988-12.com.oracle:77ff4f784c55
Iface IPaddress: 10.10.10.201
Iface HWaddress: <empty>
Iface Netdev: <empty>
SID: 6
iSCSI Connection State: LOGGED IN
iSCSI Session State: LOGGED_IN
Internal iscsid Session State: NO CHANGE

13) From the server, check the NetApp iSCSI details. Note there are two dashes “- -” in front of mode, targetname and portal.
> iscsiadm –mode node –targetname “iqn.1992-08.com.netapp:sn.4055372815″ –portal 10.10.10.141:3260
# BEGIN RECORD 6.2.0.873-21
node.name = iqn.1992-08.com.netapp:sn.4055372815
node.tpgt = 1000
node.startup = automatic
node.leading_login = No
iface.hwaddress = <empty>
iface.ipaddress = <empty>
iface.iscsi_ifacename = default
iface.net_ifacename = <empty>
iface.transport_name = tcp
iface.initiatorname = <empty>
iface.state = <empty>
iface.vlan_id = 0
iface.vlan_priority = 0
iface.vlan_state = <empty>
iface.iface_num = 0
iface.mtu = 0
iface.port = 0
iface.bootproto = <empty>
iface.subnet_mask = <empty>
iface.gateway = <empty>
iface.dhcp_alt_client_id_state = <empty>
iface.dhcp_alt_client_id = <empty>
iface.dhcp_dns = <empty>
iface.dhcp_learn_iqn = <empty>
iface.dhcp_req_vendor_id_state = <empty>
iface.dhcp_vendor_id_state = <empty>
iface.dhcp_vendor_id = <empty>
iface.dhcp_slp_da = <empty>
iface.fragmentation = <empty>
iface.gratuitous_arp = <empty>
iface.incoming_forwarding = <empty>
iface.tos_state = <empty>
iface.tos = 0
iface.ttl = 0
iface.delayed_ack = <empty>
iface.tcp_nagle = <empty>
iface.tcp_wsf_state = <empty>
iface.tcp_wsf = 0
iface.tcp_timer_scale = 0
iface.tcp_timestamp = <empty>
iface.redirect = <empty>
iface.def_task_mgmt_timeout = 0
iface.header_digest = <empty>
iface.data_digest = <empty>
iface.immediate_data = <empty>
iface.initial_r2t = <empty>
iface.data_seq_inorder = <empty>
iface.data_pdu_inorder = <empty>
iface.erl = 0
iface.max_receive_data_len = 0
iface.first_burst_len = 0
iface.max_outstanding_r2t = 0
iface.max_burst_len = 0
iface.chap_auth = <empty>
iface.bidi_chap = <empty>
iface.strict_login_compliance = <empty>
iface.discovery_auth = <empty>
iface.discovery_logout = <empty>
node.discovery_address = 10.10.10.141
node.discovery_port = 3260
node.discovery_type = send_targets
node.session.initial_cmdsn = 0
node.session.initial_login_retry_max = 8
node.session.xmit_thread_priority = -20
node.session.cmds_max = 128
node.session.queue_depth = 32
node.session.nr_sessions = 1
node.session.auth.authmethod = CHAP
node.session.auth.username = iqn.1988-12.com.oracle:77ff4f784c55
node.session.auth.password = ********
node.session.auth.username_in = iqn.1992-08.com.netapp:sn.4055372815
node.session.auth.password_in = ********
node.session.timeo.replacement_timeout = 120
node.session.err_timeo.abort_timeout = 15
node.session.err_timeo.lu_reset_timeout = 30
node.session.err_timeo.tgt_reset_timeout = 30
node.session.err_timeo.host_reset_timeout = 60
node.session.iscsi.FastAbort = Yes
node.session.iscsi.InitialR2T = No
node.session.iscsi.ImmediateData = Yes
node.session.iscsi.FirstBurstLength = 262144
node.session.iscsi.MaxBurstLength = 16776192
node.session.iscsi.DefaultTime2Retain = 0
node.session.iscsi.DefaultTime2Wait = 2
node.session.iscsi.MaxConnections = 1
node.session.iscsi.MaxOutstandingR2T = 1
node.session.iscsi.ERL = 0
node.conn[0].address = 10.10.10.141
node.conn[0].port = 3260
node.conn[0].startup = manual
node.conn[0].tcp.window_size = 524288
node.conn[0].tcp.type_of_service = 0
node.conn[0].timeo.logout_timeout = 15
node.conn[0].timeo.login_timeout = 15
node.conn[0].timeo.auth_timeout = 45
node.conn[0].timeo.noop_out_interval = 5
node.conn[0].timeo.noop_out_timeout = 5
node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
node.conn[0].iscsi.HeaderDigest = None
node.conn[0].iscsi.IFMarker = No
node.conn[0].iscsi.OFMarker = No
# END RECORD

14) From the server, find and format the new lun (new disk). Your fdisk commands are in bold red below.
> cat /var/log/messages | grep “unknown partition table”
localhost kernel: sdb: unknown partition table

> fdisk /dev/sdb

Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x195fbc72.

The device presents a logical sector size that is smaller than
the physical sector size. Aligning to a physical sector (or optimal
I/O) size boundary is recommended, or performance may be impacted.

Command (m for help): w

> fdisk /dev/sdb
Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-10485759, default 2048): <press enter>
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-10485759, default 10485759): <press enter>
Using default value 10485759
Partition 1 of type Linux and of size 5 GiB is set

Command (m for help): p
Disk /dev/sdb: 5368 MB, 5368709120 bytes, 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 65536 bytes
Disk label type: dos
Disk identifier: 0xa1c2729d

Device Boot Start End Blocks Id System
/dev/sdb1 2048 10485759 5241856 83 Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

15) On the server, create the Linux file system on the new partition.
> mkfs -t ext4 /dev/sdb1
mke2fs 1.42.9 (28-Dec-2013)
Discarding device blocks: done
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=16 blocks
327680 inodes, 1310464 blocks
65523 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1342177280
40 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

16) Verify the partition.
> blkid /dev/sdb1
/dev/sdb1: UUID=”eb7fa074-50d8-47d1-83aa-7b69568020e4″ TYPE=”ext4″

17) Create the mount point and manually mount the directory.
> mkdir /newiscsilun
> mount /dev/sdb1 /newiscsilun
> df -h | grep newiscsilun
/dev/sdb1 4.8G 20M 4.6G 1% /newiscsilun

18) Add the new mount point to /etc/fstab.
> vi /etc/fstab
/dev/sdb1 /newiscsilun ext4 _netdev 0 0
> wq!

Note: the _netdev option is important so that it doesn’t try mounting the target before the network is available.

19) Test that it survives a reboot by rebooting the server. With the _netdev set, iscsi starts and your CHAP logins should take place before it attempts to mount. After the reboot, login and verify its mounted.

> df -h | grep newiscsilun
/dev/sdb1 4.8G 20M 4.6G 1% /newiscsilun

20) On the server you can check session stats.
> iscsiadm -m session -s
Stats for session [sid: 6, target: iqn.1992-08.com.netapp:sn.4055372815, portal: 10.10.10.141,3260]
iSCSI SNMP:
txdata_octets: 137976652
rxdata_octets: 3841684
noptx_pdus: 0
scsicmd_pdus: 1127
tmfcmd_pdus: 0
login_pdus: 0
text_pdus: 0
dataout_pdus: 1827
logout_pdus: 0
snack_pdus: 0
noprx_pdus: 0
scsirsp_pdus: 1127
tmfrsp_pdus: 0
textrsp_pdus: 0
datain_pdus: 793
logoutrsp_pdus: 0
r2t_pdus: 1827
async_pdus: 0
rjt_pdus: 0
digest_err: 0
timeout_err: 0
iSCSI Extended:
tx_sendpage_failures: 0
rx_discontiguous_hdr: 0
eh_abort_cnt: 0

21) As root, change permissions on /etc/iscsi/iscsid.conf. I’m not sure why they haven’t fixed this clear text CHAP password in a file issue so just make sure only root can read/write the file.
> chmod 600 /etc/iscsi/iscsid.conf

22) On the NetApp storage you can verify the Lun and the server’s session.
netapp> lun show -v /vol/MCHAPVOL/OEL7_iSCSI_MCHAP_01
/vol/MCHAPVOL/OEL7_iSCSI_MCHAP_01 5g (5368709120) (r/w, online, mapped)
Serial#: BQVJ3]DxwBc-
Share: none
Space Reservation: enabled
Multiprotocol Type: linux
Maps: ISCSI_MCHAP_OEL7=1
Occupied Size: 134.0m (140546048)
Creation Time: Sat Aug 30 12:14:47 EST 2014
Cluster Shared Volume Information: 0x0

netapp> iscsi session show -v
Session 6
Initiator Information
Initiator Name: iqn.1988-12.com.oracle:77ff4f784c55
ISID: 00:02:3d:01:00:00
Initiator Alias: localhost.localdomain

Session Parameters
SessionType=Normal
TargetPortalGroupTag=1000
MaxConnections=1
ErrorRecoveryLevel=0
AuthMethod=CHAP
HeaderDigest=None
DataDigest=None
ImmediateData=Yes
InitialR2T=No
FirstBurstLength=65536
MaxBurstLength=65536
Initiator MaxRecvDataSegmentLength=65536
Target MaxRecvDataSegmentLength=65536
DefaultTime2Wait=2
DefaultTime2Retain=0
MaxOutstandingR2T=1
DataPDUInOrder=Yes
DataSequenceInOrder=Yes
Command Window Size: 64

Connection Information
Connection 0
Remote Endpoint: 10.10.10.201:41613
Local Endpoint: 10.10.10.141:3260
Local Interface: e0a
TCP recv window size: 131400

Command Information
No commands active

No commands active

HOWTO Secure iSCSI Luns Between Red Hat Enterprise Linux 7 (Beta) and NetApp Storage with Mutual CHAP

01 Saturday Feb 2014

Posted by Slice2 in iSCSI, Linux, NetApp, Security

≈ Leave a comment

Tags

iSCSI, Linux, NetApp, Security

This post demonstrates how to enable Bidirectional or Mutual CHAP on iSCSI luns between Red Hat Enterprise Linux 7 (Beta) and NetApp storage. The aggregate, lun and disk sizes are small in this HOWTO to keep it simple.

1) If not already installed, install the iSCSI initiator on your system.
> yum install iscsi-initiator*

2) Display your server’s new iscsi initiator or iqn nodename.
> cat /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.1994-05.com.redhat:ece5618996a9

3) On the NetApp filer, create the volume that will hold the iscsi luns. This command assumes you have aggregate aggr1 already created. If not use an aggregate that has enough room for your volume.
netapp> vol create MCHAPVOL aggr1 10g

4) Create the lun in the volume.
netapp> lun create -s 5g -t linux /vol/MCHAPVOL/RHEL7_iSCSI_MCHAP_01

5) Create an igroup and add the Linux iscsi nodename or iqn from step 2 above to the new igroup.
netapp> igroup create -i -t linux ISCSI_MCHAP_RHEL7
netapp> igroup add ISCSI_MCHAP_RHEL7 iqn.1994-05.com.redhat:ece5618996a9
netapp> igroup show ISCSI_MCHAP_RHEL7

ISCSI_MCHAP_RHEL7 (iSCSI) (ostype: linux):
iqn.1994-05.com.redhat:ece5618996a9 (not logged in)

6) Map the lun to the igroup and give it lun ID 01.
netapp> lun map /vol/MCHAPVOL/RHEL7_iSCSI_MCHAP_01 ISCSI_MCHAP_RHEL7 01

7) Obtain the NetApp target nodename.
netapp> iscsi nodename
iqn.1992-08.com.netapp:sn.84167939

8) Set the CHAP secret on the NetApp controller.
netapp> iscsi security add -i iqn.1994-05.com.redhat:ece5618996a9 -s chap -p RHEL7 -n iqn.1994-05.com.redhat:ece5618996a9 -o NETAPPMCHAP -m iqn.1992-08.com.netapp:sn.84167939

netapp> iscsi security show
init: iqn.1994-05.com.redhat:ece5618996a9 auth: CHAP Inbound password: **** Inbound username: iqn.1994-05.com.redhat:ece5618996a9 Outbound password: **** Outbound username: iqn.1992-08.com.netapp:sn.84167939

9) On the server, edit your /etc/iscsi/iscsi.conf file and set the parameters below.
> vi /etc/iscsi/iscsid.conf
node.startup = automatic
node.session.auth.authmethod = CHAP
node.session.auth.username = iqn.1994-05.com.redhat:ece5618996a9
node.session.auth.password = RHEL7
node.session.auth.username_in = iqn.1992-08.com.netapp:sn.84167939
node.session.auth.password_in = NETAPPMCHAP
discovery.sendtargets.auth.authmethod = CHAP
discovery.sendtargets.auth.username = iqn.1994-05.com.redhat:ece5618996a9
discovery.sendtargets.auth.password = RHEL7
discovery.sendtargets.auth.username_in = iqn.1992-08.com.netapp:sn.84167939
discovery.sendtargets.auth.password_in = NETAPPMCHAP
> wq!

10) On the server, restart the service and discover your iSCSI target (your storage system).
> service iscsi restart
Redirecting to /bin/systemctl restart iscsi.service

a) Verify the target.
> iscsiadm -m discovery -t st -p 10.10.10.11
10.10.10.11:3260,1000 iqn.1992-08.com.netapp:sn.84167939

> iscsiadm -m node (this should display the same as above)
10.10.10.11:3260,1000 iqn.1992-08.com.netapp:sn.84167939

11) On the server, manually login to the iSCSI target (your storage array). Note there are two dashes “- -” in front of targetname and login.
> iscsiadm -m node –targetname “iqn.1992-08.com.netapp:sn.84167939” –login
Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.84167939, portal: 10.10.10.11,3260] (multiple)
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.84167939, portal: 10.10.10.11,3260] successful.

a) On the NetApp storage console you should see the iSCSI session:
[iscsi.notice:notice]: ISCSI: New session from initiator iqn.1994-05.com.redhat:ece5618996a9 at IP addr 10.10.10.186

b) Verify the iSCSI session on the filer:
netapp> iscsi session show
Session 88
Initiator Information
Initiator Name: iqn.1994-05.com.redhat:ece5618996a9
ISID: 00:02:3d:01:00:00
Initiator Alias: rhel7

12) From the server , check your session.
> iscsiadm -m session -P 1
Target: iqn.1992-08.com.netapp:sn.84167939
Current Portal: 10.10.10.11:3260,1000
Persistent Portal: 10.10.10.11:3260,1000
**********
Interface:
**********
Iface Name: default
Iface Transport: tcp
Iface Initiatorname: iqn.1994-05.com.redhat:ece5618996a9
Iface IPaddress: 10.10.10.186
Iface HWaddress: <empty>
Iface Netdev: <empty>
SID: 1
iSCSI Connection State: LOGGED IN
iSCSI Session State: LOGGED_IN
Internal iscsid Session State: NO CHANGE

14) From the server, find and format the new lun (new disk). Your fdisk commands are in bold red below.
> cat /var/log/messages | grep “unknown partition table”
rhel7 kernel: [ 24.102281] sdb: unknown partition table

> fdisk /dev/sdb

Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x2c025f67.

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

Command (m for help): p
Disk /dev/sdb: 5368 MB, 5368709120 bytes, 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xeb560917

Device Boot Start End Blocks Id System
/dev/sdb1 2048 10485759 5241856 83 Linux

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

15) On the server, create the Linux file system on the new partition.
> mkfs -t ext4 /dev/sdb1
mke2fs 1.42.8 (20-Jun-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
327680 inodes, 1310464 blocks
65523 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1342177280
40 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

16) Verify the partition.
> blkid /dev/sdb1
/dev/sdb1: UUID=”540997d7-ee07-42b3-a4af-612af6812d18″ TYPE=”ext4″

17) Create the mount point and manually mount the directory.
> mkdir /newiscsilun
> mount /dev/sdb1 /newiscsilun
> df -h | grep newiscsilun
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 4.8G 10M 4.6G 1% /newiscsilun

18) Add the new mount point to /etc/fstab.
> vi /etc/fstab
/dev/sdb1 /newiscsilun ext4 _netdev 0 0
> wq!

Note: the _netdev option is important so that it doesn’t try mounting the target before the network is available.

> df -h | grep newiscsilun
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 5.0G 139M 4.6G 3% /newiscsilun

20) On the server you can check session stats.
> iscsiadm -m session -s
Stats for session [sid: 1, target: iqn.1992-08.com.netapp:sn.84167939, portal: 10.10.10.11,3260]
iSCSI SNMP:
txdata_octets: 17096
rxdata_octets: 748232
noptx_pdus: 0
scsicmd_pdus: 213
tmfcmd_pdus: 0
login_pdus: 0
text_pdus: 0
dataout_pdus: 0
logout_pdus: 0
snack_pdus: 0
noprx_pdus: 0
scsirsp_pdus: 213
tmfrsp_pdus: 0
textrsp_pdus: 0
datain_pdus: 204
logoutrsp_pdus: 0
r2t_pdus: 0
async_pdus: 0
rjt_pdus: 0
digest_err: 0
timeout_err: 0
iSCSI Extended:
tx_sendpage_failures: 0
rx_discontiguous_hdr: 0
eh_abort_cnt: 0

22) On the NetApp storage you can verify the Lun and the server’s session.
> lun show -v /vol/MCHAPVOL/RHEL7_iSCSI_MCHAP_01
/vol/MCHAPVOL/RHEL7_iSCSI_MCHAP_01 5g (5368709120) (r/w, online, mapped)
Serial#: hoagPJvrDTup
Share: none
Space Reservation: enabled (not honored by containing Aggregate)
Multiprotocol Type: linux
Maps: ISCSI_MCHAP_RHEL7=1

> iscsi session show -v
Session 90
Initiator Information
Initiator Name: iqn.1994-05.com.redhat:ece5618996a9
ISID: 00:02:3d:01:00:00
Initiator Alias: rhel7

Session Parameters
SessionType=Normal
TargetPortalGroupTag=1000
MaxConnections=1
ErrorRecoveryLevel=0
AuthMethod=CHAP
HeaderDigest=None
DataDigest=None
ImmediateData=Yes
InitialR2T=No
FirstBurstLength=65536
MaxBurstLength=65536
Initiator MaxRecvDataSegmentLength=65536
Target MaxRecvDataSegmentLength=65536
DefaultTime2Wait=2
DefaultTime2Retain=0
MaxOutstandingR2T=1
DataPDUInOrder=Yes
DataSequenceInOrder=Yes
Command Window Size: 32

Connection Information
Connection 0
Remote Endpoint: 10.10.10.186:59575
Local Endpoint: 10.10.10.11:3260
Local Interface: e0a
TCP recv window size: 131400

Command Information
No commands active

HOWTO Secure iSCSI Luns Between Oracle Solaris 11 and NetApp Storage Using Bidirectional CHAP

09 Thursday Jan 2014

Posted by Slice2 in iSCSI, NetApp, Oracle, Security, Solaris

≈ Leave a comment

Tags

iSCSI, NetApp, Oracle, Security, Solaris

This post demonstrates how to secure iSCSI luns between Oracle Solaris 11 and NetApp storage. Solaris calls it Bidirectional CHAP rather than Mutual CHAP. The aggregate, lun and disk sizes are small in this HOWTO to keep it simple. Research the relationship between Solaris EFI, Solaris VTOC and lun size as well as UFS vs ZFS to make sure you choose the proper type for your environment. This was done with Solaris 11 (11/11) x86. All steps except the fdisk step near the end are the same for SPARC systems.

1) Check for the iSCSI packages. They should be installed by default.
> pkginfo | grep iSCSI
system SUNWiscsir Sun iSCSI Device Driver (root)
system SUNWiscsiu Sun iSCSI Management Utilities (usr)

2) Make sure the iSCSI service is running on your Solaris host.
> svcs | grep iscsi
online 6:41:58 svc:/network/iscsi/initiator:default

If not, start it.
> svcadm enable svc:/network/iscsi/initiator:default

3) Get your local iSCSI Initiator Node Name or iqn name on the Solaris host.
> iscsiadm list initiator-node | grep iqn
Initiator node name: iqn.1986-03.com.sun:01:e00000000000.52bcad1c

4) Make sure the iscsi service is running on the NetApp.
netapp> iscsi status

5) Create the volume that will hold the iscsi luns. This command assumes you have aggregate aggr1 already created. If not use an aggregate that has enough room for your volume.
netapp> vol create MCHAPVOL aggr1 10g

6) Create a lun on the volume.
netapp> lun create -s 5g -t solaris_efi /vol/MCHAPVOL/SOL11_iSCSI_MCHAP_01

7) Create an igroup and add the Solaris iscsi node name or iqn from step 3 above to it.
netapp> igroup create -i -t solaris ISCSI_MCHAP_SOL11
netapp> igroup add ISCSI_MCHAP_SOL11 iqn.1986-03.com.sun:01:e00000000000.52bcad1c
netapp> igroup show

ISCSI_MCHAP_SOL11 (iSCSI) (ostype: solaris):
iqn.1986-03.com.sun:01:e00000000000.52bcad1c (not logged in)

8) Map the lun to the igroup and give it lun ID 01.
netapp> lun map /vol/MCHAPVOL/SOL11_iSCSI_MCHAP_01 ISCSI_MCHAP_SOL11 01

Note: Solaris EFI is for larger than 2 TB luns and Solaris VTOC for smaller disks. This lun is small just to demonstrate the configuration.

9) Obtain the NetApp target nodename.
netapp> iscsi nodename
iqn.1992-08.com.netapp:sn.4055372815

10) On the Solaris host, configure the target (NetApp controller) to be statically discovered. Note that there are two dashes “- -” in front of –static and –sendtargets. For some reason it displays as one dash in some browsers.
> iscsiadm modify discovery –static enable
> iscsiadm modify discovery –sendtargets enable
> iscsiadm add discovery-address 10.10.10.141:3260
> iscsiadm add static-config iqn.1992-08.com.netapp:sn.4055372815,10.10.10.141:3260
> iscsiadm list static-config
Static Configuration Target: iqn.1992-08.com.netapp:sn.4055372815,10.10.10.141:3260

11) Check your discovery methods. Make sure Static and Send Targets are enabled.
> iscsiadm list discovery
Discovery:
Static: enabled
Send Targets: enabled
iSNS: disabled

12) Enable Bidirectional CHAP on the Solaris host for the target NetApp controller.
> iscsiadm modify target-param –authentication CHAP iqn.1992-08.com.netapp:sn.4055372815
> iscsiadm modify target-param -B enable iqn.1992-08.com.netapp:sn.4055372815

13) Set the target device secret key that identifies the target NetApp controller. Note Solaris supports a minimum of 12 and a maximum of 16 character CHAP secrets. Also, there are two dashes “- -” in front of –CHAP-secret. You can make up your own secrets.
> iscsiadm modify target-param –CHAP-secret iqn.1992-08.com.netapp:sn.4055372815
Enter secret: NETAPPBICHAP
Re-enter secret: NETAPPBICHAP

14) Set the Solaris host initiator name and CHAP secret. Remember, there are two dashes “- -” in front of –CHAP-secret. You can make up your own secrets.
> iscsiadm modify initiator-node –authentication CHAP
> iscsiadm modify initiator-node –CHAP-name iqn.1986-03.com.sun:01:e00000000000.52bcad1c
> iscsiadm modify initiator-node –CHAP-secret
Enter secret: BIDIRCHAPSOL11
Re-enter secret: BIDIRCHAPSOL11

15) Verify your target parameters. Make sure Bidirectional Authentication is enabled and Authentication type is CHAP.
> iscsiadm list target-param -v iqn.1992-08.com.netapp:sn.4055372815
Target: iqn.1992-08.com.netapp:sn.4055372815
Alias: –
Bi-directional Authentication: enabled
Authentication Type: CHAP
CHAP Name: iqn.1992-08.com.netapp:sn.4055372815
Login Parameters (Default/Configured):
Data Sequence In Order: yes/-
Data PDU In Order: yes/-
Default Time To Retain: 20/-
Default Time To Wait: 2/-
Error Recovery Level: 0/-
First Burst Length: 65536/-
Immediate Data: yes/-
Initial Ready To Transfer (R2T): yes/-
Max Burst Length: 262144/-
Max Outstanding R2T: 1/-
Max Receive Data Segment Length: 8192/-
Max Connections: 65535/-
Header Digest: NONE/-
Data Digest: NONE/-
Tunable Parameters (Default/Configured):
Session Login Response Time: 60/-
Maximum Connection Retry Time: 180/-
Login Retry Time Interval: 60/-
Configured Sessions: 1

16) Set the Bidirectional CHAP secrets on the NetApp controller.
netapp> iscsi security add -i iqn.1986-03.com.sun:01:e00000000000.52bcad1c -s chap -p BIDIRCHAPSOL11 -n iqn.1986-03.com.sun:01:e00000000000.52bcad1c -o NETAPPBICHAP -m iqn.1992-08.com.netapp:sn.4055372815

a) View the iSCSI security configuration.
netapp> iscsi security show
init: iqn.1986-03.com.sun:01:e00000000000.52bcad1c auth: CHAP Local Inbound password: **** Inbound username: iqn.1986-03.com.sun:01:e00000000000.52bcad1c Outbound password: **** Outbound username: iqn.1992-08.com.netapp:sn.4055372815

17) On the Solaris host, reconfigure the /dev namespace to recognize the iSCSI disk (lun) you just connected.
> devfsadm -i iscsi or devfsadm -Cv -i iscsi

18) Login to server and format the disk. Note – the fdisk command below can be skipped on SPARC systems. Your input is in bold red in the next sequence.
> format
Searching for disks…done

AVAILABLE DISK SELECTIONS:
0. c4t0d0 <VMware-Virtual disk-1.0 cyl 1824 alt 2 hd 255 sec 63>
/pci@0,0/pci15ad,1976@10/sd@0,0
1. c5t2d0 <NETAPP-LUN-7350 cyl 2558 alt 2 hd 128 sec 32>
/iscsi/disk@0000iqn.1992-08.com.netapp%3Asn.8416793903E8,1
Specify disk (enter its number): 1
selecting c5t2d0
[disk formatted]
No Solaris fdisk partition found.

FORMAT MENU:
disk       – select a disk
type       – select (define) a disk type
partition – select (define) a partition table
current    – describe the current disk
format     – format and analyze the disk
fdisk      – run the fdisk program
repair     – repair a defective sector
label      – write label to the disk
analyze    – surface analysis
defect     – defect list management
backup     – search for backup labels
verify     – read and display labels
save       – save new disk/partition definitions
inquiry    – show disk ID
volname    – set 8-character volume name
!<cmd>     – execute <cmd>, then return
quit
format> fdisk   (skip this command if you are on a SPARC system)
No fdisk table exists. The default partition for the disk is:

a 100% “SOLARIS System” partition

Type “y” to accept the default partition, otherwise type “n” to edit the
partition table.
y

format> p

PARTITION MENU:
0      – change `0′ partition
1      – change `1′ partition
2      – change `2′ partition
3      – change `3′ partition
4      – change `4′ partition
5      – change `5′ partition
6      – change `6′ partition
7      – change `7′ partition
select – select a predefined table
modify – modify a predefined partition table
name   – name the current table
print – display the current table
label – write partition map and label to the disk
!<cmd> – execute <cmd>, then return
quit
partition> p
Current partition table (default):
Total disk cylinders available: 2557 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders        Size            Blocks
0 unassigned    wm       0               0         (0/0/0)           0
1 unassigned    wm       0               0         (0/0/0)           0
2     backup    wu       0 – 2556        4.99GB    (2557/0/0) 10473472
3 unassigned    wm       0               0         (0/0/0)           0
4 unassigned    wm       0               0         (0/0/0)           0
5 unassigned    wm       0               0         (0/0/0)           0
6 unassigned    wm       0               0         (0/0/0)           0
7 unassigned    wm       0               0         (0/0/0)           0
8       boot    wu       0 –    0        2.00MB    (1/0/0)        4096
9 unassigned    wm       0               0         (0/0/0)           0

partition> 0
Part Tag Flag Cylinders Size Blocks
0 unassigned wm 0 0 (0/0/0) 0

Enter partition id tag[unassigned]: <press enter>
Enter partition permission flags[wm]: <press enter>
Enter new starting cyl[0]: <press enter>
Enter partition size[0b, 0c, 0e, 0.00mb, 0.00gb]: 4.99gb

partition> l (This is a lower case “L” not a numeral one or 1. This step labels the disk.)
Ready to label disk, continue? y

partition> q

format> q

19) Create the file system. You can choose either UFS or ZFS. Both options are shown below.

a) If you will use UFS:
> newfs -Tv /dev/rdsk/c5t2d0s0
newfs: construct a new file system /dev/rdsk/c5t2d0s0: (y/n)? y
mkfs -F ufs /dev/rdsk/c5t2d0s0 10465280 32 128 8192 8192 -1 1 250 1048576 t 0 -1 8 128 y
/dev/rdsk/c5t2d0s0: 10465280 sectors in 2555 cylinders of 128 tracks, 32 sectors
5110.0MB in 18 cyl groups (149 c/g, 298.00MB/g, 320 i/g)
super-block backups (for fsck -F ufs -o b=#) at: 32, 610368, 1220704, 1831040, 2441376,
3051712, 3662048, 4272384, 4882720, 5493056,
6103392, 6713728, 7324064, 7934400, 8544736, 9155072, 9765408, 10375744

> fsck /dev/rdsk/c5t2d0s0
> mkdir /old_ufs_filesystem
> mount /dev/dsk/c5t2d0s0 /old_ufs_filesystem
> vi /etc/vfstab and add the line below to the bottom of the file. This will mount it when the system boots.
/dev/dsk/c5t2d0s0 /dev/rdsk/c5t2d0s0 /old_ufs_filesystem ufs 2 yes –
> wq! (to exit the vi session)

b) Check the new mount.
> df -h | grep old_ufs_filesystem
/dev/dsk/c5t2d0s0 5.0G 5.0M 4.9G 1% /old_ufs_filesystem

20) If you will use ZFS:
a) Create a pool.
> zpool create -f netappluns c5t2d0s0

b) Create the filesystem.
> zfs create netappluns/fs

c) List the new filesystem.
> zfs list -r netappluns
NAME           USED AVAIL REFER MOUNTPOINT
netappluns     124K 4.89G    32K /netappluns
netappluns/fs   31K 4.89G    31K /netappluns/fs

d) Use the legacy display method.
> df -h | grep netappluns
netappluns 4.9G 32K 4.9G 1% /netappluns
netappluns/fs 4.9G 31K 4.9G 1% /netappluns/fs

21) You are done. Hope this helps.

HOWTO Secure iSCSI Luns Between Oracle Solaris 10 and NetApp Storage Using Bidirectional CHAP

27 Friday Dec 2013

Posted by Slice2 in iSCSI, NetApp, Oracle, Security, Solaris

≈ Leave a comment

Tags

iSCSI, NetApp, Oracle, Security, Solaris

This post demonstrates how to secure iSCSI luns between Oracle Solaris 10 and NetApp storage. Solaris calls it Bidirectional CHAP rather than Mutual CHAP. The aggregate, lun and disk sizes are small in this HOWTO to keep it simple. Research the relationship between Solaris EFI, Solaris VTOC and lun size as well as UFS vs ZFS to make sure you choose the proper type for your environment. This was done with Solaris 10 x86. All steps except the fdisk step near the end are the same for SPARC systems.

1) You need to be running at least the Solaris 10 1/06 release. To verify, check your release file.
> cat /etc/release
Oracle Solaris 10 8/11 s10x_u10wos_17b X86

2) Check for the iSCSI packages.
> pkginfo | grep iSCSI
system SUNWiscsir Sun iSCSI Device Driver (root)
system SUNWiscsiu Sun iSCSI Management Utilities (usr)

a) For reference the iSCSI target packages are listed below. You don’t need them for this HOWTO.
SUNWiscsitgtr Sun iSCSI Target (Root)
SUNWiscsitgtu Sun iSCSI Target (Usr)

3) If not installed, mount the Solaris 10 DVD and install the packages. Note the SPARC path will be different: sol_10_811_sparc
If the DVD doesn’t mount automatically:
> mount -F hsfs /dev/rdsk/c0t2d0s2 /mnt
> cd /mnt/sol_10_811_x86/Solaris_10/Product
If it does:
> cd /cdrom/sol_10_811_x86/Solaris_10/Product
>/usr/sbin/pkgadd -d SUNWiscsir
>/usr/sbin/pkgadd -d SUNWiscsiu

4) Make sure the iSCSI service is running on your Solaris host.
> svcs | grep iscsi
online 6:41:58 svc:/network/iscsi/initiator:default

If not, start it.
> svcadm enable svc:/network/iscsi/initiator:default

5) Get your local iSCSI Initiator Node Name or iqn name on the Solaris host.
> iscsiadm list initiator-node | grep iqn
Initiator node name: iqn.1986-03.com.sun:01:ea2fccf7ffff.52b894f9

6) Make sure the iscsi service is running on the NetApp.
netapp> iscsi status
If not, start it (You need a license for iscsi. Check with the license command.)
netapp> iscsi start

7) Create the volume that will hold the iscsi luns. This command assumes you have aggregate aggr1 already created. If not use an aggregate that has enough room for your volume.
netapp> vol create MCHAPVOL aggr1 10g

8) Create a lun on the volume.
netapp> lun create -s 5g -t solaris_efi /vol/MCHAPVOL/SOL10_iSCSI_MCHAP_01

9) Create an igroup and add the Solaris iscsi node name or iqn from step 5 above to it.
netapp> igroup create -i -t solaris ISCSI_MCHAP_SOL10
netapp> igroup add ISCSI_MCHAP_SOL10 iqn.1986-03.com.sun:01:ea2fccf7ffff.52b894f9
netapp> igroup show

ISCSI_MCHAP_SOL10 (iSCSI) (ostype: solaris):
iqn.1986-03.com.sun:01:ea2fccf7ffff.52b894f9 (not logged in)

10) Map the lun to the igroup and give it lun ID 01.
netapp> lun map /vol/MCHAPVOL/SOL10_iSCSI_MCHAP_01 ISCSI_MCHAP_SOL10 01

Note: Solaris EFI is for larger than 2 TB luns and Solaris VTOC for smaller disks. This lun is small just to demonstrate the configuration.

11) Obtain the NetApp target nodename.
netapp> iscsi nodename
iqn.1992-08.com.netapp:sn.84167939

12) On the Solaris host, configure the target (NetApp controller) to be statically discovered. Note that there are two dashes “- -” in front of –static and –sendtargets. For some reason it displays as one dash in some browsers.
> iscsiadm modify discovery –static enable
> iscsiadm modify discovery –sendtargets enable
> iscsiadm add discovery-address 10.10.10.11:3260
> iscsiadm add static-config iqn.1992-08.com.netapp:sn.84167939,10.10.10.11:3260
> iscsiadm list static-config
Static Configuration Target: iqn.1992-08.com.netapp:sn.84167939,10.10.10.11:3260

13) Check your discovery methods. Make sure Statis and Send Targets are enabled.
> iscsiadm list discovery
Discovery:
Static: enabled
Send Targets: enabled
iSNS: disabled

14) Enable Bidirectional CHAP on the Solaris host for the target NetApp controller. There are two dashes “- -” in front of –authentication.
> iscsiadm modify target-param –authentication CHAP iqn.1992-08.com.netapp:sn.84167939
> iscsiadm modify target-param -B enable iqn.1992-08.com.netapp:sn.84167939

15) Set the target device secret key that identifies the target NetApp controller. Note Solaris supports a minimum of 12 and a maximum of 16 character CHAP secrets. Also, there are two dashes “- -” in front of –CHAP-secret. You can make up your own secrets.
> iscsiadm modify target-param –CHAP-secret iqn.1992-08.com.netapp:sn.84167939
Enter secret: NETAPPBICHAP
Re-enter secret: NETAPPBICHAP

16) Set the Solaris host initiator name and CHAP secret. Remember, there are two dashes “- -” in front of –authentication, –CHAP-name and –CHAP-secret. You can make up your own secrets.
> iscsiadm modify initiator-node –authentication CHAP
> iscsiadm modify initiator-node –CHAP-name iqn.1986-03.com.sun:01:ea2fccf7ffff.52b894f9
> iscsiadm modify initiator-node –CHAP-secret
Enter secret: BIDIRCHAPSOL10
Re-enter secret: BIDIRCHAPSOL10

17) Verify your target parameters. Make sure Bidirectional Authentication is enabled and Authentication type is CHAP.
> iscsiadm list target-param -v iqn.1992-08.com.netapp:sn.84167939
Target: iqn.1992-08.com.netapp:sn.84167939
Alias: –
Bi-directional Authentication: enabled
Authentication Type: CHAP
CHAP Name: iqn.1992-08.com.netapp:sn.84167939
Login Parameters (Default/Configured):
Data Sequence In Order: yes/-
Data PDU In Order: yes/-
Default Time To Retain: 20/-
Default Time To Wait: 2/-
Error Recovery Level: 0/-
First Burst Length: 65536/-
Immediate Data: yes/-
Initial Ready To Transfer (R2T): yes/-
Max Burst Length: 262144/-
Max Outstanding R2T: 1/-
Max Receive Data Segment Length: 8192/-
Max Connections: 1/-
Header Digest: NONE/-
Data Digest: NONE/-
Tunable Parameters (Default/Configured):
Session Login Response Time: 60/-
Maximum Connection Retry Time: 180/-
Login Retry Time Interval: 60/-
Configured Sessions: 1

18) Set the Bidirectional CHAP secrets on the NetApp controller.
netapp> iscsi security add -i iqn.1986-03.com.sun:01:ea2fccf7ffff.52b894f9 -s chap -p BIDIRCHAPSOL10 -n iqn.1986-03.com.sun:01:ea2fccf7ffff.52b894f9 -o NETAPPBICHAP -m iqn.1992-08.com.netapp:sn.84167939

a) View the iSCSI security configuration.
netapp> iscsi security show
init: iqn.1986-03.com.sun:01:ea2fccf7ffff.52b894f9 auth: CHAP Inbound password: **** Inbound username: iqn.1986-03.com.sun:01:ea2fccf7ffff.52b894f9 Outbound password: **** Outbound username: iqn.1992-08.com.netapp:sn.84167939

19) On the Solaris host, reconfigure the /dev namespace to recognize the iSCSI disk (lun) you just connected.
> devfsadm -i iscsi or devfsadm -Cv -i iscsi

20) Verify CHAP configuration on the server. Restart the server and you should see the iSCSI session on the NetApp console.
> reboot

a) As the server boots, on the NetApp console you should see the following message:
[iscsi.notice:notice]: ISCSI: New session from initiator iqn.1986-03.com.sun:01:ea2fccf7ffff.52b894f9 at IP addr 10.10.10.188

21) Login to server and format the disk. Note – the fdisk command below can be skipped on SPARC systems. Your input is in bold red in the next sequence.
> format
AVAILABLE DISK SELECTIONS:
0. c1t0d0 <DEFAULT cyl 1563 alt 2 hd 255 sec 63>
/pci@0,0/pci15ad,1976@10/sd@0,0
1. c2t2d0 <DEFAULT cyl 2557 alt 2 hd 128 sec 32>
/iscsi/disk@0000iqn.1992-08.com.netapp%3Asn.8416793903E8,1Specify disk (enter its number): 1
selecting c2t2d0
[disk formatted]

FORMAT MENU:
disk       – select a disk
type       – select (define) a disk type
partition – select (define) a partition table
current    – describe the current disk
format     – format and analyze the disk
fdisk      – run the fdisk program
repair     – repair a defective sector
label      – write label to the disk
analyze    – surface analysis
defect     – defect list management
backup     – search for backup labels
verify     – read and display labels
save       – save new disk/partition definitions
inquiry    – show vendor, product and revision
volname    – set 8-character volume name
!<cmd>     – execute <cmd>, then return
quit

format> fdisk (Note: this command is only necessary on x86 systems. If you are on SPARC, skip to the next step.)
No fdisk table exists. The default partition for the disk is:

a 100% “SOLARIS System” partition

Type “y” to accept the default partition, otherwise type “n” to edit the
partition table.
y

22) Partition the disk:

format> p

Current partition table (original):
Total disk cylinders available: 2556 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders        Size            Blocks
0 unassigned    wm       0               0               (0/0/0)           0
1 unassigned    wm       0               0               (0/0/0)           0
2        backup    wu        0 – 2555    4.99GB     (2556/0/0) 10469376
3 unassigned    wm       0               0               (0/0/0)           0
4 unassigned    wm       0               0               (0/0/0)           0
5 unassigned    wm       0               0               (0/0/0)           0
6 unassigned    wm       0               0               (0/0/0)           0
7 unassigned    wm       0               0               (0/0/0)           0
8            boot    wu        0 –    0       2.00MB     (1/0/0)        4096
9 unassigned    wm       0               0               (0/0/0)           0

partition> 0
Part Tag Flag Cylinders Size Blocks
0 unassigned wm 0 0 (0/0/0) 0

Enter partition id tag[unassigned]: <press enter>
Enter partition permission flags[wm]: <press enter?
Enter new starting cyl[0]: <press enter>
Enter partition size[0b, 0c, 0e, 0.00mb, 0.00gb]: 4.99gb

partition> l (This is a lower case “L” not a numeral one or 1. This step labels the disk.)
Ready to label disk, continue? y

partition> q

format> q

23) Create the file system. You can choose either UFS or ZFS. Both options are shown below.

a) If you will use UFS:
> newfs -Tv /dev/rdsk/c2t2d0s0
newfs: construct a new file system /dev/rdsk/c2t2d0s0: (y/n)? y
pfexec mkfs -F ufs /dev/rdsk/c2t2d0s0 10465280 32 128 8192 8192 -1 1 250 1048576 t 0 -1 8 128 y
/dev/rdsk/c2t2d0s0: 10465280 sectors in 2555 cylinders of 128 tracks, 32 sectors
5110.0MB in 18 cyl groups (149 c/g, 298.00MB/g, 320 i/g)
super-block backups (for fsck -F ufs -o b=#) at:
32, 610368, 1220704, 1831040, 2441376, 3051712, 3662048, 4272384, 4882720,
5493056, 6103392, 6713728, 7324064, 7934400, 8544736, 9155072, 9765408, 10375744

> fsck /dev/rdsk/c2t2d0s0
> mkdir /old_ufs_filesystem
> mount /dev/dsk/c2t2d0s0 /old_ufs_filesystem
> vi /etc/vfstab and add the line below to the bottom of the file. This will mount it when the system boots.
/dev/dsk/c2t2d0s0 /dev/rdsk/c2t2d0s0 /old_ufs_filesystem ufs 2 yes –
> wq! (to exit the vi session)

b) Check the new mount.
> df -h | grep old_ufs_filesystem
/dev/dsk/c2t2d0s0 4.9G 5.0M 4.9G 1% /old_ufs_filesystem

24) If you will use ZFS:
a) Create a pool.
> zpool create -f netappluns c2t2d0

b) Create the filesystem.
> zfs create netappluns/fs

c) List the new filesystem.
> zfs list -r netappluns
NAME            USED AVAIL REFER MOUNTPOINT
netappluns      131K 4.89G    31K /netappluns
netappluns/fs    31K 4.89G    31K /netappluns/fs

Use the legacy display method.
> df -h | grep netappluns
netappluns 4.9G 32K 4.9G 1% /netappluns
netappluns/fs 4.9G 31K 4.9G 1% /netappluns/fs

25) You are done. Hope this helps.

HOWTO Secure iSCSI Luns Between Fedora 20 and NetApp Storage with Mutual CHAP

21 Saturday Dec 2013

Posted by Slice2 in iSCSI, Linux, Security

≈ Leave a comment

Tags

iSCSI, Linux, Security

This post demonstrates how to enable Bidirectional or mutual CHAP on iSCSI luns between Fedora 20 and NetApp storage. The aggregate, lun and disk sizes are small in this HOWTO to keep it simple.

1) If not already installed, install the iSCSI initiator on your system.
> yum install iscsi-initiator*
> reboot (don’t argue with me, just do it!)

2) Display your server’s new iscsi initiator or iqn nodename.
> cat /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.1994-05.com.redhat:4622a8d25677

4) Create the lun in the volume.
netapp> lun create -s 5g -t linux /vol/MCHAPVOL/FED20_iSCSI_MCHAP_01

5) Create an igroup and add the Linux iscsi nodename or iqn from step 2 above to it.
netapp> igroup create -i -t linux ISCSI_MCHAP_FED20
netapp> igroup add ISCSI_MCHAP_FED20 iqn.1994-05.com.redhat:4622a8d25677
netapp> igroup show ISCSI_MCHAP_FED20

ISCSI_MCHAP_FED20 (iSCSI) (ostype: linux):
InitiatorName=iqn.1994-05.com.redhat:4622a8d25677 (not logged in)

6) Map the lun to the igroup and give it lun ID 01.
netapp> lun map /vol/MCHAPVOL/FED20_iSCSI_MCHAP_01 ISCSI_MCHAP_FED20 01

7) Obtain the NetApp target nodename.
netapp> iscsi nodename
iqn.1992-08.com.netapp:sn.84167939

8) Set the CHAP secret on the NetApp controller.
netapp> iscsi security add -i iqn.1994-05.com.redhat:4622a8d25677 -s chap -p FED20 -n iqn.1994-05.com.redhat:4622a8d25677 -o NETAPPMCHAP -m iqn.1992-08.com.netapp:sn.84167939

netapp> iscsi security show
init: iqn.1994-05.com.redhat:4622a8d25677 auth: CHAP Inbound password: **** Inbound username: iqn.1994-05.com.redhat:4622a8d25677 Outbound password: ****Outbound username: iqn.1992-08.com.netapp:sn.84167939

9) On the server, edit your /etc/iscsi/iscsi.conf file and set the parameters below.
> vi /etc/iscsi/iscsid.conf
node.startup = automatic
node.session.auth.authmethod = CHAP
node.session.auth.username = iqn.1994-05.com.redhat:4622a8d25677
node.session.auth.password = FED20
node.session.auth.username_in = iqn.1992-08.com.netapp:sn.84167939
node.session.auth.password_in = NETAPPMCHAP
discovery.sendtargets.auth.authmethod = CHAP
discovery.sendtargets.auth.username = iqn.1994-05.com.redhat:4622a8d25677
discovery.sendtargets.auth.password = FED20
discovery.sendtargets.auth.username_in = iqn.1992-08.com.netapp:sn.84167939
discovery.sendtargets.auth.password_in = NETAPPMCHAP
> wq!

10) On the server, restart the service and discover your iSCSI target (your storage system).
> service iscsi restart
Redirecting to /bin/systemctl restart iscsi.service

a) You should see an entry on the NetApp console:
[iscsi.notice:notice]: ISCSI: New session from initiator iqn.1994-05.com.redhat:8ef4c68cfb5 at IP addr 10.10.10.195

b) Verify the target.
> iscsiadm -m discovery -t st -p 10.10.10.11
10.10.10.11:3260,1000 iqn.1992-08.com.netapp:sn.84167939

> iscsiadm -m node (this should display the same as above)
10.10.10.11:3260,1000 iqn.1992-08.com.netapp:sn.84167939

Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.84167939, portal: 10.10.10.11,3260] (multiple)
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.84167939, portal: 10.10.10.11,3260] successful.

On the NetApp storage console you should see the iSCSI sessions:
[iscsi.notice:notice]: ISCSI: New session from initiator iqn.1994-05.com.redhat:4622a8d25677 at IP addr 10.10.10.184

a) Verify the iSCSI session on the filer:
netapp> iscsi session show
Session 25
Initiator Information
Initiator Name: iqn.1994-05.com.redhat:4622a8d25677
ISID: 00:02:3d:01:00:00
Initiator Alias: fed20

12) Stop and start the iscsi service on the server.
> service iscsi stop
Pause for 10 seconds and then run the next command.
> service iscsi start

13) From the server , check your session.
> iscsiadm -m session -P 1
Current Portal: 10.10.10.11:3260,1000
Persistent Portal: 10.10.10.11:3260,1000
**********
Interface:
**********
Iface Name: default
Iface Transport: tcp
Iface Initiatorname: iqn.1994-05.com.redhat:4622a8d25677
Iface IPaddress: 10.10.10.184
Iface HWaddress: <empty>
Iface Netdev: <empty>
iSCSI Connection State: LOGGED IN
iSCSI Session State: LOGGED_IN
Internal iscsid Session State: NO CHANGE

14) From the server, check the NetApp iSCSI details. Note there are two dashes “- -” in front of mode, targetname and portal.
> iscsiadm –mode node –targetname “iqn.1992-08.com.netapp:sn.84167939” –portal 10.10.10.11:3260

15) From the server, find and format the new lun (new disk).
> cat /var/log/messages | grep “unknown partition table”
fed20 kernel: [ 2769.356768] sdb: unknown partition table

> fdisk /dev/sdb

Welcome to fdisk (util-linux 2.24).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.

Created a new DOS disklabel with disk identifier 0xc6cb1cf2.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

> fdisk /dev/sdb
Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-10485759, default 2048): <press enter>
Last sector, +sectors or +size{K,M,G,T,P} (2048-10485759, default 10485759): <press enter>

Created a new partition 1 of type ‘Linux’ and of size 5 GiB.

Command (m for help): p
Disk /dev/sdb: 5 GiB, 5368709120 bytes, 10485760 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x702e7603

Device Boot Start End Blocks Id System
/dev/sdb1 2048 10485759 5241856 83 Linux

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

16) On the server, create the Linux file system on the new partition.
> mkfs -t ext4 /dev/sdb1
mke2fs 1.42.8 (20-Jun-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
327680 inodes, 1310464 blocks
65523 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1342177280
40 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

17) Verify the partition.
> blkid /dev/sdb1
/dev/sdb1: UUID=”c1466d95-2551-4e0a-9dcb-fd430be03fe7″ TYPE=”ext4″ PARTUUID=”702e7603-01″

18) Create the mount point and manually mount the directory.
> mkdir /newiscsilun
> mount /dev/sdb1 /newiscsilun
> df -h | grep newiscsilun
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 4.8G 10M 4.6G 1% /newiscsilun

19) Add the new mount point to /etc/fstab.
> vi /etc/fstab
/dev/sdb1 /newiscsilun ext4 _netdev 0 0
> wq!

Note: the _netdev option is important so that it doesn’t try mounting the target before the network is available.

20) Test that it survives a reboot by rebooting the server. With the _netdev set, iscsi starts and your CHAP logins should take place before it attempts to mount. After the reboot, login and verify its mounted.

> df -h | grep newiscsilun
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 5.0G 139M 4.6G 3% /newiscsilun

21) On the server you can check session stats.
> iscsiadm -m session -s
Stats for session [sid: 1, target: iqn.1992-08.com.netapp:sn.84167939, portal: 10.10.10.11,3260]
iSCSI SNMP:
txdata_octets: 22136
rxdata_octets: 377532
noptx_pdus: 0
scsicmd_pdus: 60
tmfcmd_pdus: 0
login_pdus: 0
text_pdus: 0
dataout_pdus: 0
logout_pdus: 0
snack_pdus: 0
noprx_pdus: 0
scsirsp_pdus: 60
tmfrsp_pdus: 0
textrsp_pdus: 0
datain_pdus: 56
logoutrsp_pdus: 0
r2t_pdus: 0
async_pdus: 0
rjt_pdus: 0
digest_err: 0
timeout_err: 0
iSCSI Extended:
tx_sendpage_failures: 0
rx_discontiguous_hdr: 0
eh_abort_cnt: 0

22) As root, change permissions on /etc/iscsi/iscsid.conf. I’m not sure why they haven’t fixed this clear text CHAP password in a file issue so just make sure only root can read/write the file.
> chmod 600 /etc/iscsi/iscsid.conf

23) On the NetApp storage you can verify the Lun and the server’s session.
> lun show -v /vol/MCHAPVOL/FED20_iSCSI_MCHAP_01
/vol/MCHAPVOL/FED20_iSCSI_MCHAP_01 5g (5368709120) (r/w, online, mapped)
Serial#: hoagPJvUgR5s
Share: none
Space Reservation: enabled (not honored by containing Aggregate)
Multiprotocol Type: linux
Maps: ISCSI_MCHAP_FED20=1

> iscsi session show -v
Session 28
Initiator Information
Initiator Name: iqn.1994-05.com.redhat:4622a8d25677
ISID: 00:02:3d:01:00:00
Initiator Alias: fed20

Session Parameters
SessionType=Normal
TargetPortalGroupTag=1000
MaxConnections=1
ErrorRecoveryLevel=0
AuthMethod=CHAP
HeaderDigest=None
DataDigest=None
ImmediateData=Yes
InitialR2T=No
FirstBurstLength=65536
MaxBurstLength=65536
Initiator MaxRecvDataSegmentLength=65536
Target MaxRecvDataSegmentLength=65536
DefaultTime2Wait=2
DefaultTime2Retain=0
MaxOutstandingR2T=1
DataPDUInOrder=Yes
DataSequenceInOrder=Yes
Command Window Size: 32

Connection Information
Connection 0
Remote Endpoint: 10.10.10.184:50977
Local Endpoint: 10.10.10.11:3260
Local Interface: e0a
TCP recv window size: 131400

Command Information
No commands active

Using Wireshark and Splunk to find iSCSI CHAP Negotiation Failures on VMware ESXi

02 Monday Dec 2013

Posted by Slice2 in iSCSI, NetApp, Security, VMware, Wireshark

≈ Leave a comment

Tags

iSCSI, NetApp, Security, VMware, Wireshark

This is a companion post to sniffing packets in ESXi I posted here.

Say you need to isolate traffic to troubleshoot iSCSI CHAP session negotiation failures between ESXi and NetApp storage.

Using Wireshark:

1) Dump the traffic to a pcap file and open it with Wireshark. Before you start the capture, change directories so you can easily recover the pcap file from the datastore in vCenter.

> cd /vmfs/volumes/datastore1
> tcpdump-uw -i vmk1 -s 1514 -w esxihost01.pcap
> CTRL+C
a) When done, in vCenter select the ESXi host you were sniffing packets on, then click the Configuration tab > Storage.
b) Right-click datastore1 (or the datastore were your pcap file is) and select Browse datastore.
c) Click download a file > select the location and click OK.
d) Double-click the file and it will open in Wireshark.
e) In Wireshark, in the upper left, enter iscsi.login.T in the Filter: field and click Apply. This only shows the iSCSI login packets. You can clearly see on the right in the Info column, packet 856 is an Authentication Failure packet.

Using Splunk:

Another way to see the authentication failure is with Splunk. Assuming your NetApp storage (or any vendor) is configured to send syslog to Splunk, you can easily find the event. Splunk is an excellent Syslog server. You can download and use it for free up to 500 Megs a day indexed. I won’t go into the Splunk configuration in this post. I’ll post that soon.

Download it from here: http://www.splunk.com/download?r=header

1) Login to the Splunk UI, click Search to launch the Search app, enter the string below and the results will be displayed.

> index=”*” host=”10.10.10.11″ “iSCSI” “failed”

– Note: replace the IP address with your storage controller hostname or IP.

HOWTO Secure iSCSI Luns Between VMware ESXi 5.1 and NetApp Storage with Bidirectional CHAP

29 Friday Nov 2013

Posted by Slice2 in iSCSI, Security, VMware

≈ Leave a comment

Tags

iSCSI, Security, VMware

This document demonstrates how to configure iSCSI bidirectional CHAP between ESXi 5.1 Update 1 and NetApp storage. There were to many screen captures for a regular post so I created the pdf below.

Versions in use for this document:
1) VMware vSphere vCenter 5.1 Update 1c Web Client
2) VMware ESXi 5.1 Update 1
a. Note: The ESXi software iSCSI initiator will be used for this document
3) NetApp Data ONTAP 7.3.7P3 (this also works with Data ONTAP version 8x)

The HOWTO is here: Bidirectional CHAP with vSphere 5.1u1c.pdf

HOWTO Secure iSCSI Luns Between CentOS Linux 5.10 and NetApp Storage with Mutual CHAP

17 Sunday Nov 2013

Posted by Slice2 in iSCSI, Linux, Security

≈ Leave a comment

Tags

iSCSI, Linux, Security

This post demonstrates how to enable two-way or mutual CHAP on iSCSI luns between CentOS Linux 5.10 and NetApp storage. The aggregate, lun and disk sizes are small in this HOWTO to keep it simple.

1) Install open-iscsi on your server.
> yum install iscsi-initiator*
> reboot (don’t argue with me, just do it!)

2) Display your server’s new iscsi initiator or iqn nodename.
> cat /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.1994-05.com.redhat:01241a79a24

4) Create the lun in the volume.
netapp> lun create -s 5g -t linux /vol/MCHAPVOL/CENTSOS510_iSCSI_MCHAP_01

5) Create an igroup and add the Linux iscsi nodename or iqn from step 2 above to it.
netapp> igroup create -i -t linux ISCSI_MCHAP_CENTOS510
netapp> igroup add ISCSI_MCHAP_CENTOS510 iqn.1994-05.com.redhat:01241a79a24
netapp> igroup show ISCSI_MCHAP_CENTOS510

ISCSI_MCHAP_CENTOS510 (iSCSI) (ostype: linux):
iqn.1994-05.com.redhat:01241a79a24 (not logged in)

6) Map the lun to the igroup and give it lun ID 01.
netapp> lun map /vol/MCHAPVOL/CENTSOS510_iSCSI_MCHAP_01 ISCSI_MCHAP_CENTOS510 01

7) Obtain the NetApp target nodename.
netapp> iscsi nodename
iqn.1992-08.com.netapp:sn.84167939

8) Set the CHAP secret on the NetApp controller.
netapp> iscsi security add -i iqn.1994-05.com.redhat:01241a79a24 -s chap -p MCHAPCENT510 -n iqn.1994-05.com.redhat:01241a79a24 -o NETAPPMCHAP -m iqn.1992-08.com.netapp:sn.84167939

netapp> iscsi security show
init: iqn.1994-05.com.redhat:01241a79a24 auth: CHAP Inbound password: **** Inbound username: iqn.1994-05.com.redhat:01241a79a24 Outbound password: **** Outbound username: iqn.1992-08.com.netapp:sn.84167939

9) On the server, edit your /etc/iscsi/iscsi.conf file and set the parameters below.
> vi /etc/iscsi/iscsid.conf:
node.startup = automatic
node.session.auth.authmethod = CHAP
node.session.auth.username = iqn.1994-05.com.redhat:01241a79a24
node.session.auth.password = MCHAPCENT510
node.session.auth.username_in = iqn.1992-08.com.netapp:sn.84167939
node.session.auth.password_in = NETAPPMCHAP
discovery.sendtargets.auth.authmethod = CHAP
discovery.sendtargets.auth.username = iqn.1994-05.com.redhat:01241a79a24
discovery.sendtargets.auth.password = MCHAPCENT510
discovery.sendtargets.auth.username_in = iqn.1992-08.com.netapp:sn.84167939
discovery.sendtargets.auth.password_in = NETAPPMCHAP
> wq!

10) On the server, restart the service and discover your iSCSI target (your storage system).
> service iscsi restart
> iscsiadm -m discovery -t st -p 10.10.10.11
10.10.10.11:3260,1000 iqn.1992-08.com.netapp:sn.84167939

> iscsiadm -m node (this should display the same as above)
10.10.10.11:3260,1000 iqn.1992-08.com.netapp:sn.84167939

On the NetApp storage console you should see the iSCSI sessions:
Sun Nov 17 07:51:00 EST [iscsi.notice:notice]: ISCSI: New session from initiator iqn.1994-05.com.redhat:01241a79a24 at IP addr 10.10.10.37
Sun Nov 17 07:56:38 EST [iscsi.notice:notice]: ISCSI: New session from initiator iqn.1994-05.com.redhat:01241a79a24 at IP addr 10.10.10.37

Verify the iSCSI session on the filer:
netapp> iscsi session show
Session 5
Initiator Information
Initiator Name: iqn.1994-05.com.redhat:01241a79a24
ISID: 00:02:3d:01:00:00
Initiator Alias: cent510

12) Stop and start the iscsi service on the server.
> service iscsi stop
Pause for 10 seconds and then run the next command.
> service iscsi start

13) From the server , check your session.
> iscsiadm -m session -P 1
Target: iqn.1992-08.com.netapp:sn.84167939
Current Portal: 10.10.10.11:3260,1000
Persistent Portal: 10.10.10.11:3260,1000
**********
Interface:
**********
Iface Name: default
Iface Transport: tcp
Iface Initiatorname: iqn.1994-05.com.redhat:01241a79a24
Iface IPaddress: 10.10.10.37
Iface HWaddress: <empty>
Iface Netdev: <empty>
SID: 2
iSCSI Connection State: LOGGED IN
iSCSI Session State: LOGGED_IN
Internal iscsid Session State: NO CHANGE

14) From the server, check the NetApp iSCSI details. Note there are two dashes “- -” in front of mode, targetname and portal.
> iscsiadm -–mode node –-targetname “iqn.1992-08.com.netapp:sn.84167939” –-portal 10.10.10.11:3260

15) From the server, find and format the new lun (new disk).
> cat /var/log/messages | grep “unknown partition table”
Nov 17 04:56:52 cent510 kernel: sdb: unknown partition table

> fdisk /dev/sdb

Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won’t be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

> fdisk /dev/sdb
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1018, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1018, default 1018):
Using default value 1018

Command (m for help): p

Disk /dev/sdb: 5368 MB, 5368709120 bytes
166 heads, 62 sectors/track, 1018 cylinders
Units = cylinders of 10292 * 512 = 5269504 bytes

Device Boot Start End Blocks Id System
/dev/sdb1 1 1018 5238597 83 Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

16) On the server, create the Linux file system on the new partition.
> mkfs -t ext3 /dev/sdb1
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
655360 inodes, 1309649 blocks
65482 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1342177280
40 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 26 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.

17) Verify the partition.
> blkid /dev/sdb1
/dev/sdb1: UUID=”90e6dc7f-13ac-42bd-b471-b6f8e6863414″ SEC_TYPE=”ext2″ TYPE=”ext3″

19) Add the new mount point to /etc/fstab.
> vi /etc/fstab
/dev/sdb1 /newiscsilun ext4 _netdev 0 0
> wq!

Note: the _netdev option is important so that it doesn’t try mounting the target before the network is available.

> df -h | grep newiscsilun
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 5.0G 139M 4.6G 3% /newiscsilun

23) On the NetApp storage you can verify the Lun and the server’s session.
> lun show -v /vol/MCHAPVOL/CENTSOS510_iSCSI_MCHAP_01
/vol/MCHAPVOL/CENTSOS510_iSCSI_MCHAP_01 5g (5368709120) (r/w, online, mapped)
Serial#: hoagPJupukXK
Share: none
Space Reservation: enabled
Multiprotocol Type: linux
Maps: ISCSI_MCHAP_CENTOS510=1

> iscsi session show -v
Session 7
Initiator Information
Initiator Name: iqn.1994-05.com.redhat:01241a79a24
ISID: 00:02:3d:01:00:00
Initiator Alias: cent510

Session Parameters
SessionType=Normal
TargetPortalGroupTag=1000
MaxConnections=1
ErrorRecoveryLevel=0
AuthMethod=CHAP
HeaderDigest=None
DataDigest=None
ImmediateData=Yes
InitialR2T=No
FirstBurstLength=65536
MaxBurstLength=65536
Initiator MaxRecvDataSegmentLength=65536
Target MaxRecvDataSegmentLength=65536
DefaultTime2Wait=2
DefaultTime2Retain=0
MaxOutstandingR2T=1
DataPDUInOrder=Yes
DataSequenceInOrder=Yes
Command Window Size: 32

Connection Information
Connection 0
Remote Endpoint: 10.10.10.37:44786
Local Endpoint: 10.10.10.11:3260
Local Interface: e0a
TCP recv window size: 131400

Command Information
No commands active

slice2

Tag Archives: iSCSI

HOWTO Secure iSCSI Luns Between FreeBSD 10.1 and NetApp Storage with Mutual CHAP

HOWTO Secure iSCSI Luns Between Ubuntu Server 14.10 and NetApp Storage with Mutual CHAP

HOWTO Secure iSCSI Luns Between Oracle Enterprise Linux 7 and NetApp Storage with Mutual CHAP

HOWTO Secure iSCSI Luns Between Red Hat Enterprise Linux 7 (Beta) and NetApp Storage with Mutual CHAP

HOWTO Secure iSCSI Luns Between Oracle Solaris 11 and NetApp Storage Using Bidirectional CHAP

HOWTO Secure iSCSI Luns Between Oracle Solaris 10 and NetApp Storage Using Bidirectional CHAP

HOWTO Secure iSCSI Luns Between Fedora 20 and NetApp Storage with Mutual CHAP

Using Wireshark and Splunk to find iSCSI CHAP Negotiation Failures on VMware ESXi

HOWTO Secure iSCSI Luns Between VMware ESXi 5.1 and NetApp Storage with Bidirectional CHAP

HOWTO Secure iSCSI Luns Between CentOS Linux 5.10 and NetApp Storage with Mutual CHAP