Category Archives: Security

HOWTO quickly STIG Firefox 59.01

24 Saturday Mar 2018

Posted by in Security

≈ Leave a comment

Tags

,

The latest Firefox STIG leaves out important details and lists settings that no longer exist. It took a few hours to get this config file to work with settings that are actually still available in 59.01. For reference, see: http://kb.mozillazine.org/About:config

Note that I use Notepad++ on Windows to avoid formatting issues with notepad.exe. Do yourself a favor and download the latest at https://notepad-plus-plus.org/

1) Create a file named local-settings.js and add one line. Using Notepad++, you can save it as a proper JavaScript file (.js):
pref(“general.config.filename”,”mozilla.cfg”);

2) Place local-settings.js file in the following locations depending on whether you have x32 or x64 bit Firefox:

For x64 bit:
C:\Program Files\Mozilla Firefox\defaults\pref

For x32 bit:
C:\Program Files (x86)\Mozilla Firefox\defaults\pref

3) Create a file called mozilla.txt. Add the text below (everything under contents of mozilla.txt staring with //Firefox). Launch a browser (Chrome, IE, Opera) and browse to:

http://www.alain.knaff.lu/howto/MozillaCustomization/cgi/byteshf.cgi

4) In the middle of the page, under Upload mozilla.txt to get mozilla.cfg (byteshift 13), click browse, and select your mozilla.txt file. Next, click Convert mozilla.txt to mozilla.cfg, and save the file when prompted.

5) Place the mozilla.cfg file into the root of the Firefox directory as show below.

For x64 bit:
C:\Program Files\Mozilla Firefox\

For x32 bit:
C:\Program Files (x86)\Mozilla Firefox\

6) Start Firefox. In the Location bar, enter about:config. Click I accept the risk. At the top of the page in the center, click Status to sort the status of the settings. All of the locked settings should be italicized with a status of locked.

Contents of mozilla.txt:

//Firefox settings that work from Mozilla_Firefox_V4R20 as of Firefox 59.01 March 2018
lockPref(“security.default_personal_cert”, “Ask Every Time”);
lockPref(“network.protocol-handler.external.shell”, false);
lockPref(“plugin.disable_full_page_plugin_for_types”, “application/pdf,application/doc,application/xls,application/bat,application/ppt,application/mdb,application/mde,application/fdf,application/xfdf,application/lsl,application/lso,application/lss,application/iqy,application/rqy,application/xlk,application/pot,application/pps,application/dot,application/wbk,application/ps,application/eps,application/wch,application/wcm,application/wbi,application/wb1,application/wb3,application/rtf,application/wch,application/wcm,application/ad,application/adp,application/xlt,application/dos,application/wks”);
lockPref(“browser.formfill.enable”, false);
lockPref(“signon.autofillForms”, false);
lockPref(“signon.autofillForms.http”, false);
lockPref(“signon.rememberSignons”, false);
lockPref(“dom.disable_window_open_feature.status”, true);
lockPref(“dom.disable_window_move_resize”, true);
lockPref(“security.tls.version.min”, 2);
lockPref(“security.tls.version.max”, 3);
lockPref(“dom.disable_window_flip”, true);
lockPref(“dom.event.contextmenu.enabled”, false);
lockPref(“dom.disable_window_open_feature.status”, true);
lockPref(“app.update.enabled”, false);
lockPref(“extensions.update.enabled”, false);
lockPref(“browser.search.update”, false);
lockPref(“datareporting.policy.dataSubmissionEnabled”, false);

7) Click the link below for a copy of the files. The zip has a converted mozilla.cfg, the source mozilla.txt, local-settings.js and a README file.

Firefox_STIGv4R20

HOWTO mount a Synology NAS SMB share on Linux with SMBv1 disabled

11 Sunday Mar 2018

Posted by in Linux, Security

≈ Leave a comment

If you haven’t disabled SMBv1 everywhere, on every PC, NAS, server, you should.

https://support.microsoft.com/en-us/help/2696547/how-to-detect-enable-and-disable-smbv1-smbv2-and-smbv3-in-windows-and

https://www.synology.com/en-uk/knowledgebase/DSM/help/DSM/AdminCenter/file_winmacnfs_win

After disabling SMBv1 on a Synology NAS with DSM 6.1.5-15254 Update 1, I could no longer mount the shares from Linux. On Linux Mint 18.3 with KDE, you can’t select SMBv2 or 3 in the Dolphin or Smb4K GUI (yet) so you have to mount it from the cli.

Both smbclient and mount worked when I specified the SMB version. Note that both commands will prompt you for your password.

user1@lmint ~ $ sudo mount -t cifs //192.168.1.10/data /home/user1/Synology -o username=user1,vers=2.0,sec=ntlmv2

user1@lmint ~ $ smbclient ‘//192.168.1.10/data’ -m SMB2

HOWTO quickly STIG Firefox 45.0.1

22 Tuesday Mar 2016

Posted by in Security

≈ Leave a comment

Tags

,

This Firefox STIG leaves out important details. They could make it very simple to implement but they don’t.  For more, see: http://kb.mozillazine.org/About:config

1) Create a file called local-settings.js and add one line:
pref(“general.config.filename”,”mozilla.cfg”);

2) Place local-settings.js in:

c:\<firefox path>\defaults\pref folder.

3) Create a file called mozilla.txt. Add the text below (everything under contents of mozilla.txt).  Launch Firefox and browse to:
http://www.alain.knaff.lu/howto/MozillaCustomization/cgi/byteshf.cgi

4) View the middle of the page. Under Upload mozilla.txt to get mozilla.cfg (byteshift 13), click browse, select your mozilla.txt file. Next, click Convert mozilla.txt to mozilla.cfg, save the file when prompted and place it in c:\<firefox path>\Mozilla Firefox\.

Contents of mozilla.txt:

//Firefox Settings
lockPref(“security.tls.version.min”, 1);
lockPref(“security.default_personal_cert”, “Ask Every Time”);
lockPref(“network.protocol-handler.external.shell”, false);
lockPref(“plugin.disable_full_page_plugin_for_types”, “application/pdf,application/fdf,application/xfdf,application/lsl,application/lso,application/lss,application/iqy,application/rqy,application/xlk,application/xls,application/xlt,application/pot,application/pps,application/ppt,application/dos,application/dot,application/wks,application/bat,application/ps,application/eps,application/wch,application/wcm,application/wb1,application/wb3,application/rtf,application/doc,application/mdb,application/mde,application/ad,application/,application/adp”);
lockPref(“browser.formfill.enable”, false);
lockPref(“signon.autofillForms”, false);
lockPref(“signon.rememberSignons”, false);
lockPref(“privacy.sanitize.sanitizeOnShutdown”, true);
lockPref(“dom.disable_window_open_feature.status”, true);
lockPref(“dom.disable_window_move_resize”, true);
lockPref(“security.tls.version.max”, 3);
lockPref(“dom.disable_window_flip”, true);
lockPref(“dom.event.contextmenu.enabled”, false);
lockPref(“dom.disable_window_status_change”, true);
lockPref(“dom.disable_window_open_feature.status”, true);
lockPref(“browser.startup.homepage”, “about:home”);
lockPref(“app.update.enabled”, false);
lockPref(“extensions.update.enabled”, false);
lockPref(“browser.search.update”, false);
// The end

Completing the vSphere vCenter Appliance Hardening Process

19 Saturday Mar 2016

Posted by in Linux, Security, VMware

≈ Leave a comment

Tags

, , ,

The vCenter Appliance is a SuSE Linux VM that ships fully hardened by VMware to the DoD STIG specifications. There are a few site specific settings you must perform to complete the hardening. This post provides the steps to complete the process.

1) Change the root password. Login to the appliance and as root, run:
> passwd
New password:
Retype new password:
Password changed.

> cat /etc/shadow | grep root
root:$6$(truncated)

If the root password starts with a $6$ hash, this confirms it is using a sha512 hash.

2) Set password expiry. Change the root expiry from 3 years to 1 year.
> passwd -x 365 root
Password expiry information changed.

3) Execute the Dodscript.sh script.
> cd /etc/
> ./dodscript.sh
Shutting down auditd                done
Starting auditd

4) You may be a company or site that has a custom banner. If so, edit (and verify) the banners to replace the DoD language with your own.
> vi /opt/vmware/etc/isv/welcometextDoD and paste in your banner.
> cat /opt/vmware/etc/isv/welcometextDoD > /etc/issue  (linked to issue.DoD)
> cat /opt/vmware/etc/isv/welcometextDoD > /opt/vmware/etc/isv/welcometext
> cat /opt/vmware/etc/isv/welcometextDoD > /opt/vmware/etc/isv/welcometext.template

5) Configure secure shell, admin accounts, and console access on the appliance. Add a user account that can su to root:
> useradd -s /bin/bash -m -d /home/(your username) -g users -G wheel (your username)

> passwd <your username>
Changing password for (your username)
New password:
Retype new password:
Password changed.

> su – (your username) to verify.

a) Test ability to su to root and verify identity:
> su – root
Password:
Last login: Sat 19 12 12:51:26 UTC 2016 from PC on pts/1
Directory: /root
Tue Mar 19 13:18:33 UTC 2016

> whoami
root

b) Note: The step below isn’t included in the documentation but if you don’t do it you will be locked out.
> vi /etc/security/access.conf

At the end of the file change -:ALL:ALL to +:ALL:ALL

c) Test that the user you just created can login via SSH and su – root before you proceed. Use ssh cli, PuTTY, etc.
> ssh -v (your username)@(your vCenter appliance hostname or IP)
Once logged in:
> su – root

d) Disable direct root SSH access to the appliance.
> vi /etc/ssh/sshd_config

change PermitRootLogin yes to PermitRootLogin no

e) Restrict SSH to the local network of the appliance.
> vi /etc/hosts.allow and add the following:

sshd:127.0.0.1:ALLOW
sshd:[::1]:ALLOW
sshd:(the same network your appliance is on):ALLOW

f) Restart sshd to read the changes:

> service sshd restart
Shutting down SSH daemon                  done
Starting SSH daemon

g) Disable direct root console login on the appliance. This means you must first login as a user and su to root. After setting this, when you try to login on the appliance console as root, it should say login incorrect.
> vi /etc/securetty

Set the first two lines as follows:
#tty1
console

6) Verify time synchronization. Recall that NTP is configured when you first import and setup the appliance. As root, verify:

> service ntp status
remote           refid      st t when poll reach   delay   offset  jitter
=============================================
192.168.1.252    .LOCL.       1 u   37   64    1    1.145  459.906   0.001
192.168.1.252     192.168.1.253  2 u   36   64    1    1.273  464.924   0.001

Checking for network time protocol daemon (NTPD):    Running

7) Setup log forwarding with syslog-ng and auditd. Uncomment and edit the following lines to fit your remote syslog server IP address:
> vi /etc/syslog-ng/syslog-ng.conf

destination logserver { udp(“Syslog_svr_IP_Address” port(514));};
log {source(src); destination(logserver);};

a) Restart the service.
> service syslog restart
Shutting down syslog services               done
Starting syslog services

b) Send your audit data to syslog.
> vi /etc/audisp/plugins.d/syslog.conf

change active=no to active=yes

c) Restart auditd.
> service auditd restart
Shutting down auditd                         done
Starting auditd

d) Tune audit performance.
> vi /etc/audisp/audispd.conf

change the following to 1280 and 8
q_depth = 1280
priority_boost = 8

e) Control the number and rotation of log files.
> vi /etc/logrotate.d/syslog

change all entries for rotate 15 to rotate 7

> vi /etc/logrotate.d/audit

change all entries for rotate 15 to rotate 7

8) Set a boot loader or grub password.
> cat /boot/grub/menu.lst | grep password

password –md5 (a_long_hash_will_be_here)

a) Create a password for grub. This is how the sequence goes: you enter grub and run the md5crypt command to create a hashed password. Once you type in the password, the hash is presented. Copy the password hash. Run the quit command to return to the root shell.

> grub

grub> md5crypt

Password: (Enter your password here)
Encrypted: (a_long_hash_will_be_here)
grub> quit

b) Add the following to the third line of the file:
> vi /boot/grub/menu.lst

password –md5 (the password hash from above)

9) Configure NFS and NIS. If you are not using NFS or NIS, disabled the services. You probably aren’t using them.
> chkconfig ypbind off
> chkconfig nfs off
> chkconfig rpcbind off
> service ypbind stop
> service nfs stop
> service rpcbind stop

10) Reboot to refresh your system and seat all of the changes.
> reboot

Enabling TLS 1.2 on the Splunk 6.2x Console and Forwarders using Openssl and self signed certs.

01 Friday Jan 2016

Posted by in Security, Splunk

≈ Leave a comment

Tags

,

Good luck. You will need it. Certificates are a major headache and complicated to implement. Using them with Splunk is no different. Splunk’s penchant for twiddling files all over the place makes this process time consuming an rife with error. This post will hopefully help you get it done. This covers encrypting the management console and forwarder traffic. This HOWTO is not for a clustered deployment although it could be adapted to serve that purpose. It was done on Windows 2012 R2 with a single Splunk Enterprise deployment (search head and indexer on the same server) and several forwarders. Use your own naming conventions and hosts for fqdn. Please don’t ask me questions on this post. I almost didn’t survive the process. I won’t have time to reply for a while anyway.

1) On the Splunk Search Head, set your environment.

> cd C:\Program Files\Splunk\bin
> splunk envvars > setsplunkenv.bat & setsplunkenv.bat
> setsplunkenv.bat

2) Create dir $SPLUNK_HOME\etc\auth\UScerts and cd into it.

> cd C:\Program Files\Splunk\etc\auth\UScerts

3) Create a root key.
> openssl genrsa -aes256 -out USCA_root.key 2048

4) Generate and sign the certificate.
> openssl req -new -key USCA_root.key -out USCA_root.csr

5) Generate the public certificate.
> openssl x509 -req -in USCA_root.csr -sha256 -signkey USCA_root.key -CAcreateserial -out USCA_root.pem -days 3650

6) Generate a key for your Web(search head)server certificate.
> openssl genrsa -aes256 -out me.fqdn.com.key 2048

7) Request and sign a new server certificate.
> openssl req -new -key me.fqdn.com.key -out me.fqdn.com.csr

8) Use the CSR me.fqdn.com.csr and your CA certificate and private key to generate a server certificate.
> openssl x509 -req -in me.fqdn.com.csr -sha256 -CA USCA_root.pem -CAkey USCA_root.key -CAcreateserial -out
me.fqdn.com.pem -days 730

9) Creating a (removing encryption from priv key) priv key without a passphrase. Required for webservers.
> openssl rsa -in me.fqdn.com.key -out me.fqdn.com_nopass.key

10) Create a combined cert file.
> type me.fqdn.com.pem me.fqdn.com_nopass.key USCA_root.pem > me.fqdn.com_nopass_use.pem

11) On the search head, edit the \etc\system\local\web.conf and add the following:
[settings]
enableSplunkWebSSL = 1
httpport = 8843
privKeyPath = etc\auth\UScerts\me.fqdn.com_nopass_use.pem
CaCertPath = etc\auth\UScerts\USCA_root.pem

Add to \etc\system\local\server.conf

enableSplunkdSSL = true
sslVersions = tls1.2
allowSslCompression = false
allowSslRenegotiation = false
cipherSuite = TLSv1+HIGH:@STRENGTH

12) Restart Splunk. Close your browser, relaunch and login to the console to verify (make sure to use the port defined above in web.conf; https://hostname or ip:8843).  If you scan with Nessus, Retina, etc., it should now be free from SSL errors.

Certs for Forwarders:

Create a SAN (subject alternative name) cert. Although not officially supported by Splunk when I originally wrote this, it does work.

1) Create a new folder in etc\auth\UScerts\SANcert.

2) Copy the openssl.cnf to the new folder.  C:\Program Files\Splunk\openssl.cnf to C:\Program Files\Splunk\etc\auth\UScerts\SANcert

> cd C:\Program Files\Splunk\etc\auth\UScerts\SANcert

3) In Windows 2012 R2 – Take ownership of the copied openssl.cnf file. Right-click > properties, and then add your user with Full Control to the file.

4) In Notepad or Wordpad, edit openssl.cnf. Wordpad is preferred.

a) Search (using the Find function in the upper right of Wordpad) and uncomment this line:
# req_extensions = v3_req # The extensions to add to a certificate request

b) Next, search for and modify this section to include the following if it does not already have it:
[ v3_req ] # Extensions to add to a certificate request
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAtlName = @alt_names

c) Create this section next, between [ v3_req ] and [ v3_ca ].
[alt_names]
DNS.1 = your.server.com
DNS.2 = your.next.server.com
DNS.3 = your.other.server.com
IP.1 = an IP address for a server
IP.2 = another IP address for a server
— note, add as many as you like. You will need one for each forwarder if you want to identify them individually.

5) Generate a new CSR.
openssl req -new -key me.fqdn.com.key -out me.fqdn.com_SAN.csr -config “C:\Program Files\Splunk\etc\auth\UScerts\SANcert\openssl.cnf”

Make sure you use: *.your.fqdn for Common name question. This is the wildcard for your domain, such as *.yourdomain.com

6) Check text of cert csr. You should see the items in the alt_names from above.
> openssl req -text -noout -in me.fqdn.com_SAN.csr

7) Create a cert.
> openssl x509 -req -in me.fqdn.com_SAN.csr -sha256 -CA USCA_root.pem -CAkey USCA_root.key -CAcreateserial -out me.fqdn.com_SAN.pem -extensions v3_req -days 730 -extfile “C:\Program Files\Splunk\etc\auth\UScerts\SANcert\openssl.cnf”

8) On the Indexer, edit \etc\system\local\inputs.conf and add the following and restart Splunk.

[SSL]
rootCA = etc\auth\UScerts\USCA_root.pem
servercert = etc\auth\UScerts\me.fqdn.com_SAN.pem
password = your_password
cipherSuite = TLSv1+HIGH:@STRENGTH

[splunktcp-ssl:9997]
compressed = false

9) Now restart splunk:
$SPLUNK_HOME\bin\splunk restart splunkd

10) Configure your Forwarders to use the certificates. Use your Deployment Server to distribute the certs and modified outputs.conf to your forwarders.

a) On the Search head that is acting as your deployment server, edit the outputs.conf file in etc\deployment-apps\<your name for SendToIndexer>\local\ with the following.

[tcpout]
defaultGroup = splunkssl

[tcpout:splunkssl]
server = your.ip.:9997
compressed = false
sslRootCAPath = etc\apps\<your name for SendToIndexer>\USCA_root.pem
sslCertPath = etc\apps\<your name for SendToIndexer>\me.fqdn.com_SAN.pem
sslPassword = <your password>
sslVerifyServerCert = true

b) Copy the etc\auth\UScerts\USCA_root.pem and etc\auth\UScerts\me.fqdn.com_SAN.pem files to the etc\deployment-apps\<your name for SendToIndexer>\local folder on your deployment server and they will be copied to each Forwarder for you.

11) Restart Splunk.
$SPLUNK_HOME\bin\splunk restart splunkd

12) Done. I hope.

Apply a Windows 2012 R2 Domain GPO to a standalone Windows 2012 R2 server

18 Saturday Jul 2015

Posted by in Security, Windows

≈ Leave a comment

Tags

,

This post demonstrates how to apply a Windows 2012 R2 Domain GPO to a standalone Windows 2012 R2 server that is not in the domain. For this example, I’ll use the Internet Explorer 11 (IE11) lock downs I applied using a domain GPO.

This process also worked when I applied the 2012 R2 IE policy to a standalone Windows 7 Enterprise workstation.

1) Launch Group Policy Management on the Domain Controller. Browse to the policy you want to apply to the standalone servers (in my case IE11), right-click it and select Backup. Save it to a location of your choice and give it a description, such as IE11 GPO.

2) Download and install Microsoft SCM 3.0 (not on your domain controller). I just built a VM since SCM is only needed temporarily. I was only able to get it fully installed without errors on Windows 2008 R2. It supposedly supports Vista through 2012. I opted to install the bundled SQL Express since all I want is the LocalGPO executable. No need to point to a SQL server. You can uninstall the whole thing when done. The only reason to install the full package is so you can get a copy of the LocalGPO folder. Download it from:

Security Compliance Manager (SCM) Info:
https://technet.microsoft.com/en-us/solutionaccelerators/cc835245.aspx

Download page:
https://www.microsoft.com/en-us/download/details.aspx?id=16776

a) When done downloading, double-click the Security_Compliance_Manager_Setup.exe > click Run > deselect Always check for SCM baseline updates (you don’t care about them right now) and click Next > accept the license and click Next > Next > Next > accept the SQL Express license and click Next > Install > Finish. The app will auto-load the baselines. Just let it finish.

b) When done installing, browse to C:\Program Files (x86)\Microsoft Security Compliance Manager. Copy the LocalGPO folder to a location of your choice. You will need to install the executable in this folder on each standalone server that will receive the Domain GPO.

5) Login as a local admin on the server to receive the GPO. Install LocalGPO on your standalone server. When done, browse to the C:\Program Files (x86)\LocalGPO folder, right-click LocalGPO.wsf, select Properties, select the Security tab and give your admin user full control of the file.

6) Create a folder on this server called c:\gpos. Copy your IE11 GPO backup folder into the c:\gpos folder.

7) Edit the LocalGPO.wsf file to recognize 2012 R2 (Windows 2012 R2 is version 6.3). Open C:\Program Files (x86)\LocalGPO\LocalGPO.wsf in notepad (right-click > Edit). Search for 6.2. On the first instance of 6.2, change it to 6.3.

From this: If(Left(strOpVer,3) = “6.2”) and (strProductType <> “1”) then

To this: If(Left(strOpVer,3) = “6.3“) and (strProductType <> “1”) then

8) The Windows Firewall must be running temporarily before you run this tool. Even though you may have disabled the firewall and use a third-party product like McAfee Firewall, etc., turn on the native Windows firewall in the services.msc applet now.

9) Click start (lower left corner), and then Search icon in the upper right. Enter LocalGPO. Right-click LocalGPO Command line and select Run as Administrator. Before you run the next command, close all Windows except the cmd prompt.

Enter this command:

> cscript localgpo.wsf /path:”C:\gpos\{A81C84F4-F8F5-4E8A-B077-9EA1471B3886}”

– note: your IE11 GPO backup folder name inside c:\gpos will be different. Just add your folder name in the command above.

You should see Applied valid Machine POL and Applied valid User POL. No valid audit or INF is OK.

10) Clean up after yourself. Uninstall LocalGPO if you don’t plan to use it again. Delete the gpo backup in c:\gpos.

You can run > gpupdate /force or reboot the server to apply the policy completely.

11 Verify that it applied the policy. Launch IE11 and verify your settings are locked down. Note that on a fresh system, you  may have to launch IE and then immediately close it. Launch it again and the lock downs will be set. Sometimes it takes two startups for the settings to apply. Not sure why. If you had the Windows firewall turned off, open services.msc and disable it.

Enable legacy SSL and Java SSL support in your browser for those old, crusty websites

08 Monday Jun 2015

Posted by in Security

≈ Leave a comment

Tags

,

This is a quick post to show you how to enable legacy SSL and Java SSL support in your browser for those old, crusty websites and applications you have in your organization. Note that this should not be done on Internet facing systems. Only offline or systems that are not routed to the Internet should implement these changes.

1) Launch Firefox. Type about:config in the location bar.
2) In the search bar that comes up, enter: security.tls.version.min. Double-click on the entry that appears and change the value to 0.
3) Do the same for security.tls.version.fallback-limit.
4) Try to connect to your site. It should now work for you.

Enable SSL in Java (it has been disabled for a few rev’s now)
1) Open Windows explorer and browse to either (or both if you have x32/x64 bit Java installed):

C:\Program Files (x86)\Java\jre1.8.0_45\lib\security
C:\Program Files\Java\jre1.8.0_45\lib\security

2) Double-click the file named java.security. You will be prompted to select a program to open the file. Choose select a program from a list of installed programs and click OK. Choose either Wordpad or notepad.

3) Scroll down to the bottom of the file. You should see:
jdk.tls.disabledAlgorithms=SSLv3

4) Change this by back-spacing over SSLv3 and save the file so it looks like:
jdk.tls.disabledAlgorithms=

You should now be able to access legacy sites with Java SSL support.

HOWTO Secure iSCSI Luns Between FreeBSD 10.1 and NetApp Storage with Mutual CHAP

01 Sunday Feb 2015

Posted by in FreeBSD, iSCSI, Security

≈ Leave a comment

Tags

, ,

This post demonstrates how to enable bidirectional or mutual CHAP on iSCSI luns between FreeBSD 10.1 and NetApp storage. The aggregate, lun and disk sizes are small in this HOWTO to keep it simple.  Special thanks to FreeBSD developer Edward Napierala for his help with testing -CURRENT updates and syntax issues.

1) On the NetApp filer, create the volume that will hold the iscsi luns. This command assumes you have aggregate aggr1 already created. If not, use an aggregate that has enough room for your volume.
netapp> vol create MCHAPVOL aggr1 10g

2) On the NetApp filer, create a lun in the volume. Type solaris is used to be compatible with UFS.
netapp> lun create -s 5g -t solaris /vol/MCHAPVOL/FBSD10_iSCSI_MCHAP_0

3) Obtain the NetApp target nodename.
netapp> iscsi nodename
iSCSI target nodename: iqn.1992-08.com.netapp:sn.4055372815

4) On the FreeBSD server, create an iSCSI session to the NetApp. Unfortunately, there is no way of obtaining the FreeBSD server iqn in advance. Hopefully this
function will be in the next release.
server> iscsictl -A -p 10.10.10.141 -t iqn.1992-08.com.netapp:sn.4055372815

5) Now that a basic session is established, obtain your server’s iqn.
server> iscsictl -Lv | grep “Initiator name”
Initiator name:   iqn.1994-09.org.freebsd:fbsd101

6) On the NetApp filer, create an iGroup and add the FreeBSD iscsi nodename or iqn from step 5 above. The iGroup type is Solaris because its the most similar to FreeBSD. BSD is not an iGroup option.
netapp> igroup create -i -t solaris ISCSI_MCHAP_FBSD10
netapp> igroup add ISCSI_MCHAP_FBSD10 iqn.1994-09.org.freebsd:fbsd101
netapp> igroup show

ISCSI_MCHAP_FBSD10 (iSCSI) (ostype: solaris):
iqn.1994-09.org.freebsd:fbsd101 (not logged in)

7) Map the lun to the iGroup and give it lun ID 0. Important note: as of FreeBSD 10.1, there is a lun enumeration limit that requires you to start your lun ID with 0. You cannot start with 1. This is fixed in the current baseline and will appear in 10.2.
netapp> lun map /vol/MCHAPVOL/FBSD10_iSCSI_MCHAP_0 ISCSI_MCHAP_FBSD10 0

8) Set the CHAP secret on the NetApp controller.
netapp> iscsi security add -i iqn.1994-09.org.freebsd:fbsd101 -s chap -p FREEBSD -n iqn.1994-09.org.freebsd:fbsd101 -o NETAPP -m iqn.1992-08.com.netapp:sn.4055372815

netapp> iscsi security show
init: iqn.1994-09.org.freebsd:fbsd101 auth: CHAP Local Inbound password: **** Inbound username: iqn.1994-09.org.freebsd:fbsd101 Outbound password: ****  Outbound username: iqn.1992-08.com.netapp:sn.4055372815

9) Configure iSCSI on the server.
a) Start iSCSI when booted:
server> echo iscsid_enable=”YES” >> /etc/rc.conf

b) This tells it to connect to all targets when booted.
server> echo iscsictl_enable=”YES” >> /etc/rc.conf

c) Create the iscsi.conf file. Explanation for the example below:

Example:
t0                       = Represents the target ID. Start with 0.
TargetAddress   = 10.10.10.141 (your NetApp controller)
TargetName      = iqn.1992-08.com.netapp:sn.4055372815 (your NetApp iqn)
AuthMethod      = CHAP (the authentication type)
chapIName       = iqn.1994-09.org.freebsd:fbsd101 (your FreeBSD server iqn)
chapSecret      = FREEBSD (your server’s chap secret)
tgtChapName    = iqn.1992-08.com.netapp:sn.4055372815 (your Netapp iqn)
tgtChapSecret    = NETAPP (your NetApp chap secret)

Actual:
server> vi /etc/iscsi.conf

t0 {
TargetAddress   = 10.10.10.141
TargetName      = iqn.1992-08.com.netapp:sn.4055372815
AuthMethod      = CHAP
chapIName       = iqn.1994-09.org.freebsd:fbsd101
chapSecret      = FREEBSD
tgtChapName    = iqn.1992-08.com.netapp:sn.4055372815
tgtChapSecret    = NETAPP
}

wq!

d) Change permissions on the file.
> chmod 500 /etc/iscsi.conf

e) Start the iSCSI server on the server.
server> service iscsid start
Starting iscsid.

f) Reboot to test boot and session parameters. The session should connect and your lun will be visible.
server> reboot

Note: if you can’t reboot, run the following:
server> iscsictl -An t0

10) Verify your new lun.
server>  cat /var/log/messages | grep da1
Feb  1 10:38:12 fbsd101 kernel: da1 at iscsi1 bus 0 scbus3 target 0 lun 0
Feb  1 10:38:12 fbsd101 kernel: da1: <NETAPP LUN 811a> Fixed Direct Access SCSI-4 device
Feb  1 10:38:12 fbsd101 kernel: da1: Serial Number BQVJ3]DxwBcF
Feb  1 10:38:12 fbsd101 kernel: da1: 150.000MB/s transfers
Feb  1 10:38:12 fbsd101 kernel: da1: Command Queueing enabled
Feb  1 10:38:12 fbsd101 kernel: da1: 5120MB (10485760 512 byte sectors: 255H 63S/T 652C)

server> camcontrol reportluns /dev/da1
1 LUN found
0

server> camcontrol devlist -v | grep NETAPP
<NETAPP LUN 811a> at scbus3 target 0 lun 0 (da1,pass2)

server> camcontrol inquiry /dev/da1
pass2: <NETAPP LUN 811a> Fixed Direct Access SCSI-4 device
pass2: Serial Number BQVJ3]DxwBcF
pass2: 150.000MB/s transfers, Command Queueing Enabled

11) Obtain session details on the server.
server> iscsictl -Lv
Session ID:       1
Initiator name:   iqn.1994-09.org.freebsd:fbsd101
Initiator portal:
Initiator alias:
Target name:      iqn.1992-08.com.netapp:sn.4055372815
Target portal:    10.10.10.141
Target alias:
User:             iqn.1994-09.org.freebsd:fbsd101
Secret:           FREEBSD
Mutual user:      iqn.1992-08.com.netapp:sn.4055372815
Mutual secret:    NETAPP
Session type:     Normal
Session state:    Connected
Failure reason:
Header digest:    None
Data digest:      None
DataSegmentLen:   65536
ImmediateData:    Yes
iSER (RDMA):      No
Device nodes:     da1

a) Verify the iSCSI session on the filer:
netapp> iscsi session show
Session 1
Initiator Information
Initiator Name: iqn.1994-09.org.freebsd:fbsd101
ISID: 80:33:9b:8b:a9:6d

12) From the server, format the new lun (new disk). Since you know the device ID from step 10 above (/dev/da1), perform the following steps.

a) Create the partition.
server> gpart create -s gpt da1
da1 created

b) Display the partition.
server> gpart show da1
=>      40  10485680  da1  GPT  (5.0G)
40  10485680  – free –  (5.0G)

c) Add a slice.
server> gpart add -t freebsd da1
da1s1 added

d) Format the slice.
server> newfs -L ntaplun -b 4096 /dev/da1s1
/dev/da1s1: 5120.0MB (10485680 sectors) block size 4096, fragment size 4096
using 107 cylinder groups of 48.12MB, 12320 blks, 6160 inodes.
super-block backups (for fsck_ffs -b #) at:
144, 98704, 197264, 295824, 394384, 492944, 591504, 690064, 788624, 887184, 985744, 1084304, 1182864,
1281424, 1379984, 1478544, 1577104, 1675664, 1774224, 1872784, 1971344, 2069904, 2168464, 2267024,
2365584, 2464144, 2562704, 2661264, 2759824, 2858384, 2956944, 3055504, 3154064, 3252624, 3351184,
3449744, 3548304, 3646864, 3745424, 3843984, 3942544, 4041104, 4139664, 4238224, 4336784, 4435344,
4533904, 4632464, 4731024, 4829584, 4928144, 5026704, 5125264, 5223824, 5322384, 5420944, 5519504,
5618064, 5716624, 5815184, 5913744, 6012304, 6110864, 6209424, 6307984, 6406544, 6505104, 6603664,
6702224, 6800784, 6899344, 6997904, 7096464, 7195024, 7293584, 7392144, 7490704, 7589264, 7687824,
7786384, 7884944, 7983504, 8082064, 8180624, 8279184, 8377744, 8476304, 8574864, 8673424, 8771984,
8870544, 8969104, 9067664, 9166224, 9264784, 9363344, 9461904, 9560464, 9659024, 9757584, 9856144,
9954704, 10053264, 10151824, 10250384, 10348944, 10447504

13)  Create the mount point and manually mount the directory.
server> mkdir /newiscsilun
server> mount /dev/da1s1 /newiscsilun
server> df -h | grep newiscsilun
/dev/da1s1    4.8G    8.0K    4.5G     0%    /newiscsilun

14) Add the new mount point to /etc/fstab.
server>  echo “/dev/da1s1 /newiscsilun ufs rw 1 1” >> /etc/fstab

15) Test that it survives a reboot by rebooting the server now.
server> reboot

a) After the reboot, login and check the lun mount point.
server> df -h | grep newiscsilun
/dev/da1s1    4.8G    8.0K    4.5G     0%    /newiscsilun

16) On the NetApp storage you can verify the lun and the server’s session.
netapp> lun show -v /vol/MCHAPVOL/FBSD10_iSCSI_MCHAP_0
lun show -v /vol/MCHAPVOL/FBSD10_iSCSI_MCHAP_0
/vol/MCHAPVOL/FBSD10_iSCSI_MCHAP_0      5g (5368709120)    (r/w, online, mapped)
Serial#: BQVJ3]DxwBcF
Share: none
Space Reservation: enabled
Multiprotocol Type: solaris
Maps: ISCSI_MCHAP_FBSD10=0
Occupied Size:    3.8m (4022272)
Creation Time: Sun Feb  1 10:10:17 EST 2015
Cluster Shared Volume Information: 0x0

a) On the NetApp controller, get stats with the command below.
netapp>  iscsi session show -v
Session 2
Initiator Information
Initiator Name: iqn.1994-09.org.freebsd:fbsd101
ISID: 80:0d:2c:82:e2:2a

Session Parameters
SessionType=Normal
TargetPortalGroupTag=1000
MaxConnections=1
ErrorRecoveryLevel=0
AuthMethod=CHAP
HeaderDigest=None
DataDigest=None
ImmediateData=Yes
InitialR2T=Yes
FirstBurstLength=65536
MaxBurstLength=65536
Initiator MaxRecvDataSegmentLength=65536
Target MaxRecvDataSegmentLength=65536
DefaultTime2Wait=0
DefaultTime2Retain=0
MaxOutstandingR2T=1
DataPDUInOrder=Yes
DataSequenceInOrder=Yes
Command Window Size: 64

Connection Information
Connection 0
Remote Endpoint: 10.10.10.61:46881
Local Endpoint: 10.10.10.141:3260
Local Interface: e0a
TCP recv window size: 131400

Command Information
No commands active

HOWTO Secure iSCSI Luns Between Ubuntu Server 14.10 and NetApp Storage with Mutual CHAP

29 Monday Dec 2014

Posted by in iSCSI, Linux, Security

≈ Leave a comment

Tags

, ,

This post demonstrates how to enable two-way or mutual CHAP on iSCSI luns between Ubuntu Server 14.10 and NetApp storage. The aggregate, lun and disk sizes are small in this HOWTO to keep it simple. Note that Ubuntu follows the no root model so you must use sudo on privileged command. I will not show every “enter password” prompt in the doc. When you see it, just enter your password.

1) Install open-iscsi on your server.
> sudo apt-get install open-iscsi
> sudo reboot (don’t argue with me, just do it!)

2) Display your server’s new iscsi initiator or iqn nodename.
> sudo cat /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.1993-08.org.debian:01:ce45f9ecc9b

3) On the NetApp filer, create the volume that will hold the iscsi luns. This command assumes you have aggregrate aggr1 already created. If not use an aggregate that has enough room for your volume.
netapp> vol create MCHAPVOL aggr1 10g

4) On the NetApp filer, create the lun in the volume.
netapp> lun create -s 5g -t linux /vol/MCHAPVOL/UB1410_iSCSI_MCHAP_01

5) On the NetApp filer, create an igroup and add the Linux iscsi nodename or iqn from step 2 above to it.
netapp> igroup create -i -t linux ISCSI_MCHAP_UB1410
netapp> igroup add ISCSI_MCHAP_UB1410 iqn.1993-08.org.debian:01:ce45f9ecc9b
netapp> igroup show

ISCSI_MCHAP_UB1410 (iSCSI) (ostype: linux):
iqn.1993-08.org.debian:01:ce45f9ecc9b (not logged in)

6) Map the lun to the iscsi-group and give it lun ID 01.
netapp> lun map /vol/MCHAPVOL/UB1410_iSCSI_MCHAP_01 ISCSI_MCHAP_UB1410 01

7) Obtain the NetApp target nodename.
netapp> iscsi nodename
iSCSI target nodename: iqn.1992-08.com.netapp:sn.4055372815

8) Set the CHAP secret on the NetApp controller.
netapp> iscsi security add -i iqn.1993-08.org.debian:01:ce45f9ecc9b -s chap -p MCHAPUB1410 -n iqn.1993-08.org.debian:01:ce45f9ecc9b -o NETAPPMCHAP -m iqn.1992-08.com.netapp:sn.4055372815

netapp> iscsi security show
init: iqn.1993-08.org.debian:01:ce45f9ecc9b auth: CHAP Local Inbound password: **** Inbound username: iqn.1993-08.org.debian:01:ce45f9ecc9b Outbound
password: **** Outbound username: iqn.1992-08.com.netapp:sn.4055372815

9) On the server, edit your /etc/iscsi/iscsi.conf file and set the parameters below. You can just copy this into the file under iscsid.startup = /usr/sbin/iscsid but make sure you comment out node.startup = manual just below it. Since you want it to start automatically comment out node.leading_login = No.

> sudo vi /etc/iscsi/iscsid.conf:
node.startup = automatic
node.session.auth.authmethod = CHAP
node.session.auth.username = iqn.1993-08.org.debian:01:ce45f9ecc9b
node.session.auth.password = MCHAPUB1410
node.session.auth.username_in = iqn.1992-08.com.netapp:sn.4055372815
node.session.auth.password_in = NETAPPMCHAP
discovery.sendtargets.auth.authmethod = CHAP
discovery.sendtargets.auth.username = iqn.1993-08.org.debian:01:ce45f9ecc9b
discovery.sendtargets.auth.password = MCHAPUB1410
discovery.sendtargets.auth.username_in = iqn.1992-08.com.netapp:sn.4055372815
discovery.sendtargets.auth.password_in = NETAPPMCHAP
> wq!

10) On the server, discover your iSCSI target (your NetApp storage system).
> sudo iscsiadm -m discovery -t st -p 10.10.10.141
10.10.10.141:3260,1000 iqn.1992-08.com.netapp:sn.4055372815

> sudo iscsiadm -m node  (this should display the same as above)
10.10.10.141:3260,1000 iqn.1992-08.com.netapp:sn.4055372815

11) On the server, manually login to the iSCSI target (your storage array). Note there are two “- -” dashed in front of targetname and login.
> sudo iscsiadm -m node –-targetname “iqn.1992-08.com.netapp:sn.4055372815” -–login

Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.4055372815, portal: 10.10.10.141,3260] (multiple)
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.4055372815, portal: 10.10.10.141,3260] successful.

On the NetApp storage console you should see the iSCSI sessions:
[iscsi.notice:notice]: ISCSI:
New session from initiator iqn.1993-08.org.debian:01:ce45f9ecc9b at IP addr 10.10.10.128

Verify the iSCSI session on the filer:
netapp> iscsi session show
Session 1
Initiator Information
Initiator Name: iqn.1993-08.org.debian:01:ce45f9ecc9b
ISID: 00:02:3d:01:00:00
Initiator Alias: ub1410

12) Stop and start the iscsi service on the server.
> sudo service open-iscsi stop
Pause for 10 seconds and then run the next command.
> sudo service open-iscsi start

13) From the server, check your session.
> sudo iscsiadm -m session -P 1

14) From the server, check the NetApp iSCSI details. Note that mode, targetname and portal have two “- -” dashes in front of them.
> sudo iscsiadm –mode node –targetname “iqn.1992-08.com.netapp:sn.4055372815” –portal 10.10.10.141:3260

15) From the server, find and format the new lun (new disk). The command below will find the device. In this case its sdb.
> dmesg | grep “unknown partition table”
[ 1930.949065]  sdb: unknown partition table
[ 2167.186068]  sdb: unknown partition table

> sudo fdisk /dev/sdb  (note: commands are in bold red below)

Welcome to fdisk (util-linux 2.25.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0xe4775fd7.

Command (m for help): w

The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.

> sudo fdisk /dev/sdb

Command (m for help): n
Partition type
p   primary (0 primary, 0 extended, 4 free)
e   extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-10485759, default 2048): press enter
Last sector, +sectors or +size{K,M,G,T,P} (2048-10485759, default 10485759): press enter

Created a new partition 1 of type ‘Linux’ and of size 5 GiB.

Command (m for help): p
Disk /dev/sdb: 5 GiB, 5368709120 bytes, 10485760 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 65536 bytes
Disklabel type: dos
Disk identifier: 0xc68508a4

Device     Boot Start      End  Sectors Size Id Type
/dev/sdb1        2048 10485759 10483712   5G 83 Linux

Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.

16) On the server, create the Linux file system on the new partition.
> sudo mkfs -t ext4 /dev/sdb1
mke2fs 1.42.10 (18-May-2014)
Discarding device blocks: done
Creating filesystem with 1310464 4k blocks and 327680 inodes
Filesystem UUID: d125b8ff-a690-4cbb-925d-645764d41172
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

17) Verify the partition.
> sudo blkid /dev/sdb1
/dev/sdb1: UUID=”d125b8ff-a690-4cbb-925d-645764d41172″ TYPE=”ext4″ PARTUUID=”c68508a4-01″

18) Create the mount point and manually mount the directory.
> sudo mkdir /newiscsilun
> sudo mount /dev/sdb1 /newiscsilun
> df -h | grep newiscsilun
/dev/sdb1  4.8G   10M  4.6G   1% /newiscsilun

19) Add the new mount point to /etc/fstab.
> sudo vi /etc/fstab
/dev/sdb1 /newiscsilun ext4 _netdev 0 0
> wq!

Note: the _netdev option is important so that it doesn’t try mounting the target before the network is available.

20) Test that it survives a reboot by rebooting the server now. With the _netdev set, iscsi starts and your CHAP logins should take place before it attempts to mount. After the reboot, login and verify that it is mounted.

> df -h | grep newiscsilun
/dev/sdb1  4.8G   10M  4.6G   1% /newiscsilun

21) On the server you can check session stats.
> sudo iscsiadm -m session -s

22) As root, change permissions on /etc/iscsi/iscsid.conf. I’m not sure why they haven’t fixed this clear text CHAP password in a file issue so just make sure only root can read/write the file.
> sudo chmod 600 /etc/iscsi/iscsid.conf

23) On the NetApp storage you can verify the lun and the server’s session.
netapp> lun show -v /vol/MCHAPVOL/UB1410_iSCSI_MCHAP_01
/vol/MCHAPVOL/UB1410_iSCSI_MCHAP_01      5g (5368709120)    (r/w, online, mapped)
Serial#: BQVJ3]DxwBcB
Share: none
Space Reservation: enabled
Multiprotocol Type: linux
Maps: ISCSI_MCHAP_UB1410=1
Occupied Size:  132.8m (139202560)
Creation Time: Mon Dec 29 13:33:18 EST 2014
Cluster Shared Volume Information: 0x0

You can also get stats with the command below.
netapp>  iscsi session show -v

HOWTO Secure iSCSI Luns Between Oracle Enterprise Linux 7 and NetApp Storage with Mutual CHAP

01 Monday Sep 2014

Posted by in iSCSI, Linux, Oracle, Security

≈ Leave a comment

Tags

, , ,

This post demonstrates how to enable Bidirectional or Mutual CHAP on iSCSI luns between Oracle Enterprise Linux 7 and NetApp storage. The aggregate, lun and disk sizes are small in this HOWTO to keep it simple.

1) If not already installed, install the iSCSI initiator on your server.
> yum install iscsi-initiator*

2) Display your server’s new iSCSI initiator or iqn nodename.
> cat /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.1988-12.com.oracle:77ff4f784c55

3) On the NetApp filer, create the volume that will hold the iscsi luns. This command assumes you have aggregate aggr1 already created.  If not, use an aggregate that has enough room for your volume.
netapp> vol create MCHAPVOL aggr1 10g

4) Create the lun in the volume.
netapp> lun create -s 5g -t linux /vol/MCHAPVOL/OEL7_iSCSI_MCHAP_01

5) Create an igroup and add the Linux iscsi nodename or iqn from step 2 above to the new igroup.
netapp> igroup create -i -t linux ISCSI_MCHAP_OEL7
netapp> igroup add ISCSI_MCHAP_OEL7 iqn.1988-12.com.oracle:77ff4f784c55
netapp> igroup set ISCSI_MCHAP_OEL7 report_scsi_name yes
netapp> igroup show ISCSI_MCHAP_OEL7

ISCSI_MCHAP_OEL7 (iSCSI) (ostype: linux):
iqn.1988-12.com.oracle:77ff4f784c55 (not logged in)

6) Map the lun to the igroup and give it lun ID 01.
netapp> lun map /vol/MCHAPVOL/OEL7_iSCSI_MCHAP_01 ISCSI_MCHAP_OEL7 01

7) Obtain the NetApp target nodename.
netapp> iscsi nodename
iSCSI target nodename: iqn.1992-08.com.netapp:sn.4055372815

8) Set the CHAP secret on the NetApp controller.
netapp> iscsi security add -i iqn.1988-12.com.oracle:77ff4f784c55 -s chap -p OEL7 -n iqn.1988-12.com.oracle:77ff4f784c55 -o NETAPPMCHAP -m iqn.1992-08.com.netapp:sn.4055372815

netapp> iscsi security show
Default sec is None
init: iqn.1986-03.com.sun:01:e00000000000.52bcad1c auth: CHAP Local Inbound password: **** Inbound username: iqn.1986-03.com.sun:01:e000000000bound password: **** Outbound username: iqn.1992-08.com.netapp:sn.4055372815
init: iqn.1988-12.com.oracle:77ff4f784c55 auth: CHAP Local Inbound password: **** Inbound username: iqn.1988-12.com.oracle:77ff4f784c55 Outbou** Outbound username: iqn.1992-08.com.netapp:sn.4055372815

9) On the server, edit your /etc/iscsi/iscsi.conf file and set the parameters below.
> vi /etc/iscsi/iscsid.conf
node.startup = automatic
node.session.auth.authmethod = CHAP
node.session.auth.username = iqn.1988-12.com.oracle:77ff4f784c55
node.session.auth.password = OEL7
node.session.auth.username_in = iqn.1992-08.com.netapp:sn.4055372815
node.session.auth.password_in = NETAPPMCHAP
discovery.sendtargets.auth.authmethod = CHAP
discovery.sendtargets.auth.username = iqn.1988-12.com.oracle:77ff4f784c55
discovery.sendtargets.auth.password = OEL7
discovery.sendtargets.auth.username_in = iqn.1992-08.com.netapp:sn.4055372815
discovery.sendtargets.auth.password_in = NETAPPMCHAP
> wq!

10) On the server, restart the service and discover your iSCSI target (your storage system).
> service iscsi restart
Redirecting to /bin/systemctl restart  iscsi.service

a) Verify the target.
> iscsiadm -m discovery -t st -p 10.10.10.141
10.10.10.141:3260,1000 iqn.1992-08.com.netapp:sn.4055372815

> iscsiadm -m node  (this should display the same as above)
10.10.10.141:3260,1000 iqn.1992-08.com.netapp:sn.4055372815

11) On the server, manually login to the iSCSI target (your storage array). Note there are two dashes “- -” in front of targetname and login.
> iscsiadm -m node –targetname “iqn.1992-08.com.netapp:sn.4055372815” –login
Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.4055372815, portal: 10.10.10.141,3260] (multiple)
Login to [iface: default, target: iqn.1992-08.com.netapp:sn.4055372815, portal: 10.10.10.141,3260] successful.

a) On the NetApp storage console you should see the iSCSI session:
[netapp:iscsi.notice:notice]: ISCSI: New session from initiator iqn.1988-12.com.oracle:77ff4f784c55 at IP addr 10.10.10.201

b) Verify the iSCSI session on the filer:
netapp> iscsi session show
Session 4
Initiator Information
Initiator Name: iqn.1988-12.com.oracle:77ff4f784c55
ISID: 00:02:3d:06:00:00
Initiator Alias: localhost.localdomain

12) From the server , check your session.
> iscsiadm -m session -P 1
Target: iqn.1992-08.com.netapp:sn.4055372815 (non-flash)
Current Portal: 10.10.10.141:3260,1000
Persistent Portal: 10.10.10.141:3260,1000
**********
Interface:
**********
Iface Name: default
Iface Transport: tcp
Iface Initiatorname: iqn.1988-12.com.oracle:77ff4f784c55
Iface IPaddress: 10.10.10.201
Iface HWaddress: <empty>
Iface Netdev: <empty>
SID: 6
iSCSI Connection State: LOGGED IN
iSCSI Session State: LOGGED_IN
Internal iscsid Session State: NO CHANGE

13) From the server, check the NetApp iSCSI details. Note there are two dashes “- -” in front of mode, targetname and portal.
> iscsiadm –mode node –targetname “iqn.1992-08.com.netapp:sn.4055372815″ –portal 10.10.10.141:3260
# BEGIN RECORD 6.2.0.873-21
node.name = iqn.1992-08.com.netapp:sn.4055372815
node.tpgt = 1000
node.startup = automatic
node.leading_login = No
iface.hwaddress = <empty>
iface.ipaddress = <empty>
iface.iscsi_ifacename = default
iface.net_ifacename = <empty>
iface.transport_name = tcp
iface.initiatorname = <empty>
iface.state = <empty>
iface.vlan_id = 0
iface.vlan_priority = 0
iface.vlan_state = <empty>
iface.iface_num = 0
iface.mtu = 0
iface.port = 0
iface.bootproto = <empty>
iface.subnet_mask = <empty>
iface.gateway = <empty>
iface.dhcp_alt_client_id_state = <empty>
iface.dhcp_alt_client_id = <empty>
iface.dhcp_dns = <empty>
iface.dhcp_learn_iqn = <empty>
iface.dhcp_req_vendor_id_state = <empty>
iface.dhcp_vendor_id_state = <empty>
iface.dhcp_vendor_id = <empty>
iface.dhcp_slp_da = <empty>
iface.fragmentation = <empty>
iface.gratuitous_arp = <empty>
iface.incoming_forwarding = <empty>
iface.tos_state = <empty>
iface.tos = 0
iface.ttl = 0
iface.delayed_ack = <empty>
iface.tcp_nagle = <empty>
iface.tcp_wsf_state = <empty>
iface.tcp_wsf = 0
iface.tcp_timer_scale = 0
iface.tcp_timestamp = <empty>
iface.redirect = <empty>
iface.def_task_mgmt_timeout = 0
iface.header_digest = <empty>
iface.data_digest = <empty>
iface.immediate_data = <empty>
iface.initial_r2t = <empty>
iface.data_seq_inorder = <empty>
iface.data_pdu_inorder = <empty>
iface.erl = 0
iface.max_receive_data_len = 0
iface.first_burst_len = 0
iface.max_outstanding_r2t = 0
iface.max_burst_len = 0
iface.chap_auth = <empty>
iface.bidi_chap = <empty>
iface.strict_login_compliance = <empty>
iface.discovery_auth = <empty>
iface.discovery_logout = <empty>
node.discovery_address = 10.10.10.141
node.discovery_port = 3260
node.discovery_type = send_targets
node.session.initial_cmdsn = 0
node.session.initial_login_retry_max = 8
node.session.xmit_thread_priority = -20
node.session.cmds_max = 128
node.session.queue_depth = 32
node.session.nr_sessions = 1
node.session.auth.authmethod = CHAP
node.session.auth.username = iqn.1988-12.com.oracle:77ff4f784c55
node.session.auth.password = ********
node.session.auth.username_in = iqn.1992-08.com.netapp:sn.4055372815
node.session.auth.password_in = ********
node.session.timeo.replacement_timeout = 120
node.session.err_timeo.abort_timeout = 15
node.session.err_timeo.lu_reset_timeout = 30
node.session.err_timeo.tgt_reset_timeout = 30
node.session.err_timeo.host_reset_timeout = 60
node.session.iscsi.FastAbort = Yes
node.session.iscsi.InitialR2T = No
node.session.iscsi.ImmediateData = Yes
node.session.iscsi.FirstBurstLength = 262144
node.session.iscsi.MaxBurstLength = 16776192
node.session.iscsi.DefaultTime2Retain = 0
node.session.iscsi.DefaultTime2Wait = 2
node.session.iscsi.MaxConnections = 1
node.session.iscsi.MaxOutstandingR2T = 1
node.session.iscsi.ERL = 0
node.conn[0].address = 10.10.10.141
node.conn[0].port = 3260
node.conn[0].startup = manual
node.conn[0].tcp.window_size = 524288
node.conn[0].tcp.type_of_service = 0
node.conn[0].timeo.logout_timeout = 15
node.conn[0].timeo.login_timeout = 15
node.conn[0].timeo.auth_timeout = 45
node.conn[0].timeo.noop_out_interval = 5
node.conn[0].timeo.noop_out_timeout = 5
node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
node.conn[0].iscsi.HeaderDigest = None
node.conn[0].iscsi.IFMarker = No
node.conn[0].iscsi.OFMarker = No
# END RECORD

14) From the server, find and format the new lun (new disk). Your fdisk commands are in bold red below.
> cat /var/log/messages | grep “unknown partition table”
localhost kernel: sdb: unknown partition table

> fdisk /dev/sdb

Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x195fbc72.

The device presents a logical sector size that is smaller than
the physical sector size. Aligning to a physical sector (or optimal
I/O) size boundary is recommended, or performance may be impacted.

Command (m for help): w

> fdisk /dev/sdb
Command (m for help): n
Partition type:
p   primary (0 primary, 0 extended, 4 free)
e   extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-10485759, default 2048): <press enter>
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-10485759, default 10485759): <press enter>
Using default value 10485759
Partition 1 of type Linux and of size 5 GiB is set

Command (m for help): p
Disk /dev/sdb: 5368 MB, 5368709120 bytes, 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 65536 bytes
Disk label type: dos
Disk identifier: 0xa1c2729d

Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048    10485759     5241856   83  Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

15) On the server, create the Linux file system on the new partition.
> mkfs -t ext4 /dev/sdb1
mke2fs 1.42.9 (28-Dec-2013)
Discarding device blocks: done
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=16 blocks
327680 inodes, 1310464 blocks
65523 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1342177280
40 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

16) Verify the partition.
> blkid /dev/sdb1
/dev/sdb1: UUID=”eb7fa074-50d8-47d1-83aa-7b69568020e4″ TYPE=”ext4″

17) Create the mount point and manually mount the directory.
> mkdir /newiscsilun
> mount /dev/sdb1 /newiscsilun
> df -h | grep newiscsilun
/dev/sdb1  4.8G   20M  4.6G   1% /newiscsilun

18) Add the new mount point to /etc/fstab.
> vi /etc/fstab
/dev/sdb1 /newiscsilun ext4 _netdev 0 0
> wq!

Note: the _netdev option is important so that it doesn’t try mounting the target before the network is available.

19) Test that it survives a reboot by rebooting the server. With the _netdev set, iscsi starts and your CHAP logins should take place before it attempts to mount. After the reboot, login and verify its mounted.

> df -h | grep newiscsilun
/dev/sdb1  4.8G   20M  4.6G   1% /newiscsilun

20) On the server you can check session stats.
> iscsiadm -m session -s
Stats for session [sid: 6, target: iqn.1992-08.com.netapp:sn.4055372815, portal: 10.10.10.141,3260]
iSCSI SNMP:
txdata_octets: 137976652
rxdata_octets: 3841684
noptx_pdus: 0
scsicmd_pdus: 1127
tmfcmd_pdus: 0
login_pdus: 0
text_pdus: 0
dataout_pdus: 1827
logout_pdus: 0
snack_pdus: 0
noprx_pdus: 0
scsirsp_pdus: 1127
tmfrsp_pdus: 0
textrsp_pdus: 0
datain_pdus: 793
logoutrsp_pdus: 0
r2t_pdus: 1827
async_pdus: 0
rjt_pdus: 0
digest_err: 0
timeout_err: 0
iSCSI Extended:
tx_sendpage_failures: 0
rx_discontiguous_hdr: 0
eh_abort_cnt: 0

21) As root, change permissions on /etc/iscsi/iscsid.conf. I’m not sure why they haven’t fixed this clear text CHAP password in a file issue so just make sure only root can read/write the file.
> chmod 600 /etc/iscsi/iscsid.conf

22) On the NetApp storage you can verify the Lun and the server’s session.
netapp>  lun show -v /vol/MCHAPVOL/OEL7_iSCSI_MCHAP_01
/vol/MCHAPVOL/OEL7_iSCSI_MCHAP_01      5g (5368709120)    (r/w, online, mapped)
Serial#: BQVJ3]DxwBc-
Share: none
Space Reservation: enabled
Multiprotocol Type: linux
Maps: ISCSI_MCHAP_OEL7=1
Occupied Size:  134.0m (140546048)
Creation Time: Sat Aug 30 12:14:47 EST 2014
Cluster Shared Volume Information: 0x0

netapp> iscsi session show -v
Session 6
Initiator Information
Initiator Name: iqn.1988-12.com.oracle:77ff4f784c55
ISID: 00:02:3d:01:00:00
Initiator Alias: localhost.localdomain

Session Parameters
SessionType=Normal
TargetPortalGroupTag=1000
MaxConnections=1
ErrorRecoveryLevel=0
AuthMethod=CHAP
HeaderDigest=None
DataDigest=None
ImmediateData=Yes
InitialR2T=No
FirstBurstLength=65536
MaxBurstLength=65536
Initiator MaxRecvDataSegmentLength=65536
Target MaxRecvDataSegmentLength=65536
DefaultTime2Wait=2
DefaultTime2Retain=0
MaxOutstandingR2T=1
DataPDUInOrder=Yes
DataSequenceInOrder=Yes
Command Window Size: 64

Connection Information
Connection 0
Remote Endpoint: 10.10.10.201:41613
Local Endpoint: 10.10.10.141:3260
Local Interface: e0a
TCP recv window size: 131400

Command Information
No commands active

No commands active