• List of iSCSI Mutual CHAP Posts by OS
  • Tools and Utilities for Windows
  • Unix and Linux Distros

slice2

slice2

Category Archives: Splunk

HOWTO Install the Splunk Universal Forwarder on FreeBSD

24 Sunday Apr 2022

Posted by Slice2 in Splunk

≈ Leave a comment

Tags

FreeBSD

The following steps were tested on FreeBSD x64 11.4, 12.3, 13.0 and 13.1.

1) There are a few required adjustments before installing the forwarder on FreeBSD. This ensures the forwarder functions properly.

a) As root, add the following to loader.conf.
> vi /boot/loader.conf

kern.maxdsiz=”2147483648″ # 2GB
kern.dfldsiz=”2147483648″ # 2GB
machdep.hlt_cpus=0

b) Add the following to sysctl.conf.
> vi /etc/sysctl.conf

vm.max_proc_mmap=2147483647

c) Restart for the changes to take effect.
> reboot

2) If you have a Forwarder installed, remove it. If you don’t, skip to step 3.

a) Check if running and if so stop it.
> /opt/splunkforwarder/bin/splunk status
> /opt/splunkforwarder/bin/splunk version
> /opt/splunkforwarder/bin/splunk stop

b) Get the package name and delete the older release. And yes, that is a \ before the * as you have to escape it. In this example, splunkforwarder-8.1.6 is being removed.
> pkg info -g splunkforwarder\*
> pkg delete -y splunkforwarder-8.1.6
> rm /etc/rc.d/splunk
> rm -R /opt/splunkforwarder

3) If a new install, check to see if /opt exists. If not, create it.
> ls -l /opt

If not there, create it.
> mkdir /opt

3) Move the Forwarder package to /tmp on your FreeBSD host and install it.
> cd /tmp
> pkg install splunkforwarder-8.2.3-cd0848707637-freebsd-11.3-amd64.txz

Updating FreeBSD repository catalogue…
FreeBSD repository is up to date.
All repositories are up to date.
Checking integrity… done (0 conflicting)
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
splunkforwarder: 8.2.3

Number of packages to be installed: 1

The process will require 64 MiB more space.

Proceed with this action? [y/N]: y
[1/1] Installing splunkforwarder-8.2.3…
Extracting splunkforwarder-8.2.3: 100%
complete

a) Start the Forwarder and accept the license.
> /opt/splunkforwarder/bin/splunk start –accept-license –answer-yes

This appears to be your first time running this version of Splunk.

Splunk software must create an administrator account during startup. Otherwise, you cannot log in.
Create credentials for the administrator account.
Characters do not appear on the screen when you type in credentials.

Please enter an administrator username: splunkadmin
Password must contain at least:
* 8 total printable ASCII character(s).
Please enter a new password:
Please confirm new password:

b) Enable the Forwarder to start when the system is booted.
> /opt/splunkforwarder/bin/splunk enable boot-start

Init script installed at /etc/rc.d/splunk.
Init script is configured to run at boot.

4) If you don’t use a Deployment server, skip this step. If you use a Deployment server, set the deployment server configuration. Typically this comes from your Splunk team so this is an example only.
> vi /opt/splunkforwarder/etc/system/local/deploymentclient.conf

[deployment-client]

[target-broker:deploymentServer]
targetUri=192.168.1.100:8089

5) Restart the Forwarder and verify the configuration.
> /opt/splunkforwarder/bin/splunk restart
> /opt/splunkforwarder/bin/splunk status

a) Verify that its set in rc to start at boot and that the process is running.
> service -e | grep splunk
/etc/rc.d/splunk

> ps -x | grep splunkd
1494 – S 0:02.54 splunkd -p 8089 restart
1495 – Is 0:00.04 splunkd: [splunkd pid=1494] splunkd -p 8089 restart [process-runner] (splunkd)

> cat /etc/rc.conf | grep splunk
splunk_enable=”YES”

6) Installation is complete.

HOWTO install a Splunk Universal Forwarder on Solaris 11 SPARC and x64 Using pkg(p5p) and tar

05 Saturday Mar 2022

Posted by Slice2 in Solaris, Splunk

≈ 2 Comments

Tags

Solaris

I recently had to get the Forwarders installed and there are no detailed steps in the Splunk docs for Solaris 11. If you need Solaris 10 steps, see my post here.

1) There are two installation options and platforms supported by Splunk; using pkg and tar on SPARC and x64 CPUs. The platform/CPU type is at the end of the filename shown below. The steps below cover both types of installation scenarios. Choose the steps for the way in which you want to install and the platform you have. There is a Universal Forwarder for SPARC and x64 (Intel/AMD) CPUs. Be sure to su – root before you start, or use sudo in front of the commands.

a) Native pkg (as in .p5p) formatted binary – easy to manage and upgrade, software included in inventory and vuln scans. Installed using pkg:
splunkforwarder-8.2.3-cd0848707637-solaris-intel.p5p
splunkforwarder-8.2.3-cd0848707637-solaris-sparc.p5p

b) Extracted via tar – software not seen by inventory and vuln scans, only the running splunkd process identifies it in scans (but not the version running). Installed using tar:
splunkforwarder-8.2.3-cd0848707637-SunOS-x86_64.tar.Z
splunkforwarder-8.2.3-cd0848707637-SunOS-sparc.tar.Z

2) Verify that you have the required libc installed.
> pvs /usr/lib/libc.so.1 | grep SUNW_1.22.7
> pvs /usr/lib/libc.so.1 (to get a list of all libc versions)

3) Backup your files. If you have any locally customized .conf files in the following folders, save a copy now. Note that some older forwarders
may not use upper case letters in the path.

/opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/local/
/opt/splunkforwarder/etc/system/local/

4) If an existing Forwarder is running, identify your Splunk index. You will need this value along with your hostname, IP address and platform later.
> cat /opt/splunkforwarder/etc/system/local/inputs.conf | grep index
> hostname
> ifconfig -a | grep inet (or just ifconfig -a if you have multiple NICs plumbed)
> uname -a

5) Check for existing version and state.
If the existing Forwarder was installed with pkg:
> pkg list | splunk
> pkg info -r splunkforwarder | egrep -i “Summary|Version”
> /opt/splunkforwarder/bin/splunk status
> ps aevx | grep splunkd

If the existing Forwarder was installed with tar:
> /opt/splunkforwarder/bin/splunk version
> /opt/splunkforwarder/bin/splunk status
> ps aevx | grep splunkd

6) Stop the currently installed Universal Forwarder and then remove it.
> /opt/splunkforwarder/bin/splunk stop

If installed via pkg:
> pkg uninstall splunkforwarder
> rm /etc/init.d/splunk

If installed via tar:
> cd /opt/
> rm -r splunkforwarder
> rm /etc/init.d/splunk

7) Make sure the Splunk sockets are no longer in use or locked. Copy/paste each line below one at a time and press enter.
> PORT=8089; for PID in /proc/*; do pfiles ${PID} |grep “port: ${PORT}” && echo ${PID}; done
> PORT=9997; for PID in /proc/*; do pfiles ${PID} |grep “port: ${PORT}” && echo ${PID}; done

a) If the commands above return anything (and they shouldn’t), you need to kill the process.
> pfiles process_id (from the output of the commands above to get process details)
> pkill process_id

8) Move the package to your host. Based on your CPU type (SPARC or x64) and the installation method you chose (pkg or tar), copy the Universal Forwarder to your server using scp, Filezilla, WinSCP or your preferred file transfer method.

If installing with tar:
> cp splunkforwarder-8.2.3-cd0848707637-SunOS-x86_64.tar.Z /opt/
> cd /opt
> uncompress splunkforwarder-8.2.3-cd0848707637-SunOS-x86_64.tar.Z
> tar xvf splunkforwarder-8.2.3-cd0848707637-SunOS-x86_64.tar
> chown -R root:root splunkforwarder

If installing with pkg:
> cp splunkforwarder-8.2.3-cd0848707637-solaris-intel.p5p /opt/
> cd /opt/
> pkg list -g file:////opt/splunkforwarder-8.2.3-cd0848707637-solaris-intel.p5p
> pkgrepo info -s /opt/splunkforwarder-8.2.3-cd0848707637-solaris-intel.p5p
> pkg set-publisher -p /opt/splunkforwarder-8.2.3-cd0848707637-solaris-intel.p5p splunk
> pkg install -nv splunkforwarder-8.2.3-cd0848707637-solaris-intel.p5p
> pkg install -g ./splunkforwarder-8.2.3-cd0848707637-solaris-intel.p5p splunkforwarder

a) For both pkgadd and tar, start and accept the license.
> /opt/splunkforwarder/bin/splunk start –accept-license –answer-yes

This appears to be your first time running this version of Splunk.

Splunk software must create an administrator account during startup. Otherwise, you cannot log in.
Create credentials for the administrator account.
Characters do not appear on the screen when you type in credentials.

Please enter an administrator username: splunkadm  (<– you can use any account name you want here, its for splunkd, not an OS account)
Password must contain at least:
* 8 total printable ASCII character(s).
Please enter a new password:
Please confirm new password:

b) For both pkgadd and tar, enable auto-start when the server is rebooted.
> /opt/splunkforwarder/bin/splunk enable boot-start

c) For both pkgadd and tar, copy the deploymentclient.conf file (provided by the Splunk team) to the path listed below. Note that if a Deployment server is not being used, skip this step.
> cp deploymentclient.conf /opt/splunkforwarder/etc/system/local/
> /opt/splunkforwarder/etc/system/local/
> chown root:root deploymentclient.conf

8) Restart the Forwarder, verify that its running and contact your Splunk team via email. Send them your index, hostname, ip and platform so they can associate your new Forwarder with the proper inputs.
> /opt/splunkforwarder/bin/splunk restart
> /opt/splunkforwarder/bin/splunk status
> ps aevx | grep splunkd

9) Connectivity issues? See if the Solaris firewall is on and has any rules for Splunk ports 8089 and 9997. Adjust rules as needed.
> svcs -x firewall:default
> pfbash pfctl -s rules

HOWTO install a Splunk Universal Forwarder on Solaris 10 SPARC and x64 Using pkgadd and tar

13 Sunday Feb 2022

Posted by Slice2 in Solaris, Splunk

≈ 1 Comment

OK, OK, I know. Solaris 10 is old and dusty. Extended support has been pushed out to January 2024 so there are still plenty of systems in use out there. I recently had to get the Forwarders installed and there are no detailed steps in the Splunk docs. This covers installing via pkgadd and tar. Also, the last available Forwarder I could find on their site that supports Solaris 10 is v7.3.9.

1) There are two installation options and platforms supported by Splunk; using pkgadd and tar on SPARC and x64 CPUs. The platform/CPU type is at the end of the filename shown below. The steps below cover both types of installation scenarios. Choose the steps for the way in which you want to install and the platform you have. There is a Universal Forwarder for SPARC and x64 (Intel/AMD) CPUs so simply insert the Forwarder filename you need in the steps listed below. This HOWTO was done using Solaris 10 x64 so if you have a SPARC host, use the sparc Forwarder filename.

a) Native pkg formatted binary – easy to manage and upgrade, software included in inventory and vuln scans. Installed using pkgadd:
splunkforwarder-7.3.9-39a78bf1bc5b-solaris-10-intel.pkg.Z
splunkforwarder-7.3.9-39a78bf1bc5b-solaris-10-sparc.pkg.Z

b) Extracted via tar – software not seen by inventory and vuln scans, only the running splunkd process identifies it in scans (but not the version running). Installed using tar:
splunkforwarder-7.3.9-39a78bf1bc5b-SunOS-x86_64.tar.Z
splunkforwarder-7.3.9-39a78bf1bc5b-SunOS-sparc.tar.Z

2) Verify that you have the required libc installed.
> pvs /usr/lib/libc.so.1 | grep SUNW_1.22.7
> pvs /usr/lib/libc.so.1 (to get a list of all libc versions)

a) Solaris 10 hosts must be updated to libc SUNW_1.22.7 or later. See the Oracle Support page that lists patches that include libc changes (Oracle Support required): https://support.oracle.com/rs?type=doc&id=2069855.1

SUNW_1.22.7
SPARC: Patch 144500-19
X64: Patch 144501-19

3) Backup your files. If you have any locally customized .conf files in the following folders, save a copy now. Note that some older forwarders
may not use upper case letters in the path.

/opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/local/
/opt/splunkforwarder/etc/system/local/

4) Identify your Splunk index. You will need this value along with your hostname, IP address and platform later.
> cat /opt/splunkforwarder/etc/system/local/inputs.conf | grep index
> hostname
> ifconfig -a | grep inet (or just ifconfig -a if you have multiple NICs plumbed)
> uname -a

5) Check for existing version and state.
If the existing Forwarder was installed with pkgadd:
> pkginfo -l | grep splunk
> pkginfo -l splunkforwarder* | grep VERSION
> /opt/splunkforwarder/bin/splunk status
> ps -ef | grep splunkd

If the existing Forwarder was installed with tar:
> /opt/splunkforwarder/bin/splunk version
> /opt/splunkforwarder/bin/splunk status
> ps -ef | grep splunkd

6) Stop the currently installed Universal Forwarder and then remove it.
> /opt/splunkforwarder/bin/splunk stop

If installed via pkgadd:
> pkgrm splunkforwarder
> rm /etc/init.d/splunk

If installed via tar:
> cd /opt/
> rm -r splunkforwarder
> rm /etc/init.d/splunk

7) Make sure the Splunk sockets are no longer in use or locked. Copy/paste each line below one at a time and press enter.
> PORT=8089; for PID in /proc/*; do pfiles ${PID} |grep “port: ${PORT}” && echo ${PID}; done
> PORT=9997; for PID in /proc/*; do pfiles ${PID} |grep “port: ${PORT}” && echo ${PID}; done

a) If the commands above return anything (and they shouldn’t), you need to kill the process.
> pfiles process_id (from the output of the commands above to get process details)
> pkill process_id

8) Move the package to your host. Based on your CPU type (SPARC or x64) and the installation method you chose (pkgadd or tar), copy the Universal Forwarder to your server using scp, Filezilla, WinSCP or your preferred file transfer method.

If installing with tar:
> cp splunkforwarder-7.3.9-39a78bf1bc5b-SunOS-x86_64.tar.Z /opt/
> cd /opt
> uncompress splunkforwarder-7.3.9-39a78bf1bc5b-SunOS-x86_64.tar.Z
> tar xvf splunkforwarder-7.3.9-39a78bf1bc5b-SunOS-x86_64.tar
> chown -R root:root splunkforwarder

If installing with pkgadd:

> uncompress splunkforwarder-7.3.9-39a78bf1bc5b-solaris-10-intel.pkg.Z

> pkgadd -d splunkforwarder-7.3.9-39a78bf1bc5b-solaris-10-intel.pkg all

Do you want to continue with the installation of <splunkforwarder> [y,n,?] y

a) For both pkgadd and tar, start and accept the license.
> /opt/splunkforwarder/bin/splunk start –accept-license –answer-yes

This appears to be your first time running this version of Splunk.

Splunk software must create an administrator account during startup. Otherwise, you cannot log in.
Create credentials for the administrator account.
Characters do not appear on the screen when you type in credentials.

Please enter an administrator username: splunkadm  (<– you can use any account name you want here, its for splunkd, not an OS account)
Password must contain at least:
* 8 total printable ASCII character(s).
Please enter a new password:
Please confirm new password:

b) For both pkgadd and tar, enable auto-start when the server is rebooted.
> /opt/splunkforwarder/bin/splunk enable boot-start

c) For both pkgadd and tar, copy the deploymentclient.conf file to the path listed below. Note that if a Deployment server is not being used, skip this step.
> cp deploymentclient.conf /opt/splunkforwarder/etc/system/local/
> /opt/splunkforwarder/etc/system/local/
> chown root:root deploymentclient.conf

8) Restart the Forwarder, verify that its running and contact your Splunk team via email. Send them your index, hostname, ip and platform so they can associate your new Forwarder with the proper inputs.
> /opt/splunkforwarder/bin/splunk restart
> /opt/splunkforwarder/bin/splunk status
> ps -ef | grep splunkd

9) Connectivity issues? See if the Solaris firewall is on and has any rules for Splunk ports 8089 and 9997. Adjust rules as needed.
> svcs | egrep ‘(pfil|ipfilter)’
> ipfstat -io

Sysmon Event ID 1 Process Creation rules for Splunk Universal Forwarder and McAfee All Access

20 Saturday Nov 2021

Posted by Slice2 in Security, Splunk

≈ Leave a comment

Tags

Security, Splunk, Sysmon

When using Sysmon to monitor processes in Splunk, you can exclude the chattiest processes that eat away at your license and indexing I/O. Splunk recommends excluding its own Universal Forwarder as well as antivirus products.

1) Splunk’s Sysmon Add-on docs list a few options for the sysmon config file. I used the SwiftOnSecurity template and simply added the UF and McAfee executables.

Splunk Add-on for Sysmon Doc:
https://docs.splunk.com/Documentation/AddOns/released/MSSysmon/ConfigureSysmon

Ready to run Sysmon config file:
https://github.com/SwiftOnSecurity/sysmon-config

Sysmon:
https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon

2) Open your Sysmon xml config file with Notepad++ or VSCode. Use Notepad++ or VSCode when editing xml on Windows. They recognize UNIX newline format and XML syntax highlighting so your file will be easier to read. Don’t use Notepad.exe. Under the heading <!–SYSMON EVENT ID 1 : PROCESS CREATION [ProcessCreate]–> enter the text below and save the file.

3) When done editing, reload the active config.

> sysmon -c c:\path\to\your_sysmon_config_file_v2.xml

<!–SECTION: Splunk:Universal Forwarder–>
<Image condition=”is”>C:\Program Files\SplunkUniversalForwarder\bin\splunkd.exe</Image> <!–Splunk:Universal Forwarder: binary that accesses, processes, and indexes streaming data–>
<Image condition=”is”>C:\Program Files\SplunkUniversalForwarder\bin\btool.exe</Image> <!–Splunk:Universal Forwarder: cli util used to troubleshoot config file issues–>
<Image condition=”is”>C:\Program Files\SplunkUniversalForwarder\bin\splunk.exe</Image> <!–Splunk:Universal Forwarder: the control application for the Windows version of Splunk–>
<Image condition=”is”>C:\Program Files\SplunkUniversalForwarder\bin\splunk-admon.exe</Image> <!–Splunk:Universal Forwarder: runs whenever you configure an Active Directory monitoring input–>
<Image condition=”is”>C:\Program Files\SplunkUniversalForwarder\bin\splunk-perfmon.exe</Image> <!–Splunk:Universal Forwarder: runs when you monitor performance data–>
<Image condition=”is”>C:\Program Files\SplunkUniversalForwarder\bin\splunk-netmon.exe</Image> <!–Splunk:Universal Forwarder: runs when you configure Splunk to monitor network information–>
<Image condition=”is”>C:\Program Files\SplunkUniversalForwarder\bin\splunk-regmon.exe</Image> <!–Splunk:Universal Forwarder: runs when you configure a Registry monitoring input–>
<Image condition=”is”>C:\Program Files\SplunkUniversalForwarder\bin\splunk-winevtlog.exe</Image> <!–Splunk:Universal Forwarder: event log collections, outputs events as they are collected–>
<Image condition=”is”>C:\Program Files\SplunkUniversalForwarder\bin\splunk-winhostmon</Image> <!–Splunk:Universal Forwarder: runs when you configure a Windows host monitoring input–>
<Image condition=”is”>C:\Program Files\SplunkUniversalForwarder\bin\splunk-winprintmon.exe</Image> <!–Splunk:Universal Forwarder: runs when you configure a Windows print monitoring input –>
<Image condition=”is”>C:\Program Files\SplunkUniversalForwarder\bin\splunk-wmi.exe</Image> <!–Splunk:Universal Forwarder: runs when you configure a performance monitor–>
<Image condition=”is”>C:\Program Files\SplunkUniversalForwarder\bin\splunk-powershell.exe</Image> <!–Splunk:Universal Forwarder: runs when you configure a PowerShell command or script monitor–>
<Image condition=”is”>C:\Program Files\SplunkUniversalForwarder\bin\splunk-MonitorNoHandle.exe</Image> <!–Splunk:Universal Forwarder: this input monitors files without using Windows file handles.–>
<!–SECTION: McAfee:AntiVirus–>
<Image condition=”is”>C:\Program Files\Common Files\McAfee\Platform\Core\mchost.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\Common Files\McAfee\Platform\CommonBuild\McCBEntAndInstru.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\Common Files\McAfee\Platform\McPltCmd.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\McAfee\MQS\QcShm.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\McAfee.com\Agent\mcupdate.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\Common Files\McAfee\MSGSDK\msgrunner.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\Common Files\McAfee\TaskScheduler\McAMTaskAgent.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\Common Files\McAfee\UPDMGR\8.1.149.1\mcupdatemgr.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\Common Files\McAfee\CSP\4.1.106.0\McCSPServiceHost.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\McAfee\MSC\mcinfo.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\Common Files\McAfee\Platform\MSM\McSmtFwk.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\Common Files\McAfee\Platform\McUICnt.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\McAfee\MSC\OOBE\mcocrollback.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\McAfee\MSC\McInstru.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\Common Files\McAfee\UPDMGR\8.1.149.1\mcupdutl.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\Common Files\McAfee\DynamicAppDownloader\DADUpdater.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\Common Files\McAfee\HackerWatch\hwupdchk.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\McAfee\MSC\mcsync.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\Common Files\McAfee\ChromiumContainer\delegate.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\McAfee\VUL\McVulCtr.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\McAfee\VUL\McVulCon.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\McAfee\MfeAV\MfeAVSvc.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\McAfee\CoreUI\Launch.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>
<Image condition=”is”>C:\Program Files\McAfee.com\Agent\mcagent.exe</Image> <!–McAfee:AntiVirus: no description – TBD–>

Enabling TLS 1.2 on the Splunk 6.2x Console and Forwarders using Openssl and self signed certs.

01 Friday Jan 2016

Posted by Slice2 in Security, Splunk

≈ Leave a comment

Tags

Security, Splunk

Good luck. You will need it. Certificates are a major headache and complicated to implement. Using them with Splunk is no different. Splunk’s penchant for twiddling files all over the place makes this process time consuming an rife with error. This post will hopefully help you get it done. This covers encrypting the management console and forwarder traffic. This HOWTO is not for a clustered deployment although it could be adapted to serve that purpose. It was done on Windows 2012 R2 with a single Splunk Enterprise deployment (search head and indexer on the same server) and several forwarders. Use your own naming conventions and hosts for fqdn. Please don’t ask me questions on this post. I almost didn’t survive the process. I won’t have time to reply for a while anyway.

1) On the Splunk Search Head, set your environment.

> cd C:\Program Files\Splunk\bin
> splunk envvars > setsplunkenv.bat & setsplunkenv.bat
> setsplunkenv.bat

2) Create dir $SPLUNK_HOME\etc\auth\UScerts and cd into it.

> cd C:\Program Files\Splunk\etc\auth\UScerts

3) Create a root key.
> openssl genrsa -aes256 -out USCA_root.key 2048

4) Generate and sign the certificate.
> openssl req -new -key USCA_root.key -out USCA_root.csr

5) Generate the public certificate.
> openssl x509 -req -in USCA_root.csr -sha256 -signkey USCA_root.key -CAcreateserial -out USCA_root.pem -days 3650

6) Generate a key for your Web(search head)server certificate.
> openssl genrsa -aes256 -out me.fqdn.com.key 2048

7) Request and sign a new server certificate.
> openssl req -new -key me.fqdn.com.key -out me.fqdn.com.csr

8) Use the CSR me.fqdn.com.csr and your CA certificate and private key to generate a server certificate.
> openssl x509 -req -in me.fqdn.com.csr -sha256 -CA USCA_root.pem -CAkey USCA_root.key -CAcreateserial -out
me.fqdn.com.pem -days 730

9) Creating a (removing encryption from priv key) priv key without a passphrase. Required for webservers.
> openssl rsa -in me.fqdn.com.key -out me.fqdn.com_nopass.key

10) Create a combined cert file.
> type me.fqdn.com.pem me.fqdn.com_nopass.key USCA_root.pem > me.fqdn.com_nopass_use.pem

11) On the search head, edit the \etc\system\local\web.conf and add the following:
[settings]
enableSplunkWebSSL = 1
httpport = 8843
privKeyPath = etc\auth\UScerts\me.fqdn.com_nopass_use.pem
CaCertPath = etc\auth\UScerts\USCA_root.pem

Add to \etc\system\local\server.conf

enableSplunkdSSL = true
sslVersions = tls1.2
allowSslCompression = false
allowSslRenegotiation = false
cipherSuite = TLSv1+HIGH:@STRENGTH

12) Restart Splunk. Close your browser, relaunch and login to the console to verify (make sure to use the port defined above in web.conf; https://hostname or ip:8843).  If you scan with Nessus, Retina, etc., it should now be free from SSL errors.

Certs for Forwarders:

Create a SAN (subject alternative name) cert. Although not officially supported by Splunk when I originally wrote this, it does work.

1) Create a new folder in etc\auth\UScerts\SANcert.

2) Copy the openssl.cnf to the new folder.  C:\Program Files\Splunk\openssl.cnf to C:\Program Files\Splunk\etc\auth\UScerts\SANcert

> cd C:\Program Files\Splunk\etc\auth\UScerts\SANcert

3) In Windows 2012 R2 – Take ownership of the copied openssl.cnf file. Right-click > properties, and then add your user with Full Control to the file.

4) In Notepad or Wordpad, edit openssl.cnf. Wordpad is preferred.

a) Search (using the Find function in the upper right of Wordpad) and uncomment this line:
# req_extensions = v3_req # The extensions to add to a certificate request

b) Next, search for and modify this section to include the following if it does not already have it:
[ v3_req ] # Extensions to add to a certificate request
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAtlName = @alt_names

c) Create this section next, between [ v3_req ] and [ v3_ca ].
[alt_names]
DNS.1 = your.server.com
DNS.2 = your.next.server.com
DNS.3 = your.other.server.com
IP.1 = an IP address for a server
IP.2 = another IP address for a server
— note, add as many as you like. You will need one for each forwarder if you want to identify them individually.

5) Generate a new CSR.
openssl req -new -key me.fqdn.com.key -out me.fqdn.com_SAN.csr -config “C:\Program Files\Splunk\etc\auth\UScerts\SANcert\openssl.cnf”

Make sure you use: *.your.fqdn for Common name question. This is the wildcard for your domain, such as *.yourdomain.com

6) Check text of cert csr. You should see the items in the alt_names from above.
> openssl req -text -noout -in me.fqdn.com_SAN.csr

7) Create a cert.
> openssl x509 -req -in me.fqdn.com_SAN.csr -sha256 -CA USCA_root.pem -CAkey USCA_root.key -CAcreateserial -out me.fqdn.com_SAN.pem -extensions v3_req -days 730 -extfile “C:\Program Files\Splunk\etc\auth\UScerts\SANcert\openssl.cnf”

8) On the Indexer, edit \etc\system\local\inputs.conf and add the following and restart Splunk.

[SSL]
rootCA = etc\auth\UScerts\USCA_root.pem
servercert = etc\auth\UScerts\me.fqdn.com_SAN.pem
password = your_password
cipherSuite = TLSv1+HIGH:@STRENGTH

[splunktcp-ssl:9997]
compressed = false

9) Now restart splunk:
$SPLUNK_HOME\bin\splunk restart splunkd

10) Configure your Forwarders to use the certificates. Use your Deployment Server to distribute the certs and modified outputs.conf to your forwarders.

a) On the Search head that is acting as your deployment server, edit the outputs.conf file in etc\deployment-apps\<your name for SendToIndexer>\local\ with the following.

[tcpout]
defaultGroup = splunkssl

[tcpout:splunkssl]
server = your.ip.:9997
compressed = false
sslRootCAPath = etc\apps\<your name for SendToIndexer>\USCA_root.pem
sslCertPath = etc\apps\<your name for SendToIndexer>\me.fqdn.com_SAN.pem
sslPassword = <your password>
sslVerifyServerCert = true

b) Copy the etc\auth\UScerts\USCA_root.pem and etc\auth\UScerts\me.fqdn.com_SAN.pem files to the etc\deployment-apps\<your name for SendToIndexer>\local folder on your deployment server and they will be copied to each Forwarder for you.

11) Restart Splunk.
$SPLUNK_HOME\bin\splunk restart splunkd

12) Done. I hope.

Follow Blog via Email

Enter your email address to follow this blog and receive notifications of new posts by email.

Recent Posts

  • Patch Alma Linux 8.7 on an Offline or Air-Gapped System
  • HOWTO Remove /home logical volume and add that space to the root partition
  • Patch Rocky Linux 8.6 on an Offline or Air-Gapped System
  • HOWTO Install the Splunk Universal Forwarder on FreeBSD
  • HOWTO install a Splunk Universal Forwarder on Solaris 11 SPARC and x64 Using pkg(p5p) and tar
  • HOWTO install a Splunk Universal Forwarder on Solaris 10 SPARC and x64 Using pkgadd and tar
  • Recover Files from a Windows NTFS partition using Linux based SystemRescue
  • Sysmon Event ID 1 Process Creation rules for Splunk Universal Forwarder and McAfee All Access
  • Upgrading CentOS 7.2003 to 7.2009 on an Offline or Air-Gapped System
  • HOWTO Easily Resize the Default LVM Volume on Ubuntu 18.04
  • Create a Docker Container for your Cisco ESA, SMA or WSA Offline Content Updates
  • Apply the Mozilla Firefox STIG to Firefox on Ubuntu Linux 18.04
  • Dynamically Resize Those Tiny BlackArch Linux Terminals and Add a Scrollbar
  • Kali Linux OVA for Air-Gapped Use Build Process
  • HOWTO install the XFCE 4 Desktop on NetBSD 8.1
  • Build a Kali Linux ISO with the latest OS patches and packages
  • HOWTO quickly STIG Firefox 59.01
  • HOWTO mount a Synology NAS SMB share on Linux with SMBv1 disabled
  • Howto safely delete the WSUS WID on Windows 2012R2
  • HOWTO quickly STIG Firefox 45.0.1
  • Completing the vSphere vCenter Appliance Hardening Process
  • HOWTO install the XFCE 4.12 Desktop on NetBSD 7
  • Enabling TLS 1.2 on the Splunk 6.2x Console and Forwarders using Openssl and self signed certs.
  • HOWTO enable SSH on a Cisco ASA running 9.1.x
  • Apply a Windows 2012 R2 Domain GPO to a standalone Windows 2012 R2 server
  • Enable legacy SSL and Java SSL support in your browser for those old, crusty websites
  • HOWTO update FreeBSD 10.1 to the latest 11-current release
  • HOWTO Secure iSCSI Luns Between FreeBSD 10.1 and NetApp Storage with Mutual CHAP
  • HOWTO install the XFCE 4 Desktop on NetBSD 6.1.5
  • HOWTO Secure iSCSI Luns Between Ubuntu Server 14.10 and NetApp Storage with Mutual CHAP

Categories

  • Cisco (2)
  • ESXi (4)
  • FreeBSD (2)
  • HP (5)
  • iSCSI (12)
  • Linux (31)
  • Nessus (3)
  • NetApp (31)
  • NetBSD (10)
  • Oracle (9)
  • Security (48)
  • Solaris (9)
  • Splunk (5)
  • VMware (19)
  • Windows (20)
  • Wireshark (4)
  • XFCE (3)

Archives

  • February 2023
  • August 2022
  • July 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • November 2021
  • January 2021
  • December 2020
  • November 2020
  • August 2020
  • May 2020
  • September 2019
  • August 2019
  • March 2018
  • November 2016
  • March 2016
  • January 2016
  • November 2015
  • July 2015
  • June 2015
  • February 2015
  • January 2015
  • December 2014
  • November 2014
  • October 2014
  • September 2014
  • August 2014
  • July 2014
  • May 2014
  • April 2014
  • March 2014
  • February 2014
  • January 2014
  • December 2013
  • November 2013
  • October 2013
  • September 2013
  • August 2013
  • July 2013
  • June 2013
  • May 2013
  • April 2013
  • March 2013

Blogroll

  • Adobe Security Bulletins
  • CentOS Blog
  • Cisco Security Blog
  • CSO Magazine
  • DHS National Vulnerability Database
  • Eric Sloof's NTPRO
  • HT SSL Tests
  • Intel Corp Security Advisories
  • Internet Usage World Stats
  • Kali Linux Blog
  • Linux Mint Blog
  • Meltdown and Spectre
  • Microsoft Security Blog
  • Microsoft Security Intelligence Report
  • Microsoft Security Research & Defense
  • Microsoft Security Response Center
  • MITRE CVE Site
  • NetApp Blogs
  • NetBSD Blog
  • Oracle OTN Security
  • Oracle Security Blog
  • PacketStorm
  • Redhat Security Blog
  • SC Magazine
  • Shodan Search Engine
  • US-CERT Alerts
  • US-CERT Bulletins
  • US-CERT Vulnerability Notes KB
  • VMware Blogs
  • VMware Security Advisories

Category Cloud

Cisco ESXi FreeBSD HP iSCSI Linux Nessus NetApp NetBSD Oracle Security Solaris Splunk VMware Windows Wireshark XFCE

Follow Blog via Email

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 38 other subscribers

Powered by WordPress.com.

 

Loading Comments...