Requirements:
Analyst Console
A SI(E)M or firewall event manager capable of launching external tools from log items.
Perl with base TK module
cmd line ssh & scp (putty and pscp)
GetData Perl Script ver 0.02
Sensor
Snort,TCPdump or dumpcap
logpackets.sh from sguil sensor distribution(if using snort)
Summary:
Using Network Security Monitoring concepts learned from using Sguil , I was able create a framework to query full content data directly from almost any SIM event. The framework utilizes only open-source tools and enforces best practices on building the sensor to store and transfer the data back to the SIM console for further analysis.
When setup is complete, you will have a sensor collecting "full-content data" off the wire and (optimally) you will need only 1 port open on the sensor(sshd). All sensor queries and file copies will be executed over sshd(via the ssh/plink and scp/pscp clients).
The perl script on the console will recieve arguments via the SIM from the event chosen. The script will parse the data and formulate a BPF query string that is passed to tcpdump via ssh from the console. The resulting outputfile is then scp'ed back to the console and launched into your PCAP analyzer of choice(ethereal,wireshark).
Screenshots
Step 1: Choose the firewall/IDS event to investigate
Step 2: Login to full-content datastore sensor
Step 3: The script parses the data,queries the sensor,scp's the PCAP file and launches Wireshark
Step4: Analyze the Packets in Wireshark
Preparing a Sensor:
My original setup was based on Sun Solaris 10, but the methods involved can be applied to any *nix/*BSD setup. The hardware used was a Sun Sparc V210 with 1GB/RAM and an UltraSparc 4 1.2ghz processor. The onboard Quad Fast Ethernet (Broadcomm 4 x 100Mbs) was utilized for both administrative (SSH access) and as the sniffing port.
Minimal install
A minimal (core) install of Solaris 10 was done as besides the third party application and libs, only a TCP/IP stack is needed. SSH(and dependent packages) need to be installed (off the media) manually as Sun does not provide it with 'core' installs. The next step regardless of Solaris version is to go to sunsolve.sun.com and download the latest "Recommended and Security" patches for your Solaris version your own workstation. This was then burned to a CD and inserted into the server/sensor for installation. Keep in mind as this point, I have not connected any network cables to the sensor.
Patch and Harden
After the patches are installed and the system is rebooted it's time to start hardening the OS. This is accomplished with the assistance of the "Freely Available" Benchmark Documents available at www.cisecurity.org(or use your hardening tool of choice). When the hardening is complete and you have rebooted, running "netstat -an | grep LISTEN" should only produce a single line(unless you have sshd listening on IPV6) representing the administrative IP listening on port 22. (tcpwrappers "hosts.allow" for sshd should reflect consoles administrative IP only). Extra levels of protection can be added such as iptables/ipf/BSM auditing etc to further improve the posture of OS.
Setup PCAP storage area
The Packetdumps should be stored on their own partition/drive. Sizing should be done based on the amount of data you intend to store. For example, if you are saving 4GB's an hour and your partition is 40GB's , you can always refer back to the last 9 hours traffic as a process will be in place to delete the oldest capture file when disk utilization hits 90%.
Third party apps
On a minimally installed OS, you should not have a compiler installed. If possible, all binaries should be created on a separate development box, then copied over. Download the lastest version of libpcap(for everything :-) ), tcpdump(for remote query of file) and a packetlogger, either snort (to run in packetlogger mode) or dumpcap (a packetlogger, packaged with wireshark). Having previously used Sguil, I choose to keep logpackets.sh to manage my packetlogger process and manage the filesystem on which it exists. You can really use any packetlogger and any script to manage it, just make sure the files it output contain a timestamp(epoch) so that the GetData.pl script can figure out what time that file started.
Sguils logpacket.sh uses snort as the packet logger. The Wireshark project(formerly Ethereal) now packages "dumpcap" which I have not done any testing on yet.
When it comes down to it, All the sensor is doing is running a packetlogger, and managing the files it outputs. These files will be queried via ssh and the data will be copied back via scp.
As is good practice, using open source software or not, you should subscribe to the announce list for the software you are running, either snort, wireshark or ssh.
Another good mailing list for security consciuous individuals is the Internet Storm Centers isc-notify mailing list, typically a message a day of always relevant information to security professionals.
Preparing the Console/desktop
Download the latest version of GetData.pl from simquery.sourceforge.net and uncompress onto your workstation. Set your variables in the Config.PM in the GetData subdir. Hardcode the paths in the main script to your executables and sensor filepaths.
Check the README file to instructions on adding the tool to your SIM (Arcsight only SIM tested/listed)
The script expects to see the data/time stamp image in this format:
"DD Mon YYYY HH:MM:SS TZ"
If you are using a different date/time image amd you can't change it, email me the image you are using and I will provide a secondary option for it.
Question and comments: aspina at gmail dot com