Command-line Network Forensics with NetworkMinerCLI

NetworkMinerCLI is a Command Line Interface version of NetworkMiner Professional. Apart from being faster than the GUI version NetworkMinerCLI also has the benefit of being easy to integrate into scripts of various kinds (Batch / perl / python / PowerShell / etc).

Running NetworkMinerCLI.exe from your NetworkMiner Professional USB flash drive shows the syntax and arguments to use:

F:\NetworkMinerProfessional_1-0>NetworkMinerCLI.exe
Usage: F:\NetworkMinerProfessional_1-0\NetworkMinerCLI.exe [OPTIONS]

 -r <input_file>        : Set the pcap file to read
 -w <output_directory>  : Directory to store output files in
 -b <frame_buffer_size> : Number of frames to buffer in memory (5000 = default)
 -noHeader              : Disables column headers for CSV files

Example: F:\NetworkMinerProfessional_1-0\NetworkMinerCLI.exe -r evidence.pcap -w D:\exported_data\

When a pcap is loaded by NetworkMinerCLI it will generate eight CSV files containing data about Sessions, Parameters, Credentials, CleartextWords, DnsRecords, FileInfos, Messages and Hosts. All assembled files will also be written to disk. The assembled files, as well as the seven CSV files, are written to the current working directory or to the directory specified with the -w argument.

This is what it looks like when loading suspect.pcap from the DFRWS 2008 challenge into NetworkMinerCLI (don't let my Swedish cmd.exe shell confuse you):

F:\pcap_files\DFRWS\Challenge 2008>F:\NetworkMinerProfessional_1-0\NetworkMinerCLI.exe -r suspect.pcap
Closing file handles...
10243 frames parsed in 56,084 seconds.

F:\pcap_files\DFRWS\Challenge 2008>dir
 Volymen i enhet F har etiketten NETRESEC
 Volymens serienummer är 7017-B488

 Innehåll i katalogen F:\pcap_files\DFRWS\Challenge 2008

2011-02-23 21:19 <KAT>        .
2011-02-23 21:19 <KAT>        ..
2007-12-16 23:32    5 110 493 suspect.pcap
2011-03-04 17:20 <KAT>        AssembledFiles
2011-03-04 17:21       15 098 suspect.pcap.Sessions.csv
2011-03-04 17:21      459 868 suspect.pcap.Parameters.csv
2011-03-04 17:21       83 034 suspect.pcap.Credentials.csv
2011-03-04 17:21    3 271 213 suspect.pcap.CleartextWords.csv
2011-03-04 17:21       23 748 suspect.pcap.DnsRecords.csv
2011-03-04 17:21      182 382 suspect.pcap.FileInfos.csv
2011-03-04 17:21          837 suspect.pcap.Messages.csv
2011-03-04 17:21        8 009 suspect.pcap.Hosts.csv
             9 fil(er)       9 154 682 byte
             3 katalog(er) 711 819 264 byte ledigt

F:\pcap_files\DFRWS\Challenge 2008>

I've uploaded the generated suspect.pcap.Parameters.csv file so that you can have a look at what the CSV files from NetworkMiner look like. Corey Harrell has by the way recently written a great blog post on how to do forensic work with CSV files in OpenOffice Calc on his Journey into Incident Response Methodology blog.

The CSV files can also be parsed directly from command line with help of some script-fu. I admit being a Windows nerd, but I have still not gone through the pain of learning Windows PowerShell properly. I therefore prefer to use Cygwin, or actually xterm in Cygwin/X, in order to assemble the power of the GNU bash shell while running Windows. I can for example use the following command in order to get a list of all detected gmail accounts and the logged in clients' IPs:

cat suspect.pcap.Parameters.csv | grep gmailchat | awk -F',' '{print $1" "$9}' | sort -u
192.168.151.130 steve.vogon@gmail.com/244489

Here is a short explanation of the command above for people without UNIX fingers:

• grep gmailchat : selects all lines in the csv file containing the string “gmailchat”
• awk -F',' '{print $1" "$9}' : tokenize text between commas and display first token (IP address) and ninth token (gmail account)
• sort -u : sort each outputed line and only display unique lines (remove duplicates)

The same method can also be used in order to quickly filter out Google searches from a pcap file (remember that the search strings are sent to Google with the 'q' parameter):

cat suspect.pcap.Parameters.csv | grep 'HTTP QueryString,q,'|awk -F',' '{print $1" "$9}'
192.168.151.130 overseas credit card payments
192.168.151.130 hurricane

Or to get all the frame numbers (in the pcap file) where the word “bank” has been sent in clear text:

grep bank suspect.pcap.CleartextWords.csv -i | awk -F',' '{print$1" "$2}' | sort -u
Bank 1551
Bank 1553
Bank 1559
Bank 1566
Bank 1568
Bank 1617
Bank 1619
Bank 1629
Bank 1708
Bank 1710
Bank 1716
Bank 1724
bank 2063
banks 5797

It should also be noted that NetworkMinerCLI extracts and writes all identified files to disk, just as the normal GUI version of NetworkMiner Professional does. A good overview of these extracted files can be found in the suspect.pcap.FileInfos.csv CSV file. So if you are looking for a particular file you can search the suspect.pcap.FileInfos.csv for the MD5 sum like this:

grep e516cd1dbad131024693d31155a6577f suspect.pcap.FileInfos.csv | awk -F',' '{print $9" "$5}'
e516cd1dbad131024693d31155a6577f XRamp Security Services G.cer
e516cd1dbad131024693d31155a6577f XRamp Security Services G[1].cer
e516cd1dbad131024693d31155a6577f XRamp Security Services G[2].cer

In order to get your hands on a copy of the NetworkMinerCLI tool you need to buy a license for NetworkMiner Professional, since the USB flash drive contains the GUI as well as the CLI version of NetworkMiner Professional.

Posted by Erik Hjelmvik on Friday, 04 March 2011 17:12:00 (UTC/GMT)

Tags: #NetworkMinerCLI​ #DFRWS​