Split or filter your PCAP files with SplitCap

Erik Hjelmvik

Sunday, 08 May 2011 17:41:00 (UTC/GMT)

Split or filter your PCAP files with SplitCap

I've released version 1.6 of a tool called SplitCap today. SplitCap is a really fast PCAP file splitter, which can be used to split large pcap files based on for example IP addresses or sessions. You can also use SplitCap in order to filter out traffic to/from specific IP addresses or port numbers from a large pcap file.

The best thing about SplitCap is that it is FAST, much faster than the alternatives.

Split frames in a PCAP file into one file per IP

The -s switch can be used with arguments flow, host, hostpair or nosplit and tells SplitCap how the frames should be separated into the output pcap files.

SplitCap.exe -r huge.pcap -s host

When running the command above SplitCap creates a directory called "huge.pcap" and fills it with lots of pcap files -- one pcap file per identified IP-address (i.e. host). The generated files are named according to the IP address they contain traffic for. Here is how the files from huge.pcap are named:

huge.pcap.Host_128-183-104-74.pcap
huge.pcap.Host_129-33-21-40.pcap
huge.pcap.Host_129-70-4-55.pcap
huge.pcap.Host_129-82-103-72.pcap
huge.pcap.Host_130-212-20-225.pcap
huge.pcap.Host_130-212-255-255.pcap
huge.pcap.Host_131-107-39-4.pcap
huge.pcap.Host_131-107-8-43.pcap
huge.pcap.Host_216-109-126-56.pcap
huge.pcap.Host_64-236-24-4.pcap
huge.pcap.Host_80-67-66-6.pcap
huge.pcap.Host_fe80--202-2dff-fe38-85c0.pcap
huge.pcap.Host_ff02--2.pcap

Split frames based on session

What I believe is a unique feature of SplitCap is the ability to split a pcap file based on session, i.e. the frames from each TCP or UDP session are placed in a separate pcap file. The session split mode is the default one, but you can also use the -s session argument to tell SplitCap to use the session split mode.

SplitCap.exe -r huge.pcap -s session

The generated files get names describing the 5-tuple (proto, src_ip, src_port, dst_ip, dst_port) for which they contain traffic. Here is an example showing how the files can be named:

huge.pcap.TCP_128-183-104-74_80_192-168-21-249_32788.pcap
huge.pcap.TCP_128-183-104-74_80_192-168-21-249_32789.pcap
huge.pcap.TCP_80-67-66-6_80_192-168-22-22_49208.pcap
huge.pcap.TCP_80-67-66-6_80_192-168-22-22_49209.pcap
huge.pcap.UDP_130-212-20-225_1038_239-255-255-250_1900.pcap
huge.pcap.UDP_130-212-20-225_137_130-212-255-255_137.pcap
huge.pcap.UDP_130-212-20-225_138_130-212-255-255_138.pcap

Note that the session parameter tells SplitCap to join frames going both to and from the server into the same pcap file. If you instead wanna separate the two directions into separate pcap files, then you can use the -s flow switch.

Filter PCAP file on IP address

You can filter a pcap file based on address with the -ip switch like this:

SplitCap.exe -r huge.pcap -ip 128.183.104.74 -s nosplit

The -s nosplit argument is used to tell SplitCap not to split the pcap into one file per session. The generated file "huge.pcap.NoSplit.pcap" will only contain frames going to or from the IP address 128.183.104.74. You can also specify multiple IP addresses if you are interested in traffic to/from more than one IP address:

SplitCap.exe -r huge.pcap -ip 128.183.104.74 -ip 80.67.66.6 -s nosplit

Filter PCAP file on port number

What if you are only interested in DNS (UDP+TCP 53) and HTTP (TCP 80) traffic in a PCAP file? Well, then you can specify these port numbers as arguments to SplitCap like this:

SplitCap.exe -r huge.pcap -port 53 -port 80 -s nosplit

This command creates a file named "huge.pcap.NoSplit.pcap" that only contains the frames going to or from ports 53 and 80.

Extract application layer (L7) contents

A pretty cool thing with SplitCap is the ability to extract application (i.e. layer 7) data from a pcap file with the -y L7 argument. I'll use the file "dump.eth0.1059726000.pcap" from Defcon 11 here to demonstrate how to extract layer 7 payload from SMTP (TCP 25) traffic to files (one per session).

SplitCap.exe -r dump.eth0.1059726000.pcap -s session -port 25 -y L7

This command creates 10 files with a ".bin" file extention in the output directory. Each such bin file contains the application layer data for both directions (server->client and vice versa) from an SMTP session.

One of the generated SMTP session files is called "dump.eth0.1059726000.pcap.TCP_64-48-248-30_25_192-168-17-79_33443.bin" and looks something like this (I've redacted some of the contents):

220 smtp102.mail.sc5.yahoo.com ESMTP
[SMTP LOGIN REDACTED]
MAIL FROM: <[REDACTED]@nytimes.com>
250 ok
RCPT TO: <[REDACTED]@[REDACTED].senate.gov>
250 ok
DATA
354 go ahead
Reply-To: <[REDACTED]@nytimes.com>
From: "[REDACTED]" <[REDACTED]@nytimes.com>
To: "[REDACTED]" <[REDACTED]@[REDACTED].senate.gov>
Subject: RE: Just checking in
Date: Fri, 1 Aug 2003 16:28:25 -0400
Message-ID: [REDACTED]
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="----=_NextPart_000_0700_01C35849.EE94EC60"
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106
Importance: Normal
In-Reply-To: [REDACTED]

This is a multi-part message in MIME format.

------=_NextPart_000_0700_01C35849.EE94EC60
Content-Type: text/plain;
charset="US-ASCII"
Content-Transfer-Encoding: 7bit

i am going to gulfport mississippi for the story. am in las vegas now. will
be in mississippi on sunday.
the guy who is really good on this and helped me actually was [REDACTED]
from uspirg. he put me in contact with a whistle blower.

[REDACTED]
-----Original Message-----
From: [REDACTED] [mailto:[REDACTED]@[REDACTED].senate.gov]
Sent: Friday, August 01, 2003 4:17 PM
To: [REDACTED]@nytimes.com
Subject: Just checking in

I wanted to make sure you are getting everything you need from us and to
check in on the story. Let me know if/when you want to sit down with
[REDACTED] on this. Hope all is well, [REDACTED]

SplitCap is FAST, really FAST!

So how fast is SplitCap at splitting or filtering a pcap file? Let's do a simple benchmark and compare it to the well known tool Tshark. Again, lets use the 189 MB file "dump.eth0.1059726000.pcap" from Defcon 11 for this example.

Filtering the file based on IP with Tshark takes 50 seconds (4 MB/s).