Carving Packets from Memory

Someone who says "We're gonna pull the packet captures out of the router" probably has no clue how to capture network traffic. In the Lindell case, statements like these were results of an elaborate hoax.

Nevertheless, such a statement doesn't have to be nonsense — if it comes from someone who knows how to dump the physical memory from the router. There are actually more packets available in the RAM of a router, or computer for that matter, than you might think.

The Forensic Challenge from DFRWS 2016 contains a memory dump from an SDN switch. If you drag-and-drop SDN.ram.raw from that challenge to CapLoader then you'll be asked if you wanna carve packets from the memory dump.

This packet carving feature is also available in the free trial version of CapLoader.

Clicking "Yes" in the dialogue brings up a configuration window. The default settings are okay in most cases.

After pressing "Start" CapLoader will start identifying packets in the memory dump from the SDN switch. The packets will be saved to a Pcap-NG file located in the %TEMP% directory, unless you specified a different output location in the config window.

You can download a copy of the Pcap-NG file that I generated with CapLoader 1.9.2 here:
https://media.netresec.com/pcap/SDN.ram.raw.pcapng (661 kB, 2959 packets)

Here's what it looks like when the carved packets have been loaded into NetworkMiner Professional.

As you can see, a great deal of information can be extracted about the hosts on this network just by examining the dumped memory from the SDN switch.

What about Bulk Extractor?

Simson Garfinkel's bulk_extractor can also extract packets from memory dumps. It was actually a research paper by Simson that inspired me to implement a packet carver in the first place.

There are a few significant differences between bulk_extractor and CapLoader with regards to packet carving though. One difference is that bulk_extractor identifies network packets by looking for Ethernet frames containing IPv4 packets, while CapLoader looks for IPv4 or IPv6 packets containing TCP or UDP packets. The output from bulk_extractor is usually quite similar to that of CapLoader, and so is the parsing speed. CapLoader was just slightly faster in our tests and carved about 3% more packets compared to bulk_extractor, these additional packets were primarily IPv6 packets and packets that weren't encapsulated by an Ethernet frame.

Where can I download memory dumps?

I posted a question on Twitter, asking the #DFIR community for their favorite publicly available memory dumps prior to writing this blog post, and I received lots of great answers. Thank you all for contributing! I have now compiled the following list of places from where you can download memory dumps:

• Memory Images from the Stolen Szechuan Sauce case by James Smith
• Digital Corpora 2018 Lone Wolf Scenario by Thomas Moore
• Digital Corpora 2009 M57 Patents Scenario by Naval Postgraduate School
• DFRWS 2005 Challenge
• DFRWS 2008 Challenge
• DFRWS 2016 Challenge (SDN switch)
• CTF-styled memory forensic challenges by Abhiram Kumar
• Mini Memory CTF contest from 2020 by 13Cubed
• Web Server Case by Ali Hadi
• Locky Ransomware infection by Jiří Vinopal
• Memory images from the Art of Memory Forensics (book) by the Volatility Team
• Memory Forensic challenges on CyberDefenders
• GrrCON 2015 CTF by Wyatt Roersma
• GrrCON 2017 Memory Dumps by Wyatt Roersma
• GrrCON 2019 CTF by Wyatt Roersma

For a more detailed blog post on CapLoader's packet carving functionality, please see our Carving Network Packets from Memory Dump Files blog post from 2014.

Posted by Erik Hjelmvik on Tuesday, 31 August 2021 15:10:00 (UTC/GMT)

Tags: #Forensics​ #RAM​ #PCAP​ #Pcap-NG​ #PcapNG​ #DFIR​ #carve​ #carver​ #packets​ #dump​ #CapLoader​ #memory forensics​ #DFRWS​

CapLoader 1.9 Released

A new version of the PCAP filtering tool CapLoader has been released today. The new CapLoader version 1.9 is now even better at identifying protocols and periodic beacons than before. The user interface has also been improved to make it easier to filter and drill down in network traffic to extract interesting, malicious or unusual traffic.

More Protocols Identified

We’ve added port-independent protocol detection for over 20 new protocols since the last release. The newly added protocols include some that are used by malicious tools and backdoors such as hTran, RevengeRAT, Tofsee and Winsecsrv, as well as legitimate protocols like WireGuard (VPN) and RemoteFX (UDP based remote desktop). We’ve also improved our support for ICS traffic analysis by adding protocol identification of SCADA protocols DNP3 and IEC 60870-5-104.

CapLoader also detects what we call “sub-protocols”, which are communication protocols that use other L7 protocols as transport. We have extended the sub-protocol detection in CapLoader 1.9 to include traffic like Anchor_DNS and dnscat traffic, which both run on top of DNS. We have also added detection of Cobalt Strike beacons over HTTP and HTTPS, even though the latter is quite difficult to detect due to the application data being encrypted.

Improved Usability

CapLoader 1.9 comes with several user interface improvements that help you solve the “needle in the haystack” problem even more efficiently than before.

The context menus in the Flows, Services and Hosts tabs can now be used to select rows based on values in any column, such as “Select all flows where Duration > 10 minutes” (when right-clicking a 10 minute flow).

The “Keyword Filter” is now called “Row Filter” in order to avoid getting it mixed up with the “Find Keyword” feature. The Row Filter has also been enhanced with a new filtering mode, to complement the Contains / All Words / Any Words / RegEx options, which is called “Column Criteria”. The Column Criteria can be used to filter the displayed rows based on the values in a user-specified column. The Column Criteria “Duration > 00:10:00” will, for example, only show flows that are 10 minutes or longer, while “ASN = 3301” shows the flows going to Telia’s AS3301.

Image: CapLoader with Row Filter Column Criteria "Duration > 00:10:00"

We have also extended CapLoader's BPF implementation to support VLAN id’s, so that you can use expressions like “vlan 100” as input filter as well as display filter. The BPF implementation also supports logic operators, so that more advanced filters like “(tcp port 80 or port 443) and not net 149.154.172.0/22” can be used.

CapLoader has a method for detecting periodic connection patterns, which was introduced in CapLoader 1.4. This feature can be used to detect clients that connect to a service at regular intervals, such as a beacon used for command-and-control or email client connecting to a mail server. We have improved the periodicity detection in CapLoader 1.9 so that it now detects periodic services more accurately.

The Initial Round Trip Time (iRRT) in the Flows and Services tabs is now measured in milliseconds instead of seconds in order to avoid “bulky numbers” (h/t Eddi).

There was previously a significant delay when selecting many flows at once (like 100.000). We’ve improved the performance of this feature in CapLoader 1.9, so that you can now select several hundred thousands flows at once without having to wait for an unresponsive GUI to update.

More OSINT Lookup Services

A feature in CapLoader that often comes in handy is the ability to right-click a flow, service or host and open a website with OSINT information about the clicked IP address or domain name. We have now replaced some of the OSINT services with new better ones.

The new services we’ve added to CapLoader 1.9 for performing online OSINT lookups of IP addresses, network services and domain names are:

• CIRCL BGP Ranking
• Hacker Target Reverse Lookup
• IPAddress.com
• OpenDNS
• RIPEstat
• RiskIQ
• Scamalytics
• Spur
• Spyse
• ThreatFox
• URLhaus
• ZoomEye

Bug fixes and Credits

Several bugs have been fixed in this new release of CapLoader, much thanks to feedback we’ve received from our users. We’d like to thank Anders Regert and Mandy van Oosterhout for reporting bugs in CapLoaders “Save As” feature. We’d also like to thank Hyun Dowon for reporting a snap length corruption bug that previously appeared when exporting flows from Pcap-NG files We have also fixed an issue where capture files were previously not always merged in chronological order when being aggregated.

Updating to the Latest Release

Users who have purchased a license for CapLoader can download a free update to version 1.9 from our customer portal. All others can download a free 30 day trial from the CapLoader product page (no registration required).

Posted by Erik Hjelmvik on Tuesday, 25 May 2021 12:20:00 (UTC/GMT)

Tags: #Netresec​ #CapLoader​ #PCAP​ #Pcap-NG​ #IEC-104​ #CobaltStrike​ #BPF​ #periodicity​ #OSINT​

CapLoader 1.5 Released

We are today happy to announce the release of CapLoader 1.5. This new version of CapLoader parses pcap and pcap-ng files even faster than before and comes with new features, such as a built-in TCP stream reassembly engine, as well as support for Linux and macOS.

Support for ICMP Flows

CapLoader is designed to group packets together that belong to the same bi-directional flow, i.e. all UDP, TCP and SCTP packets with the same 5-tuple (regardless of direction) are considered being part of the same flow.

5-tuple

/fʌɪv ˈtjuːp(ə)l/

noun

A combination of source IP, destination IP, source port, destination port and transport protocol (TCP/UDP/SCTP) used to uniquely identify a flow or layer 4 session in computer networking.

The flow concept in CapLoader 1.5 has been extended to also include ICMP. Since there are no port numbers in the ICMP protocol CapLoader sets the source and destination port of ICMP flows to 0. The addition of ICMP in CapLoader also allows input filters and display filters like “icmp” to be leveraged.

TCP Stream Reassembly

One of the foundations for making CapLoader a super fast tool for reading and filtering PCAP files is that it doesn’t attempt to reassemble TCP streams. This means that CapLoader’s Transcript view will show out-of-order segments in the order they were received and retransmitted segments will be displayed twice.

The basic concept has been to let other tools do the TCP reassembly, for example by exporting a PCAP for a flow from CapLoader to Wireshark or NetworkMiner.

The steps required to reassemble a TCP stream to disk with Wireshark are:

1. Right-click a TCP packet in the TCP session of interest.
2. Select “Follow > TCP Stream”.
3. Choose direction in the first drop-down-list (client-to-server or server-to-client).
4. Change format from “ASCII” to “Raw” in the next drop-down-menu.
5. Press the “Save as...” button to save the reassembled TCP stream to disk.

Unfortunately Wireshark fails to properly reassemble some TCP streams. As an example the current stable release of Wireshark (version 2.2.5) shows duplicate data in “Follow TCP Stream” when there are retransmissions with partially overlapping segments. We have also noticed some additional  bugs related to TCP stream reassembly in other recent releases of Wireshark. However, we’d like to stress that Wireshark does perform a correct reassembly of most TCP streams; it is only in some specific situations that Wireshark produces a broken reassembly. Unfortunately a minor bug like this can cause serious consequences, for example when the TCP stream is analyzed as part of a digital forensics investigation or when the extracted data is being used as input for further processing. We have therefore decided to include a TCP stream reassembly engine in CapLoader 1.5. The steps required to reassemble a TCP stream in CapLoader are:

1. Double click a TCP flow of interest in the “Flows” tab to open a flow transcript.
2. Click the “Save Client Byte Stream” or “Save Server Byte Stream” button to save the data stream for the desired direction to disk.

Extracting TCP streams from PCAP files this way not only ensures that the data stream is correctly reassembled, it is also both faster and simpler than having to pivot through Wireshark’s Follow TCP Stream feature.

PCAP Icon Context Menu

The PCAP icon in CapLoader is designed to allow easy drag-and-drop operations in order to open a set of selected flows in an external packet analysis tool, such as Wireshark or NetworkMiner. Right-clicking this PCAP icon will bring up a context menu, which can be used to open a PCAP with the selected flows in an external tool or copy the PCAP to the clipboard. This context menu has been extended in CapLoader 1.5 to also include a “Save As” option. Previous versions of CapLoader required the user to drag-and-drop from the PCAP icon to a folder in order to save filtered PCAP data to disk.

Faster Parsing with Protocol Identification

CapLoader can identify over 100 different application layer protocols, including HTTP, SSL, SSH, RTP, RTCP and SOCKS, without relying on port numbers. The protocol identification has previously slowed down the analysis quite a bit, which has caused many users to disable this powerful feature. This new release of of CapLoader comes with an improved implementation of the port-independent protocol identification feature, which enables PCAP files to be loaded twice as fast as before with the “Identify protocols” feature enabled.

Works in Linux and macOS

One major improvement in CapLoader 1.5 is that this release is compatible with the Mono framework, which makes CapLoader platform independent. This means that you can now run CapLoader on your Mac or Linux machine if you have Mono installed. Please refer to our previous blog posts about how to run NetworkMiner in various flavors of Linux and macOS to find out how to install Mono on your computer. You will, however, notice a performance hit when running CapLoader under Mono instead of using Windows since the Mono framework isn't yet as fast as Microsoft's .NET Framework.

Credits

We’d like to thank Sooraj for reporting a bug in the “Open With” context menu of CapLoader’s PCAP icon. This bug has been fixed in CapLoader 1.5 and Sooraj has been awarded an official “PCAP or it didn’t happen” t-shirt for reporting the bug.

Have a look at our Bug Bounty Program if you also wanna get a PCAP t-shirt!

Downloading CapLoader 1.5

Everything mentioned in this blog post, except for the protocol identification feature, is available in our free trial version of CapLoader. To try it out, simply grab a copy here:
https://www.netresec.com/?page=CapLoader#trial (no registration needed)

All paying customers with an older version of CapLoader can download a free update to version 1.5 from our customer portal.

Posted by Erik Hjelmvik on Tuesday, 07 March 2017 09:11:00 (UTC/GMT)

Tags: #CapLoader​ #PCAP​ #pcap-ng​ #Follow TCP Stream​ #Wireshark​ #Linux​ #Mac​ #macOS​ #RTP​ #SOCKS​ #Mono​ #Flow​