CapLoader 1.9 Released

A new version of the PCAP filtering tool CapLoader has been released today. The new CapLoader version 1.9 is now even better at identifying protocols and periodic beacons than before. The user interface has also been improved to make it easier to filter and drill down in network traffic to extract interesting, malicious or unusual traffic.

More Protocols Identified

We’ve added port-independent protocol detection for over 20 new protocols since the last release. The newly added protocols include some that are used by malicious tools and backdoors such as hTran, RevengeRAT, Tofsee and Winsecsrv, as well as legitimate protocols like WireGuard (VPN) and RemoteFX (UDP based remote desktop). We’ve also improved our support for ICS traffic analysis by adding protocol identification of SCADA protocols DNP3 and IEC 60870-5-104.

CapLoader also detects what we call “sub-protocols”, which are communication protocols that use other L7 protocols as transport. We have extended the sub-protocol detection in CapLoader 1.9 to include traffic like Anchor_DNS and dnscat traffic, which both run on top of DNS. We have also added detection of Cobalt Strike beacons over HTTP and HTTPS, even though the latter is quite difficult to detect due to the application data being encrypted.

Improved Usability

CapLoader 1.9 comes with several user interface improvements that help you solve the “needle in the haystack” problem even more efficiently than before.

The context menus in the Flows, Services and Hosts tabs can now be used to select rows based on values in any column, such as “Select all flows where Duration > 10 minutes” (when right-clicking a 10 minute flow).

The “Keyword Filter” is now called “Row Filter” in order to avoid getting it mixed up with the “Find Keyword” feature. The Row Filter has also been enhanced with a new filtering mode, to complement the Contains / All Words / Any Words / RegEx options, which is called “Column Criteria”. The Column Criteria can be used to filter the displayed rows based on the values in a user-specified column. The Column Criteria “Duration > 00:10:00” will, for example, only show flows that are 10 minutes or longer, while “ASN = 3301” shows the flows going to Telia’s AS3301.

Image: CapLoader with Row Filter Column Criteria "Duration > 00:10:00"

We have also extended CapLoader's BPF implementation to support VLAN id’s, so that you can use expressions like “vlan 100” as input filter as well as display filter. The BPF implementation also supports logic operators, so that more advanced filters like “(tcp port 80 or port 443) and not net 149.154.172.0/22” can be used.

CapLoader has a method for detecting periodic connection patterns, which was introduced in CapLoader 1.4. This feature can be used to detect clients that connect to a service at regular intervals, such as a beacon used for command-and-control or email client connecting to a mail server. We have improved the periodicity detection in CapLoader 1.9 so that it now detects periodic services more accurately.

The Initial Round Trip Time (iRRT) in the Flows and Services tabs is now measured in milliseconds instead of seconds in order to avoid “bulky numbers” (h/t Eddi).

There was previously a significant delay when selecting many flows at once (like 100.000). We’ve improved the performance of this feature in CapLoader 1.9, so that you can now select several hundred thousands flows at once without having to wait for an unresponsive GUI to update.

More OSINT Lookup Services

A feature in CapLoader that often comes in handy is the ability to right-click a flow, service or host and open a website with OSINT information about the clicked IP address or domain name. We have now replaced some of the OSINT services with new better ones.

The new services we’ve added to CapLoader 1.9 for performing online OSINT lookups of IP addresses, network services and domain names are:

• CIRCL BGP Ranking
• Hacker Target Reverse Lookup
• IPAddress.com
• OpenDNS
• RIPEstat
• RiskIQ
• Scamalytics
• Spur
• Spyse
• ThreatFox
• URLhaus
• ZoomEye

Bug fixes and Credits

Several bugs have been fixed in this new release of CapLoader, much thanks to feedback we’ve received from our users. We’d like to thank Anders Regert and Mandy van Oosterhout for reporting bugs in CapLoaders “Save As” feature. We’d also like to thank Hyun Dowon for reporting a snap length corruption bug that previously appeared when exporting flows from Pcap-NG files We have also fixed an issue where capture files were previously not always merged in chronological order when being aggregated.

Updating to the Latest Release

Users who have purchased a license for CapLoader can download a free update to version 1.9 from our customer portal. All others can download a free 30 day trial from the CapLoader product page (no registration required).

Posted by Erik Hjelmvik on Tuesday, 25 May 2021 12:20:00 (UTC/GMT)

Tags: #Netresec​ #CapLoader​ #PCAP​ #Pcap-NG​ #IEC-104​ #CobaltStrike​ #BPF​ #periodicity​ #OSINT​

CapLoader 1.6 Released

CapLoader is designed to simplify complex tasks, such as digging through gigabytes of PCAP data looking for traffic that sticks out or shouldn’t be there. Improved usability has therefore been the primary goal, when developing CapLoader 1.6, in order to help our users do their work even more efficiently than before.

Some of the new features in CapLoader 1.6 are:

• Context aware selection and filter suggestions when right-clicking a flow, session or host.
• Support for IPv6 addresses in the BPF syntax for Input Filter as well as Display Filter.
• Flows that are inactive for more than 60 minutes are considered closed. This timeout is configurable in Tools > Settings.

Latency Measurements

CapLoader 1.6 also introduces a new column in the Flows tab labeled “Initial_RTT”, which shows the Round Trip Time (RTT) measured during the start of a session. The RTT is defined as “the time it takes for a signal to be sent plus the time it takes for an acknowledgment of that signal to be received”. RTT is often called “ping time” because the ping utility computes the RTT by sending ICMP echo requests and measuring the delay until a reply is received.

But using a PCAP file to measure the RTT between two hosts isn’t as straight forward as one might think. One complicating factor is that the PCAP might be generated by the client, server or by any device in between. If we know that the sniffing point is at the client then things are simple, because we can then use the delta-time between an ICMP echo request and the returning ICMP echo response as RTT. In lack of ping traffic the same thing can be achieved with TCP by measuring the time between a SYN and the returning SYN+ACK packet. However, consider the situation when the sniffer is located somewhere between the client and server. The previously mentioned method would then ignore the latency between the client and sniffer, the delta-time will therefore only show the RTT between the sniffer and the server.

This problem is best solved by calculating the Initial RTT (iRTT) as the delta-time between the SYN packet and the final ACK packet in a TCP three-way handshake, as shown here:

Jasper Bongertz does a great job of explaining why and how to use the iRTT in his blog post “Determining TCP Initial Round Trip Time”, so I will not cover it in any more detail here. However, keep in mind that iRTT can only be calculated this way for TCP sessions. CapLoader therefore falls back on measuring the delta time between the first packet in each direction when it comes to transport protocols like UDP and ICMP.

Exclusive Features Not Available in the Free Trail

The new features mentioned so far are all available in the free 30 day CapLoader trial, which can be downloaded from our CapLoader product page (no registration required). But we’ve also added features that are only available in the commercial/professional edition of CapLoader. One such exclusive feature is the matching of hostnames against the Cisco Umbrella top 1 million domain list. CapLoader already had a feature for matching domain names against the Alexa top 1 million list, so the addition of the Umbrella list might seem redundant. But it’s actually not, the two lists are compiled using different data sources and therefore complement each other (see our blog post “Domain Whitelist Benchmark: Alexa vs Umbrella” for more details). Also, the Umbrella list contains subdomains (such as www.google.com, safebrowsing.google.com and accounts.google.com) while the Alexa list only contains main domains (like “google.com”). CapLoader can therefore do more fine-granular domain matching with the Umbrella list (requiring a full match of the Umbrella domain), while the Alexa list enables a more rough “catch ‘em all” approach (allowing *.google.com to be matched).

CapLoader 1.6 also comes with an ASN lookup feature, which presents the autonomous system number (ASN) and organization name for IPv4 and IPv6 addresses in a PCAP file (see image above). The ASN lookup is built using the GeoLite database created by MaxMind. The information gained from the MaxMind ASN database is also used to provide intelligent display filter CIDR suggestions in the context menu that pops up when right-clicking a flow, service or host.

Users who have previously purchased a license for CapLoader can download a free update to version 1.6 from our customer portal.

Credits and T-shirts

We’d like to thank Christian Reusch for suggesting the Initial RTT feature and Daan from the Dutch Ministry of Defence for suggesting the ASN lookup feature. We’d also like to thank David Billa, Ran Tohar Braun and Stephen Bell for discovering and reporting bugs in CapLoader which now have been fixed. These three guys have received a “PCAP or it didn’t happen” t-shirt as promised in our Bug Bounty Program.

If you too wanna express your view of outlandish cyber attack claims without evidence, then please feel free to send us your bug reports and get rewarded with a “PCAP or it didn’t happen” t-shirt!

Posted by Erik Hjelmvik on Monday, 09 October 2017 08:12:00 (UTC/GMT)

Tags: #CapLoader​ #free​ #IPv6​ #BPF​ #CIDR​ #PCAP​ #Umbrella​ #Alexa​

BPF is your Friend

CapLoader comes with support for Berkeley Packet Filter (BPF), which makes it possible to filter network traffic based on IP addresses, protocols and port numbers without using external tools. Being able to filter captured network traffic is crucial when analyzing large sets of PCAP files as well as in order to hunt down compromised hosts with Rinse Repeat Intrusion Detection.

There are two ways to apply filters with BPF in CapLoader; you can either apply an input filter before loading your PCAPs, or you can apply a display filter after the capture files have been loaded.

Input Filter

The fastest way to filter a large set of PCAP files with CapLoader is to enter an Input Filter before loading the capture files. Having an input filter will speed up the loading time significantly, since CapLoader will not need to analyze packets and flows that don't match the BPF syntax. The downside is that you will need to know beforehand what filter you want to use. In order to apply a changed input filter you need to reload the loaded PCAP files (pressing F5 will do this for you).

Display Filter

CapLoader supports display filters in order to allow filters to be changed on the fly, without having to reload the PCAP files. As the name implies, display filters affect what flows/services/hosts that are displayed in CapLoader. A changed display filter does not require the dataset to be reloaded, but it does require the GUI to update the visible flows. This GUI update will be somewhat slower compared to when setting an input filter.

BPF Syntax

CapLoader's BPF implementation does not support the full BPF syntax. In fact, only the most central primitives are implemented, which are:

More complex filter expressions can be built up by using the words and, or, not and parentheses to combine primitives. Here are some examples:

• host 8.8.8.8 and udp port 53
• net 199.16.156.0/22 and port 80
• (port 80 or port 443) and not host 192.168.0.1

For all boolean algebra geeks out there we can confirm that our BPF implementation gives and precedence over or, which means that the last example above would give a different result if the parentheses were removed.

Keeping it Short

Steve McCanne gave a keynote presentation at SharkFest 2011, where he talked about how he created BPF. Steve's work was guided by Van Jacobson, who challenged him to make the BPF syntax human friendly rather than requiring the user to type a clunky filtering syntax. We've adopted this thinking and therefore allow filters like these:

• 10.1.1.3
  Flows to or from IP address 10.1.1.3. Translates to “ip host 10.1.1.3”


• 128.3/16
  Flows to or from the 128.3.0.0/16 network. Translates to “ip net 128.3.0.0/16”


• port 53
  Flows to or from TCP, UDP or SCTP port 53.

Try it for Free!

We've made the BPF implementation available even in the free version of CapLoader. You don't need to register to get the free version; just download, extract and run. The tool is portable, so you won't even have to install it. Visit https://www.netresec.com/?page=CapLoader to grab a copy and start filtering!

UPDATE 2016-05-23

With the release of CapLoader 1.4 it is now possible to apply Display Filters not only to the Flows tab, but also to the Services and Hosts tab.

Posted by Erik Hjelmvik on Monday, 30 November 2015 08:15:00 (UTC/GMT)

Tags: #CapLoader​ #BPF​ #PCAP​ #Berkeley Packet Filter​ #filter​ #IP​ #port​ #CIDR​

CapLoader 1.3 Released

A new version of our heavy-duty PCAP parser tool CapLoader is now available. There are many new features and improvements in this release, such as the ability to filter flows with BPF, domain name extraction via passive DNS parser and matching of domain names against a local white list.

Filtering with BPF

The main focus in the work behind CapLoader 1.3 has been to fully support the Rinse-Repeat Intrusion Detection methodology. We've done this by improving the filtering capabilities in CapLoader. For starters, we've added an input filter, which can be used to specify IP addresses, IP networks, protocols or port numbers to be parsed or ignored. The input filter uses the Berkeley Packet Filter (BPF) syntax, and is designed to run really fast. So if you wanna analyze only HTTP traffic you can simply write “port 80” as your input filter to have CapLoader only parse and display flows going to or from port 80. We have also added a display filter, which unlike Wireshark also uses BPF. Thus, once a set of flows is loaded one can easily apply different display filters, like “host 194.9.94.80” or “net 192.168.1.0/24”, to apply different views on the parsed data.

The main differences between the input filter and display filter are:

• Input filter is much faster than the display filter, so if you know beforehand what ports, protocols or IP addresses you are interested in then make sure to apply them as an input filter. You will notice a delay when applying a display filter to a view of 10.000 flows or more.
• In order to apply a new input filter CapLoader has to reload all the opened PCAP files (which is done by pressing F5). Modifying display filters, on the other hand, only requires you to press Enter or hit the “Apply” button.
• Previously applied display filters are accessible in a drop-down menu in the GUI, but no history is kept of previous input filters.

NetFlow + DNS == true

The “Flows” view in CapLoader gives a great overview of all TCP, UDP and SCTP flows in the loaded PCAP files. However, it is usually not obvious to an analyst what every IP address is used for. We have therefore added a DNS parser to CapLoader, so that all DNS packets can be parsed in order to map IP addresses to domain names. The extracted domain names are displayed for each flow, which is very useful when performing Rinse-Repeat analysis in order to quickly remove “known good servers” from the analysis.

Leveraging the Alexa top 1M list

As we've show in in our previous blog post “DNS whitelisting in NetworkMiner”, using a list of popular domain names as a whitelist can be an effective method for finding malware. We often use this approach in order to quickly remove lots of known good servers when doing Rinse-Repeat analysis in large datasets.

Therefore, just as we did for NetworkMiner 1.5, CapLoader now includes Alexa's list of the 1 million most popular domain names on the Internet. All domain names, parsed from DNS traffic, are checked against the Alexa list. Domains listed in the whitelist are shown in CapLoader's “Server_Alexa_Domian” column. This makes it very easy to sort on this column in order to remove (hide) all flows going to “normal” servers on the Internet. After removing all those flows, what you're left with is pretty much just:

• Local traffic (not sent over the Internet)
• Outgoing traffic to either new or obscure domains

Manually going through the remaining flows can be very rewarding, as it can reveal C2 traffic from malware that has not yet been detected by traditional security products like anti-virus or IDS.

Many new features in CapLoader 1.3

The new features highlighted above are far from the only additions made to CapLoader 1.3. Here is a more complete list of improvements in this release:

• Support for “Select Flows in PCAP” to extract and select 5-tuples from a PCAP-file. This can be a Snort PCAP with packets that have triggered IDS signatures. This way you can easily extract the whole TCP or UDP flow for each signature match, instead of just trying to make sense of one single packet per alert.
• Improved packet carver functionality to better carve IP, TCP and UPD packets from any file. This includes memory dumps as well as proprietary and obscure packet capture formats.
• Support for SCTP flows.
• DNS parser.
• Alexa top 1M matching.
• Input filter and display filter with BPF syntax.
• Flow Producer-Consumer-Ratio PCR.
• Flow Transcript can be opened simply by double-clicking a flow.
• Find form updated with option to hide non-matching flows instead of just selecting the flows that matched the keyword search criteria.
• New flow transcript encoding with IP TTL, TCP flags and sequence numbers to support analysis of Man-on-the-Side attacks.
• Faster loading of previously opened files, MD5 hashes don't need to be recalculated.
• A selected set of flows in the GUI can be inverted simply by right-clicking the flow list and selecting “Invert Selection” or by hitting Ctrl+I.

Downloading CapLoader 1.3

All these new features, except for the Alexa lookup of domain names, are available in our free trial version of CapLoader. So to try out these new features in CapLoader, simply grab a trial download here:
https://www.netresec.com/?page=CapLoader#trial (no registration needed)

All paying customers with an older version of CapLoader can grab a free update for version 1.3 at our customer portal.

Posted by Erik Hjelmvik on Monday, 28 September 2015 07:30:00 (UTC/GMT)

Tags: #CapLoader​ #BPF​ #Berkeley Packet Filter​ #Rinse-Repeat​ #DNS​ #Alexa​ #PCAP​ #Passive DNS​ #NetFlow​ #Malware​ #C2​