Detecting Periodic Flows with CapLoader 1.4

I am happy to announce a new release of our super-fast PCAP handling tool CapLoader! One of the new features in CapLoader makes it even easier to detect malicious network traffic without having to rely on blacklists, such as IDS signatures.

The new version of CapLoader includes new features such as:

• Services Tab (more details below)
• Input filter to limit number of parsed frames
• Flow Transcript in Hosts and Services tabs
• Keyword filtering
• Full filtering capability for all tabs
• Wireshark style coloring of flows, services and hosts

Services Tab

The biggest addition to version 1.4 of CapLoader is the Services tab, which presents a somewhat new way of aggregating the flows found in a PCAP file. Each row (or “service”) in the services tab represents a unique combination of <Client-IP, Server-IP, Server-port and Transport-protocol>. This means that if a single host makes multiple DNS requests to 8.8.8.8, then all those flows will be merged together as one row in the services tab.

This view makes it easy to see if a host is frequently accessing a particular network service. CapLoader even shows if the requests are made with regular intervals, in which case we measure the regularity and determine the most likely period between connections. The idea for measuring regularity comes from Sebastian Garcia's Stratosphere IPS, which can identify botnets by analyzing the periodicity of flows going to a C2 server.

Malware Example: Kovter.B

Here's what the Services tab looks like when loading 500 MB of PCAP files from a network where one of the hosts has been infected with malware (Win32/Kovter.B).

The services in the screenshot are sorted on the “Regularity” column, so that the most periodic ones are shown at the top. Services with a regularity value greater than 20 can be treated as periodic. In our case we see the top two services having a regularity of 36.9 with an estimated period of roughly 6h 2min. We can visualize the periodic behavior by opening the flows for those two services in a new instance if CapLoader. To do this, simply select the two services' rows, right-click the PCAP icon (in the top-right corner) and select “Open With > CapLoader 1.4.0.0”

As you can see in the flows tab, these services are accessed by the client on a regular interval of about 6h 2min. Doing a flow transcript of one such flow additionally reveals that the payload seems suspicious (not HTTP on TCP 80).

Image: Kovter.B malware trying to communicate with a C2 server

The Kovter malware failed to reach the C2 server in the attempt above, but there is a successful connection going to a C2 server at 12.25.99.131 every 3'rd hour (see service number 8 in the list of the most periodically accessed services). Here's a flow transcript of one such beacon:

Image: Kovter.B malware talking to C2 server at 12.25.99.131

Legitimate Periodic Services

Seven out of the 10 most periodically accessed services are actually caused by the Kovter malware trying to reach various C2 servers. The three most periodically accessed services that aren't malicious are:

• Service #3 is a legitimate Microsoft service (SeaPort connecting to toolbar.search.msn.com.akadns.net)
• Service #5 is a mail client connecting to the local POP3 server every 30 minutes.
• Service #6 is Microsoft-CryptoAPI updating its Certificate Revocation List from crl.microsoft.com every 5 hours.

Signature-Free Intrusion Detection

As shown in this blog post, analyzing the regularity of services is an efficient way of detecting C2 beacons without having to rely on IDS signatures. This method goes hand-in-hand with our Rinse-Repeat Intrusion Detection approach, which can be used to find malicous network traffic simply by ignoring traffic that seems “normal”.

Credits

Several bugs have been fixed in CapLoader 1.4, such as:

• Support for frames with Captured Length > Real Lenght (Thanks to Dietrich Hasselhorn for finding this bug)
• Delete key is no longer hijacked by the “Hide Selected Flows” button (Thanks to Dominik Andreansky for finding this bug).
• CapLoader GUI now looks okay even with graphics are scaled through "custom sizing". Thanks to Roland Wagner for finding this.

Downloading CapLoader 1.4

The regularity and period detection is available in our free trial version of CapLoader. To try it out simply grab a copy here:
https://www.netresec.com/?page=CapLoader#trial (no registration needed)

All paying customers with an older version of CapLoader can grab a free update to version 1.4 at our customer portal.

UPDATE June 2, 2016

We're happy to announce that it is now possible to detect Kovter's C2 communication with help of an IDS signature thanks to Edward Fjellskål. Edward shared his IDS signature "NT TROJAN Downloader/Malware/ClickFraud.Win32.Kovter Client CnC Traffic" on the Emerging-Sigs mailing list yesterday. We have worked with Edward on this and the signature has been verified on our Kovter C2 dataset.

UPDATE June 8, 2016

Edward Fjellskål's IDS signature "ET TROJAN Win32.Kovter Client CnC Traffic" has now been published as an Emerging Threats open rule with SID 2022861.

#alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:"ET TROJAN Win32.Kovter Client CnC? Traffic"; flow:established,to_server; dsize:4<>256; content:!"HTTP"; content:"|00 00 00|"; offset:1; depth:3; pcre:"/^[\x11\x21-\x26\x41\x45\x70-\x79]/R"; content:!"|00 00|"; distance:0; byte_jump:1,0,from_beginning,post_offset 3; isdataat:!2,relative; pcre:!"/\x00$/"; reference:url,symantec.com/connect/blogs/kovter-malware-learns-poweliks-persistent-fileless-registry-update; classtype:trojan-activity; sid:2022861; rev:1;)

Posted by Erik Hjelmvik on Monday, 23 May 2016 11:55:00 (UTC/GMT)

Tags: #CapLoader​ #PCAP​ #Periodicity​ #C2​ #beacon​ #Intrusion Detection​

Rinse-Repeat Intrusion Detection

I am a long time skeptic when it comes to blacklists and other forms of signature based detection mechanisms. The information security industry has also declared the signature based anti-virus approach dead several times during the past 10 years. Yet, we still rely on anti-virus signatures, IDS rules, IP blacklists, malware domain lists, YARA rules etc. to detect malware infections and other forms of intrusions in our networks. This outdated approach puts a high administrative burden on IT and security operations today, since we need to keep all our signature databases up to date, both when it comes to end point AV signatures as well as IDS rules and other signature based detection methods and threat feeds. Many organizations probably spend more time and money on updating all these blacklists and signature databases than actually investigating the security alerts these detection systems generate. What can I say; the world is truly upside down...

I would therefore like to use this blog post to briefly describe an effective blacklist-free approach for detecting malware and intrusions just by analyzing network traffic. My approach relies on a combination of whitelisting and common sense anomaly detection (i.e. not the academic statistical anomaly detection algorithms that never seem to work in reality). I also encourage CERT/CSIRT/SOC/SecOps units to practice Sun Tzu's old ”know yourself”, or rather ”know your systems and networks” approach.

Know your enemy and know yourself and you can fight a hundred battles without disaster.

My method doesn't rely on any dark magic, it is actually just a simple Rinse-Repeat approach built on the following steps:

After looping through these steps a few times you'll be left with some odd network traffic, which will have a high ratio of maliciousness. The key here is, of course, to know what traffic to classify as ”normal”. This is where ”know your systems and networks” comes in.

What Traffic is Normal?

I recently realized that Mike Poor seems to be thinking along the same lines, when I read his foreword to Chris Sanders' and Jason Smith's book Applied NSM:

The next time you are at your console, review some logs. You might think... "I don't know what to look for". Start with what you know, understand, and don't care about. Discard those. Everything else is of interest.

Following Mike's advice we might, for example, define“normal” traffic as:

• HTTP(S) traffic to popular web servers on the Internet on standard ports (TCP 80 and 443).
• SMB traffic between client networks and file servers.
• DNS queries from clients to your name server on UDP 53, where the servers successfully answers with an A, AAAA, CNAME, MX, NS or SOA record.
• ...any other traffic which is normal in your organization.

Whitelisting IP ranges belonging to Google, Facebook, Microsoft and Akamai as ”popular web servers” will reduce the dataset a great deal, but that's far from enough. One approach we use is to perform DNS whitelisting by classifying all servers with a domain name listed in Alexa's Top 1 Million list as ”popular”.

You might argue that such a method just replaces the old blacklist-updating-problem with a new whitelist-updating-problem. Well yes, you are right to some extent, but the good part is that the whitelist changes very little over time compared to a blacklist. So you don't need to update very often. Another great benefit is that the whitelist/rinse-repeat approach also enables detection of 0-day exploits and C2 traffic of unknown malware, since we aren't looking for known badness – just odd traffic.

Threat Hunting with Rinse-Repeat

Mike Poor isn't the only well merited incident handler who seems to have adopted a strategy similar to the Rinse-Repeat method; Richard Bejtlich (former US Air Force CERT and GE CIRT member) reveal some valuable insight in his book “The Practice of Network Security Monitoring”:

I often use Argus with Racluster to quickly search a large collection of session data via the command line, especially for unexpected entries. Rather than searching for specific data, I tell Argus what to omit, and then I review what’s left.

In his book Richard also mentions that he uses a similar methodology when going on “hunting trips” (i.e. actively looking for intrusions without having received an IDS alert):

Sometimes I hunt for traffic by telling Wireshark what to ignore so that I can examine what’s left behind. I start with a simple filter, review the results, add another filter, review the results, and so on until I’m left with a small amount of traffic to analyze.

I personally find Rinse-Repeat Intrusion Detection ideal for threat hunting, especially in situations where you are provided with a big PCAP dataset to answer the classic question “Have we been hacked?”. However, unfortunately the “blacklist mentality” is so conditioned among incident responders that they often choose to crunch these datasets through blacklists and signature databases in order to then review thousands of alerts, which are full of false positives. In most situations such approaches are just a huge waste of time and computing power, and I'm hoping to see a change in the incident responders' mindsets in the future.

I teach this “rinse-repeat” threat hunting method in our Network Forensics Training. In this class students get hands-on experience with a dataset of 3.5 GB / 40.000 flows, which is then reduced to just a fraction through a few iterations in the rinse-repeat loop. The remaining part of the PCAP dataset has a very high ratio of hacking attacks as well as command-and-control traffic from RAT's, backdoors and botnets.

UPDATE 2015-10-07

We have now published a blog post detailing how to use dynamic protocol detection to identify services running on non-standard ports. This is a good example on how to put the Rinse-Repeat methodology into practice.

Posted by Erik Hjelmvik on Monday, 17 August 2015 08:45:00 (UTC/GMT)

Tags: #Rinse-Repeat​ #PCAP​ #NSM​ #PCAP​ #Intrusion Detection​ #IDS​ #network forensics​