Sniffing Decrypted TLS Traffic with Security Onion

Wouldn't it be awesome to have a NIDS like Snort, Suricata or Zeek inspect HTTP requests leaving your network inside TLS encrypted HTTPS traffic? Yeah, we think so too! We have therefore created this guide on how to configure Security Onion to sniff decrypted TLS traffic with help of PolarProxy.

PolarProxy is a forward TLS proxy that decrypts incoming TLS traffic from clients, re-encrypts it and forwards it to the server. One of the key features in PolarProxy is the ability to export the proxied traffic in decrypted form using the PCAP format (a.k.a. libpcap/tcpdump format). This makes it possible to read the decrypted traffic with external tools, without having to perform the decryption again. It also enables packet analysis using tools that don't have built-in TLS decryption support.

This guide outlines how to configure PolarProxy to intercept HTTPS traffic and send the decrypted HTTP traffic to an internal network interface, where it can be sniffed by an IDS.

STEP 1 ☆ Install Ubuntu

Download and install the latest SecurityOnion ISO image, but don't run the "Setup" just yet.

STEP 2 ☆ Add a Dummy Network Interface

Add a dummy network interface called "decrypted", to which decrypted packets will be sent.

STEP 3 ☆ Install Updates

Install updates in Security Onion by running "sudo soup".

STEP 4 ☆ Run the Security Onion Setup

Run the Security Onion setup utility by double-clicking the "Setup" desktop shortcut or executing "sudo sosetup" from a terminal. Follow the setup steps in the Production Deployment documentation and select "decrypted" as your sniffing interface.

Reboot and run Setup again to continue with the second phase of Security Onion's setup. Again, select "decrypted" as the interface to be monitored.

STEP 5 ☆ Install PolarProxy Service

Download and install PolarProxy:

Edit /etc/systemd/system/PolarProxy.service and add "--pcapoverip 57012" at the end of the ExecStart command.

Start the PolarProxy systemd service:

STEP 6 ☆ Install Tcpreplay Service

The decrypted traffic can now be accessed via PolarProxy's PCAP-over-IP service on TCP 57012. We can leverage tcpreplay and netcat to replay these packets to our dummy network interface in order to have them picked up by Security Onion.

Start the tcpreplay systemd service:

STEP 7 ☆ Add firewall rules

Security Onion only accepts incoming connections on TCP 22 by default, we also need to allow connections to TCP port 10443 (proxy port), and 10080 (root CA certificate download web server). Add allow rules for these services to the Security Onion machine's firewall:

Verify that the proxy is working by running this curl command on a PC connected to the same network as the Security Onion machine:

Add the following lines at the top of /etc/ufw/before.rules (before the *filter section) to redirect incoming packets on TCP 443 to PolarProxy on port 10443.

Note: Replace "enp0s3" with the Security Onion interface to which clients will connect.

After saving before.rules, reload ufw to activate the port redirection:

Verify that you can reach the proxy on TCP 443 with this command:

STEP 8 ☆ Redirect HTTPS traffic to PolarProxy

It's now time to configure a client to run its HTTPS traffic through PolarProxy. Download and install the PolarProxy X.509 root CA certificate from PolarProxy's web service on TCP port 10080:

http://[SecurityOnionIP]:10080/polarproxy.cer

Install the certificate in the operating system and browser, as instructed in the PolarProxy documentation.

You also need to forward packets from the client machine to the Security Onion machine running PolarProxy. This can be done either by configuring a local NAT rule on each monitored client (STEP 8.a) or by configuring the default gateway's firewall to forward HTTPS traffic from all clients to the proxy (STEP 8.b).

STEP 8.a ☆ Local NAT

Use this firewall rule on a Linux client to configure it to forward outgoing HTTPS traffic to the Security Onion machine:

STEP 8.b ☆ Global NAT

If the client isn't running Linux, or if you wanna forward HTTPS traffic from a whole network to the proxy, then apply the following iptables rules to the firewall in front of the client network. See "Routing Option #2" in the PolarProxy documentation for more details.

1. Add a forward rule on the gateway to allow forwarding traffic to our PolarProxy server:
   sudo iptables -A FORWARD -i eth1 -d [SecurityOnionIP] -p tcp --dport 10443 -m state --state NEW -j ACCEPT
2. Add a DNAT rule to forward 443 traffic to PolarProxy on port 10443:
   sudo iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 443 -j DNAT --to [SecurityOnionIP]:10443
3. If the reverse traffic from PolarProxy to the client doesn't pass the firewall (i.e. they are on the same LAN), then we must add this hide-nat rule to fool PolarProxy that we are coming from the firewall:
   sudo iptables -t nat -A POSTROUTING -o eth1 -d [SecurityOnionIP] -p tcp --dport 10443 -j MASQUERADE

STEP 9 ☆ Inspect traffic in SecurityOnion

Wait for the Elastic stack to initialize, so that the intercepted network traffic becomes available through the Kibana GUI. You can check the status of the elastic initialization with "sudo so-elastic-status".

You should now be able to inspect decrypted traffic in Security Onion using Kibana, Squert, Sguil etc., just as if it was unencrypted HTTP.

Image: Kibana showing HTTP traffic info from decrypted HTTPS sessions

Image: MIME types in Kibana

Image: NIDS alerts from payload in decrypted traffic shown in Kibana

Image: Snort alerts from decrypted traffic shown in Squert

Security Considerations and Hardening

Security Onion nodes are normally configured to only allow access by SOC/CERT/CSIRT analysts, but the setup described in this blog post requires that "normal" users on the client network can access the PolarProxy service running on the Security Onion node. We therefore recommend installing PolarProxy on a dedicated Security Onion Forward Node, which is configured to only monitor traffic from the proxy.

We also recommend segmenting the client network from the analyst network, for example by using separate network interfaces on the Security Onion machine or putting it in a DMZ. Only the PolarProxy service (TCP 10080 and 10443) should be accessable from the client network.

PolarProxy could be used to pivot from the client network into the analyst network or to access the Apache webserver running on the Security Onion node. For example, the following curl command can be used to access the local Apache server running on the Security Onion machine via PolarProxy:

Hardening Steps:

• Configure the Security Onion node as a Forward Node
• Segment client network from analyst network
• Add firewall rules to prevent PolarProxy from accessing services on the local machine and analyst network

For additional info on hardening, please see the recommendations provided by Wes Lambert on the Security-Onion mailing list.

Posted by Erik Hjelmvik on Monday, 20 January 2020 09:40:00 (UTC/GMT)

Tags: #SecurityOnion​ #Security Onion​ #PCAP​ #Bro​ #Zeek​ #PolarProxy​ #Snort​ #Suricata​ #TLS​ #SSL​ #HTTPS​ #tcpreplay​ #PCAP-over-IP​ #IDS​ #NIDS​ #netcat​ #curl​ #UFW​ #ASCII-art​

Reverse Engineering Proprietary ICS Protocols

One of the highlights at this year’s SEC-T conference in Stockholm was Steve Miller’s talk titled "Reversing the TriStation Network Protocol". In this talk Steve covered his quest to better understand the TRITON malware, which had been used in a targeted attack of an industrial control system (ICS). Steve didn’t disclose the type or location of the plant, saying “Don’t ask me who it was, ‘cause I can’t say” when the Q&A started. However, an article in the Wall Street Journal points out that it was a petrochemical plant in Saudi Arabia that had been hacked.

Targeting Safety Instrumented System

The TRITON malware (also called TRISIS) was used to target a safety instrumented system (SIS) from Schneider Electric called Triconex. A SIS is typically not used to control the process of a plant, but rather to detect abnormal operating conditions and safely shut down the industrial process if needed.

I could elaborate a lot regarding the consequences of attacking the SIS, but the good guys from Dragos have already done a great job explaining this in their “TRISIS Malware” report.

Reverse Engineering the ICS Protocol

The communication protocol used by the Triconex controllers is called TriStation, which is a proprietary protocol. This means that there were no publicly available specifications available for the protocol at that time. There was also no Wireshark dissector that could parse TriStation traffic. Nevertheless, Steve’s initial reaction to this was “Awesome, undocumented things are my favorite things!”

Unfortunately Steve wasn’t able to get hold of a single PCAP file with the TriStation network protocol, which made it really difficult to reverse engineer the protocol implementation in the TRITON malware. The only piece of actual TriStation network traffic he was able to get hold of was a hex dump of a TriStation packet in an academic paper.

Armed with only the hexdump and Wireshark’s text2pcap Steve managed to piece together an actual PCAP file containing a single frame with a TriStation packet inside.

As you can see in the image above, Wireshark doesn’t decode any of the application layer data coming from TCP port 1502 (which TriStation uses). He therefore implemented a Wireshark Lua dissector for the TriStation protocol. And some time later the people from Nozomi Networks even implemented a proper Wireshark dissector for the TriStation protocol.

BSI’s ICS-SEC team have now also created Snort IDS rules specifically for the TriStation protocol. These IDS rules trigger on events like:

• Packets sent to the controller from an unauthorized host
• Malicious commands used by the TRITON malware to read and write to the RAM of the SIS controller as well as to execute code

The Importance of Sniffing ICS Traffic

I’ve been trying to convince asset owners, who use ICS in their power plants, factories, water treatment facilities etc, to start capturing the network traffic and storing it as PCAP files for many years now. However, asset owners sometimes try to argue that there is no point in capturing their traffic since it is using a proprietary protocol. Even Ralph Langner has opposed to the idea of capturing ICS network traffic in a blog post, which I have criticized. So, how difficult is it to write a parser for a proprietary protocol?

I have personally implemented support for over 30 application layer protocols in NetworkMiner, but unlike Steve I’ve always had access to at least one PCAP file and some form of documentation of the protocol. However, I’ve found that many real-world protocol implementations don’t follow specifications properly. In these cases I’ve found that having access to PCAP files with real-world network traffic is more important than having a full protocol specification.

Even complex proprietary protocols like the old proprietary Skype protocol has been reverse engineered, so with access to network traffic of a protocol combined with a binary that uses this protocol I’d say that pretty much any network protocol can be reverse engineered.

Steve’s SEC-T talk also proves that ICS protocols are no different, since they too can be reverse engineered without having a protocol specification or RFC.

Capturing network traffic in ICS networks is never wrong. There might not be parsers available today for all the protocols you’re using. But once a parser or IDS signature becomes available for the protocol you’re using, you can simply use that to analyze previously captured network traffic from your ICS network. Also, in the wake of an incident you might actually end up writing a parser (as in the TRITON case) or a custom IDS rule, in which case having historical network traffic from your plant in invaluable!

For more information on this topic I’d suggest reading my blog post titled “Monitor those Control System Networks!” from 2011, which still is highly relevant.

I’m also happy to announce that two PCAP files containing TriStation network traffic have been linked from our list of publicly accessible PCAP files today (see the “SCADA/ICS Network Captures” section).

And remember: PCAP or it didn’t happen!

Posted by Erik Hjelmvik on Friday, 21 September 2018 14:20:00 (UTC/GMT)

Tags: #ICS​ #PCAP​ #SCADA​ #SEC-T​ #protocol​ #Wireshark​ #Snort​

Detecting the Pony Trojan with RegEx using CapLoader

This short video demonstrates how you can search through PCAP files with regular expressions (regex) using CapLoader and how this can be leveraged in order to improve IDS signatures.

The EmergingThreats snort/suricata rule mentioned in the video is SID 2014411 “ET TROJAN Fareit/Pony Downloader Checkin 2”.

The header accept-encoding header with quality factor 0 used by the Pony malware is:
Accept-Encoding: identity, *;q=0

And here is the regular expression used to search for that exact header: \r\nAccept-Encoding: identity, \*;q=0\r\n

After recording the video I noticed that the leaked source code for Pony 2.0 actually contains this accept-encoding header as a hard-coded string. Have a look in the redirect.php file, where they set curl’s CURLOPT_HTTPHEADER to this specific string.

Wanna learn more about the intended use of quality factors in HTTP accept headers? Then have a look at section 14.1 of RFC 2616section 5.3.4 of RFC 7231, which defines how to use qvalues (i.e. quality factors) in the Accept-Encoding header.

Finally, I'd like to thank Brad Duncan for running the malware-traffic-analysis.net website, your PCAP files often come in handy!

Update 2018-07-05

I submitted a snort/suricata signature to the Emerging-Sigs mailinglist after publishing this blog post, which resulted in the Emerging Threats signature 2014411 being updated on that same day to include:

Thank you @EmergingThreats for the fast turnaround!

Posted by Erik Hjelmvik on Wednesday, 04 July 2018 07:39:00 (UTC/GMT)

Tags: #video​ #regex​ #malware​ #IDS​ #curl​ #malware-traffic-analysis.net​ #videotutorial​